anthology-of-the-sota

❯

❯

literature.d

Graph View

737 items under this folder.

Oct 06, 2026
LIT-799: Olmo Hybrid: From Theory to Practice and Back
Oct 06, 2026
LIT-800: Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
Oct 06, 2026
LIT-801: Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context Extension
Oct 06, 2026
LIT-802: Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention
- attention-techniques
- model-architecture
Oct 03, 2026
LIT-771: Glow: Generative Flow with Invertible 1x1 Convolutions
Oct 03, 2026
LIT-772: Neural Ordinary Differential Equations
Oct 03, 2026
LIT-773: Improving Video Generation with Human Feedback
Oct 03, 2026
LIT-774: Diffusion Model Alignment Using Direct Preference Optimization
- adaptation-and-tuning
- generative-modeling
Oct 03, 2026
LIT-775: Inductive Moment Matching
Oct 03, 2026
LIT-776: Training Diffusion Models with Reinforcement Learning
Oct 03, 2026
LIT-777: Flow Matching Guide and Code
- generative-modeling
- flows-and-transport
Oct 03, 2026
LIT-778: DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models
- adaptation-and-tuning
- generative-modeling
Oct 03, 2026
LIT-779: Flow-GRPO: Training Flow Matching Models via Online RL
Oct 03, 2026
LIT-780: FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
- generative-modeling
- flows-and-transport
Oct 03, 2026
LIT-781: Density estimation using Real NVP
Oct 03, 2026
LIT-782: Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation
Oct 03, 2026
LIT-783: Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency
Oct 03, 2026
LIT-784: Mean Flows for One-step Generative Modeling
Oct 03, 2026
LIT-785: STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
Oct 03, 2026
LIT-786: Improved Techniques for Training Consistency Models
Oct 03, 2026
LIT-787: One Step Diffusion via Shortcut Models
Oct 03, 2026
LIT-788: ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation
Oct 03, 2026
LIT-789: Multisample Flow Matching: Straightening Flows with Minibatch Couplings
Oct 03, 2026
LIT-790: Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models
Oct 03, 2026
LIT-791: Improving the Training of Rectified Flows
Oct 03, 2026
LIT-792: Discrete Flow Matching
Oct 03, 2026
LIT-793: Flow map matching with stochastic interpolants: A mathematical framework for consistency models
Oct 03, 2026
LIT-794: Normalizing Flows are Capable Generative Models
Oct 03, 2026
LIT-795: Scaling Laws for Reward Model Overoptimization
Oct 03, 2026
LIT-796: InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation
Oct 03, 2026
LIT-797: Aligning Text-to-Image Models using Human Feedback
Oct 03, 2026
LIT-798: Improving and generalizing flow-based generative models with minibatch optimal transport
Oct 02, 2026
LIT-770: DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
Oct 01, 2026
LIT-733: Language Models are Unsupervised Multitask Learners
Oct 01, 2026
LIT-734: Fine-Tuning Language Models from Human Preferences
- adaptation-and-tuning
- analysis-and-evaluation
Oct 01, 2026
LIT-735: Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads
- distributed-optimization
- systems-optimization
Oct 01, 2026
LIT-736: Reducing Activation Recomputation in Large Transformer Models
- distributed-optimization
- systems-optimization
Oct 01, 2026
LIT-737: 10Cache: Heterogeneous Resource-Aware Tensor Caching and Migration for LLM Training
- distributed-optimization
- systems-optimization
Oct 01, 2026
LIT-738: Shortformer: Better Language Modeling Using Shorter Inputs
- training-optimization
- attention-techniques
Oct 01, 2026
LIT-739: TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Oct 01, 2026
LIT-740: Role-Play with Large Language Models
Oct 01, 2026
LIT-741: Textoshop: Interactions Inspired by Drawing Software to Facilitate Text Editing
Oct 01, 2026
LIT-742: The Computational Foundations of Collective Intelligence
- agents-and-environments
Oct 01, 2026
LIT-743: Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation
Oct 01, 2026
LIT-744: Text Embeddings by Weakly-Supervised Contrastive Pre-training
Oct 01, 2026
LIT-745: SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
Oct 01, 2026
LIT-746: Position: Curvature Matrices Should Be Democratized via Linear Operators
Oct 01, 2026
LIT-747: Perceiver: General Perception with Iterative Attention
Oct 01, 2026
LIT-748: SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
Oct 01, 2026
LIT-749: bert2BERT: Towards Reusable Pretrained Language Models
- training-optimization
- model-stability
Oct 01, 2026
LIT-750: ZipNN: Lossless Compression for AI Models
Oct 01, 2026
LIT-751: "Principal Components" Enable A New Language of Images
Oct 01, 2026
LIT-752: Position: AI/ML Influencers Have a Place in the Academic Process
- deployment-and-society
- analysis-and-evaluation
Oct 01, 2026
LIT-753: Rigorous dynamical mean field theory for stochastic gradient descent methods
- training-optimization
- analysis-and-evaluation
Oct 01, 2026
LIT-754: Efficient AlphaFold2 Training using Parallel Evoformer and Branch Parallelism
Oct 01, 2026
LIT-755: Wonderland: Navigating 3D Scenes from a Single Image
Oct 01, 2026
LIT-756: The Work Capacity of Channels with Memory: Maximum Extractable Work in Percept-Action Loops
- agents-and-environments
Oct 01, 2026
LIT-757: Promptagator: Few-shot Dense Retrieval From 8 Examples
Oct 01, 2026
LIT-758: Net2Net: Accelerating Learning via Knowledge Transfer
Oct 01, 2026
LIT-759: From monoliths to modules: Decomposing transducers for efficient world modelling
Oct 01, 2026
LIT-760: Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
- model-architecture
- attention-techniques
Oct 01, 2026
LIT-761: Synaptic Field Theory for Neural Networks
- analysis-and-evaluation
- training-optimization
Oct 01, 2026
LIT-762: Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models
- adaptation-and-tuning
- distributed-optimization
Oct 01, 2026
LIT-763: Erwin: A Tree-based Hierarchical Transformer for Large-scale Physical Systems
Oct 01, 2026
LIT-764: A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima
Oct 01, 2026
LIT-765: Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
- training-optimization
- model-stability
Oct 01, 2026
LIT-766: Community Detection on Networks with Ricci Flow
- graphs-and-networks
Oct 01, 2026
LIT-767: Understanding Deep Learning via Notions of Rank
Oct 01, 2026
LIT-768: Kronecker-factored Approximate Curvature (KFAC) From Scratch
- training-optimization
- analysis-and-evaluation
Oct 01, 2026
LIT-769: Structural Decoupling: A Scaffold-Flow Theory of Generalization and Alignment
Sep 27, 2026
LIT-732: Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
Sep 26, 2026
LIT-722: Score-Based Generative Modeling through Stochastic Differential Equations
Sep 26, 2026
LIT-723: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Sep 26, 2026
LIT-724: Investigating Sanity Checks for Saliency Maps with Image and Text Classification
- analysis-and-evaluation
- representation-and-encoding
Sep 26, 2026
LIT-725: On the (In)fidelity and Sensitivity for Explanations
- analysis-and-evaluation
- representation-and-encoding
Sep 26, 2026
LIT-726: Talagrand Meets Talagrand: Upper and Lower Bounds on Expected Soft Maxima of Gaussian Processes with Finite Index Sets
- attention-techniques
- analysis-and-evaluation
Sep 26, 2026
LIT-727: Striving for Simplicity: The All Convolutional Net
Sep 26, 2026
LIT-728: Visualizing and Understanding Convolutional Networks
Sep 26, 2026
LIT-729: When Explanations Lie: Why Many Modified BP Attributions Fail
Sep 26, 2026
LIT-730: A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations
Sep 26, 2026
LIT-731: SmoothGrad: removing noise by adding noise
- analysis-and-evaluation
- vision-and-graphics
Sep 25, 2026
LIT-665: Transformer Language Models without Positional Encodings Still Learn Positional Information
Sep 25, 2026
LIT-666: Sparse Distributed Memory
- model-architecture
- attention-techniques
Sep 25, 2026
LIT-667: Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
Sep 25, 2026
LIT-668: ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Sep 25, 2026
LIT-669: Sparse Distributed Memory and Related Models
- model-architecture
- analysis-and-evaluation
Sep 25, 2026
LIT-670: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Sep 25, 2026
LIT-671: RoBERTa: A Robustly Optimized BERT Pretraining Approach
Sep 25, 2026
LIT-672: Should You Mask 15% in Masked Language Modeling?
Sep 25, 2026
LIT-673: Averaging Weights Leads to Wider Optima and Better Generalization
Sep 25, 2026
LIT-674: Robust fine-tuning of zero-shot models
Sep 25, 2026
LIT-675: Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Sep 25, 2026
LIT-676: DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models
Sep 25, 2026
LIT-677: Diffusion Models With Learned Adaptive Noise
- generative-modeling
- training-optimization
Sep 25, 2026
LIT-678: Diffusion Meets Flow Matching: Two Sides of the Same Coin
Sep 25, 2026
LIT-679: Dynamic Chunking for End-to-End Hierarchical Sequence Modeling
Sep 25, 2026
LIT-680: Distilling the Knowledge in a Neural Network
Sep 25, 2026
LIT-681: Cramming: Training a Language Model on a Single GPU in One Day
Sep 25, 2026
LIT-682: ScheduleFree+: Scaling Learning-Rate-Free & Schedule-Free Learning to Large Language Models
- training-optimization
- model-stability
Sep 25, 2026
LIT-683: Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
Sep 25, 2026
LIT-684: Model Merging in Pre-training of Large Language Models
Sep 25, 2026
LIT-685: Sub-Scaling Laws: On the Role of Data Density and Training Strategies in LLMs
Sep 25, 2026
LIT-686: On the distance between two neural networks and the stability of learning
- training-optimization
- model-stability
Sep 25, 2026
LIT-687: Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think
Sep 25, 2026
LIT-688: Reconciling Kaplan and Chinchilla Scaling Laws
- training-optimization
- analysis-and-evaluation
Sep 25, 2026
LIT-689: Common Diffusion Noise Schedules and Sample Steps are Flawed
Sep 25, 2026
LIT-690: Resolving Discrepancies in Compute-Optimal Scaling of Language Models
- training-optimization
- analysis-and-evaluation
Sep 25, 2026
LIT-691: Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation
Sep 25, 2026
LIT-692: Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation
Sep 25, 2026
LIT-693: Classifier-Free Diffusion Guidance
Sep 25, 2026
LIT-694: Rigor with Machine Learning from Field Theory to the Poincaré Conjecture
- analysis-and-evaluation
Sep 25, 2026
LIT-695: Language Design as Information Renormalization
- signal-structure
Sep 25, 2026
LIT-696: Rarely categorical, highly separable representations along the cortical hierarchy
- concept-geometry
Sep 25, 2026
LIT-697: AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society
- deployment-and-society
- agents-and-environments
Sep 25, 2026
LIT-698: Topos and Stacks of Deep Neural Networks
- model-architecture
- concept-geometry
Sep 25, 2026
LIT-699: Diffusion Models Beat GANs on Image Synthesis
Sep 25, 2026
LIT-700: Non-Euclidean Gradient Descent Operates at the Edge of Stability
Sep 25, 2026
LIT-701: Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution
Sep 25, 2026
LIT-702: Simple and Effective Masked Diffusion Language Models
Sep 25, 2026
LIT-703: Improved Precision and Recall Metric for Assessing Generative Models
Sep 25, 2026
LIT-704: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
- model-stability
- analysis-and-evaluation
Sep 25, 2026
LIT-705: Understanding the Disharmony between Dropout and Batch Normalization by Variance Shift
- model-stability
- analysis-and-evaluation
Sep 25, 2026
LIT-706: Scaling Language-Image Pre-training via Masking
Sep 25, 2026
LIT-707: Analysis of dropout learning regarded as ensemble learning
- model-stability
Sep 25, 2026
LIT-708: A Controlled Study of Attention-Only Transformers
Sep 25, 2026
LIT-709: Primer: Searching for Efficient Transformers for Language Modeling
Sep 25, 2026
LIT-710: Efficient Diffusion Training via Min-SNR Weighting Strategy
- generative-modeling
- training-optimization
Sep 25, 2026
LIT-711: Do Transformer Modifications Transfer Across Implementations and Applications?
Sep 25, 2026
LIT-712: Axiomatic Attribution for Deep Networks
- analysis-and-evaluation
- vision-and-graphics
Sep 25, 2026
LIT-713: Sanity Checks for Saliency Maps
- analysis-and-evaluation
- vision-and-graphics
Sep 25, 2026
LIT-714: Analyzing and Improving the Training Dynamics of Diffusion Models
Sep 25, 2026
LIT-715: Surya: Foundation Model for Heliophysics
Sep 25, 2026
LIT-716: The Impact of Generative AI on Social Media: An Experimental Study
- deployment-and-society
Sep 25, 2026
LIT-717: Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task
- deployment-and-society
- human-ai-interaction
Sep 25, 2026
LIT-718: Automated decision-making as domination
- deployment-and-society
- analysis-and-evaluation
Sep 25, 2026
LIT-719: Discovering cognitive strategies with tiny recurrent neural networks
- analysis-and-evaluation
- tiny-models
Sep 25, 2026
LIT-720: Analyzing and Improving the Training Dynamics of Diffusion Models
Sep 25, 2026
LIT-721: Guiding a Diffusion Model with a Bad Version of Itself
Sep 24, 2026
LIT-617: Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Sep 24, 2026
LIT-618: LTX-Video: Realtime Video Latent Diffusion
Sep 24, 2026
LIT-619: Wan: Open and Advanced Large-Scale Video Generative Models
Sep 24, 2026
LIT-620: HunyuanVideo: A Systematic Framework For Large Video Generative Models
Sep 24, 2026
LIT-621: Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
Sep 24, 2026
LIT-622: CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
Sep 24, 2026
LIT-623: Language Model Beats Diffusion — Tokenizer is Key to Visual Generation
Sep 24, 2026
LIT-624: Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model
Sep 24, 2026
LIT-625: Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
Sep 24, 2026
LIT-626: Movie Gen: A Cast of Media Foundation Models
Sep 24, 2026
LIT-627: Video Diffusion Models
- generative-modeling
- vision-and-graphics
Sep 24, 2026
LIT-628: Better plain ViT baselines for ImageNet-1k
Sep 24, 2026
LIT-629: Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion
Sep 24, 2026
LIT-630: Flow Matching for Generative Modeling
- generative-modeling
- flows-and-transport
Sep 24, 2026
LIT-631: From Slow Bidirectional to Fast Autoregressive Video Diffusion Models
Sep 24, 2026
LIT-632: Make-A-Video: Text-to-Video Generation without Text-Video Data
Sep 24, 2026
LIT-633: AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
Sep 24, 2026
LIT-634: Open-Sora 2.0: Training a Commercial-Level Video Generation Model in $200k
Sep 24, 2026
LIT-635: Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioning
- generative-modeling
- vision-and-graphics
Sep 24, 2026
LIT-636: Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
Sep 24, 2026
LIT-637: Imagen Video: High Definition Video Generation with Diffusion Models
Sep 24, 2026
LIT-638: A Length-Extrapolatable Transformer
Sep 24, 2026
LIT-639: ResiDual: Transformer with Dual Residual Connections
Sep 24, 2026
LIT-640: Query-Key Normalization for Transformers
Sep 24, 2026
LIT-641: Attention Approximates Sparse Distributed Memory
Sep 24, 2026
LIT-642: Professor Forcing: A New Algorithm for Training Recurrent Networks
- training-optimization
- generative-modeling
Sep 24, 2026
LIT-643: One-step Diffusion with Distribution Matching Distillation
Sep 24, 2026
LIT-644: Building Normalizing Flows with Stochastic Interpolants
Sep 24, 2026
LIT-645: Stochastic Interpolants: A Unifying Framework for Flows and Diffusions
Sep 24, 2026
LIT-646: Improved Distribution Matching Distillation for Fast Image Synthesis
Sep 24, 2026
LIT-647: Sequence Level Training with Recurrent Neural Networks
Sep 24, 2026
LIT-648: Improving Image Generation with Better Captions
Sep 24, 2026
LIT-649: Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
- training-optimization
- generative-modeling
Sep 24, 2026
LIT-650: Competition-Level Code Generation with AlphaCode
Sep 24, 2026
LIT-651: Transformers without Tears: Improving the Normalization of Self-Attention
Sep 24, 2026
LIT-652: Video generation models as world simulators
Sep 24, 2026
LIT-653: Softmax is not Enough (for Sharp Size Generalisation)
Sep 24, 2026
LIT-654: Muon Outperforms Adam in Tail-End Associative Memory Learning
Sep 24, 2026
LIT-655: Self-attention Does Not Need O(n²) Memory
Sep 24, 2026
LIT-656: Intriguing Properties of Quantization at Scale
Sep 24, 2026
LIT-657: Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution
Sep 24, 2026
LIT-658: FiT: Flexible Vision Transformer for Diffusion Model
Sep 24, 2026
LIT-659: On the Importance of Noise Scheduling for Diffusion Models
- generative-modeling
- signal-structure
Sep 24, 2026
LIT-660: simple diffusion: End-to-end diffusion for high resolution images
Sep 24, 2026
LIT-661: Photorealistic Video Generation with Diffusion Models
Sep 24, 2026
LIT-662: Vision Transformers Need Registers
Sep 24, 2026
LIT-663: DINOv3
Sep 24, 2026
LIT-664: Emerging Properties in Self-Supervised Vision Transformers
Sep 23, 2026
LIT-550: Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
Sep 23, 2026
LIT-551: Implicit Neural Representations with Periodic Activation Functions
Sep 23, 2026
LIT-552: XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models
- inference-optimization
- systems-optimization
Sep 23, 2026
LIT-553: Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise
- generative-modeling
- analysis-and-evaluation
Sep 23, 2026
LIT-554: Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion
- generative-modeling
- vision-and-graphics
Sep 23, 2026
LIT-555: Warm Diffusion: Recipe for Blur-Noise Mixture Diffusion Models
Sep 23, 2026
LIT-556: Tensor Programs III: Neural Matrix Laws
Sep 23, 2026
LIT-557: Tensor Programs II: Neural Tangent Kernel for Any Architecture
Sep 23, 2026
LIT-558: Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes
Sep 23, 2026
LIT-559: Alias-Free Generative Adversarial Networks
Sep 23, 2026
LIT-560: Analyzing and Improving the Image Quality of StyleGAN
Sep 23, 2026
LIT-561: A Style-Based Generator Architecture for Generative Adversarial Networks
Sep 23, 2026
LIT-562: Projected GANs Converge Faster
Sep 23, 2026
LIT-563: The Role of ImageNet Classes in Fréchet Inception Distance
- analysis-and-evaluation
- generative-modeling
Sep 23, 2026
LIT-564: Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
- generative-modeling
- vision-and-graphics
Sep 23, 2026
LIT-565: FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmark
Sep 23, 2026
LIT-566: SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
- generative-modeling
- vision-and-graphics
Sep 23, 2026
LIT-567: MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
Sep 23, 2026
LIT-568: Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
- analysis-and-evaluation
- concept-geometry
Sep 23, 2026
LIT-569: Eliciting Latent Predictions from Transformers with the Tuned Lens
- analysis-and-evaluation
Sep 23, 2026
LIT-570: interpreting GPT: the logit lens
- analysis-and-evaluation
Sep 23, 2026
LIT-571: Scaling and evaluating sparse autoencoders
- analysis-and-evaluation
- concept-geometry
Sep 23, 2026
LIT-572: FLUX.2: Analyzing and Enhancing the Latent Space of FLUX – Representation Comparison
Sep 23, 2026
LIT-573: FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space
Sep 23, 2026
LIT-574: Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models
- analysis-and-evaluation
- adaptation-and-tuning
Sep 23, 2026
LIT-575: Transformer Feed-Forward Layers Are Key-Value Memories
- analysis-and-evaluation
- model-architecture
Sep 23, 2026
LIT-576: Mass-Editing Memory in a Transformer
- adaptation-and-tuning
- analysis-and-evaluation
Sep 23, 2026
LIT-577: Evaluating the Ripple Effects of Knowledge Editing in Language Models
Sep 23, 2026
LIT-578: Locating and Editing Factual Associations in GPT
- analysis-and-evaluation
- adaptation-and-tuning
Sep 23, 2026
LIT-579: Where Did the Gap Go? Reassessing the Long-Range Graph Benchmark
Sep 23, 2026
LIT-580: How Powerful are Graph Neural Networks?
Sep 23, 2026
LIT-581: Pitfalls of Graph Neural Network Evaluation
Sep 23, 2026
LIT-582: Semi-Supervised Classification with Graph Convolutional Networks
- graphs-and-networks
- model-architecture
Sep 23, 2026
LIT-583: Highly accurate protein structure prediction with AlphaFold
Sep 23, 2026
LIT-584: Accurate structure prediction of biomolecular interactions with AlphaFold 3
Sep 23, 2026
LIT-585: AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- numerics-and-precision
- inference-optimization
Sep 23, 2026
LIT-586: LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
Sep 23, 2026
LIT-587: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Sep 23, 2026
LIT-588: Learning Transferable Visual Models From Natural Language Supervision
Sep 23, 2026
LIT-589: Representation Learning with Contrastive Predictive Coding
- representation-and-encoding
- analysis-and-evaluation
Sep 23, 2026
LIT-590: Momentum Contrast for Unsupervised Visual Representation Learning
Sep 23, 2026
LIT-591: A Simple Framework for Contrastive Learning of Visual Representations
Sep 23, 2026
LIT-592: Latent Dirichlet Allocation
Sep 23, 2026
LIT-593: Exploring Simple Siamese Representation Learning
Sep 23, 2026
LIT-594: Bootstrap your own latent: A new approach to self-supervised Learning
Sep 23, 2026
LIT-595: VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning
Sep 23, 2026
LIT-596: Barlow Twins: Self-Supervised Learning via Redundancy Reduction
Sep 23, 2026
LIT-597: Reading Tea Leaves: How Humans Interpret Topic Models
- analysis-and-evaluation
- representation-and-encoding
Sep 23, 2026
LIT-598: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Sep 23, 2026
LIT-599: DINOv2: Learning Robust Visual Features without Supervision
Sep 23, 2026
LIT-600: Automatic Evaluation of Topic Coherence
- analysis-and-evaluation
- representation-and-encoding
Sep 23, 2026
LIT-601: Masked Autoencoders Are Scalable Vision Learners
Sep 23, 2026
LIT-602: GloVe: Global Vectors for Word Representation
Sep 23, 2026
LIT-603: Distributed Representations of Words and Phrases and their Compositionality
Sep 23, 2026
LIT-604: Efficient Estimation of Word Representations in Vector Space
Sep 23, 2026
LIT-605: Sigmoid Loss for Language Image Pre-Training
Sep 23, 2026
LIT-606: The Linear Representation Hypothesis and the Geometry of Large Language Models
- concept-geometry
- analysis-and-evaluation
Sep 23, 2026
LIT-607: Improving Distributional Similarity with Lessons Learned from Word Embeddings
Sep 23, 2026
LIT-608: Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors
- analysis-and-evaluation
- representation-and-encoding
Sep 23, 2026
LIT-609: Distributed Representations of Words and Phrases and their Compositionality
Sep 23, 2026
LIT-610: Efficient Estimation of Word Representations in Vector Space
Sep 23, 2026
LIT-611: GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Sep 23, 2026
LIT-612: Neural Word Embedding as Implicit Matrix Factorization
- representation-and-encoding
- signal-structure
Sep 23, 2026
LIT-613: A Latent Variable Model Approach to PMI-based Word Embeddings
Sep 23, 2026
LIT-614: Improved Techniques for Training GANs
Sep 23, 2026
LIT-615: Demystifying MMD GANs
Sep 23, 2026
LIT-616: On Calibration of Modern Neural Networks
- analysis-and-evaluation
- training-optimization
Sep 22, 2026
LIT-516: From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications
Sep 22, 2026
LIT-517: Small Singular Values Matter: A Random Matrix Analysis of Transformer Models
- analysis-and-evaluation
- inference-optimization
Sep 22, 2026
LIT-518: The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction
- inference-optimization
- analysis-and-evaluation
Sep 22, 2026
LIT-519: Complexity-Guided Component-wise Initialization for Language Model Pretraining
Sep 22, 2026
LIT-520: The Stability of Singular Distribution: A Spectral Perspective on the Two-Phase Dynamics of Language Model Pre-training
Sep 22, 2026
LIT-521: Taming Transformer Without Using Learning Rate Warmup
Sep 22, 2026
LIT-522: Pion: A Spectrum-Preserving Optimizer via Orthogonal Equivalence Transformation
- training-optimization
- model-stability
Sep 22, 2026
LIT-523: Stabilizing Transformer Training by Preventing Attention Entropy Collapse
Sep 22, 2026
LIT-524: Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures
- deployment-and-society
- analysis-and-evaluation
Sep 22, 2026
LIT-525: Transformers Learn Low Sensitivity Functions: Investigations and Implications
Sep 22, 2026
LIT-526: Concepts Whisper: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations
Sep 22, 2026
LIT-527: Leaner Transformers: More Heads, Less Depth
- model-architecture
- attention-techniques
Sep 22, 2026
LIT-528: The emergence of clusters in self-attention dynamics
Sep 22, 2026
LIT-529: Linformer: Self-Attention with Linear Complexity
- attention-techniques
- inference-optimization
Sep 22, 2026
LIT-530: Attention is not all you need: pure attention loses rank doubly exponentially with depth
Sep 22, 2026
LIT-531: Deep Learning and the Information Bottleneck Principle
Sep 22, 2026
LIT-532: What learning algorithm is in-context learning? Investigations with linear models
Sep 22, 2026
LIT-533: Transformers learn in-context by gradient descent
Sep 22, 2026
LIT-534: What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
Sep 22, 2026
LIT-535: Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression
Sep 22, 2026
LIT-536: Do pretrained Transformers Learn In-Context by Gradient Descent?
Sep 22, 2026
LIT-537: Grokking as the Transition from Lazy to Rich Training Dynamics
Sep 22, 2026
LIT-538: Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
Sep 22, 2026
LIT-539: Explaining grokking through circuit efficiency
Sep 22, 2026
LIT-540: Omnigrok: Grokking Beyond Algorithmic Data
Sep 22, 2026
LIT-541: Deep Learning is Singular, and That's Good
Sep 22, 2026
LIT-542: The Local Learning Coefficient: A Singularity-Aware Complexity Measure
Sep 22, 2026
LIT-543: Loss Landscape Degeneracy and Stagewise Development in Transformers
Sep 22, 2026
LIT-544: Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition
Sep 22, 2026
LIT-545: Gemma 3n model overview
Sep 22, 2026
LIT-546: MatFormer: Nested Transformer for Elastic Inference
Sep 22, 2026
LIT-547: Matryoshka Representation Learning
- representation-and-encoding
- inference-optimization
Sep 22, 2026
LIT-548: Feature Learning in Infinite-Width Neural Networks
Sep 22, 2026
LIT-549: ReFT: Representation Finetuning for Language Models
- adaptation-and-tuning
- analysis-and-evaluation
Sep 21, 2026
LIT-463: Comment on The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- analysis-and-evaluation
Sep 21, 2026
LIT-464: Theoretical limitations of multi-layer Transformer
Sep 21, 2026
LIT-465: Auxiliary task demands mask the capabilities of smaller language models
- analysis-and-evaluation
Sep 21, 2026
LIT-466: The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
- analysis-and-evaluation
Sep 21, 2026
LIT-467: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
- in-context-learning
- adaptation-and-tuning
Sep 21, 2026
LIT-468: Self-Consistency Improves Chain of Thought Reasoning in Language Models
- in-context-learning
- inference-optimization
Sep 21, 2026
LIT-469: Large Language Models are Zero-Shot Reasoners
- in-context-learning
Sep 21, 2026
LIT-470: Emergent Abilities of Large Language Models
Sep 21, 2026
LIT-471: Are Emergent Abilities of Large Language Models a Mirage?
Sep 21, 2026
LIT-472: nGPT: Normalized Transformer with Representation Learning on the Hypersphere
Sep 21, 2026
LIT-473: Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost
Sep 21, 2026
LIT-474: The Coverage Principle: How Pre-Training Enables Post-Training
Sep 21, 2026
LIT-475: Dropout Reduces Underfitting
Sep 21, 2026
LIT-476: Estimating the Probability of Sampling a Trained Neural Network at Random
Sep 21, 2026
LIT-477: NoProp: Training Neural Networks without Full Back-propagation or Full Forward-propagation
Sep 21, 2026
LIT-478: Neural networks with late-phase weights
- training-optimization
- model-stability
Sep 21, 2026
LIT-479: The Diffusion Duality
Sep 21, 2026
LIT-480: Auditing Political Exposure Bias: Algorithmic Amplification on Twitter/X During the 2024 U.S. Presidential Election
- deployment-and-society
- analysis-and-evaluation
Sep 21, 2026
LIT-481: Industry Influence in High-Profile Social Media Research
- deployment-and-society
- analysis-and-evaluation
Sep 21, 2026
LIT-482: Moving towards informative and actionable social media research
- deployment-and-society
- analysis-and-evaluation
Sep 21, 2026
LIT-483: Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
Sep 21, 2026
LIT-484: TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
Sep 21, 2026
LIT-485: Parameterized Synthetic Text Generation with SimpleStories
Sep 21, 2026
LIT-486: Parallel Scaling Law for Language Models
Sep 21, 2026
LIT-487: Introducing AI to an Online Petition Platform Changed Outputs but not Outcomes
- deployment-and-society
- analysis-and-evaluation
Sep 21, 2026
LIT-488: gen2seg: Generative Models Enable Generalizable Instance Segmentation
Sep 21, 2026
LIT-489: Measuring In-Context Computation Complexity via Hidden State Prediction
Sep 21, 2026
LIT-490: Denoising as Projection: Constrained Optimization with Gradient-Guided Diffusion
- generative-modeling
- analysis-and-evaluation
Sep 21, 2026
LIT-491: Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models
Sep 21, 2026
LIT-492: The unintended consequences of large language models as a labor-augmenting technology in science
- deployment-and-society
- analysis-and-evaluation
Sep 21, 2026
LIT-493: Language Modeling by Language Models
Sep 21, 2026
LIT-494: GaussianToken: An Effective Image Tokenizer with 2D Gaussian Splatting
Sep 21, 2026
LIT-495: On the Statistical Query Complexity of Learning Semiautomata: a Random Walk Approach
Sep 21, 2026
LIT-496: Taming Transformers for High-Resolution Image Synthesis
Sep 21, 2026
LIT-497: Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation
Sep 21, 2026
LIT-498: MaskGIT: Masked Generative Image Transformer
Sep 21, 2026
LIT-499: Neural Discrete Representation Learning
- representation-and-encoding
- generative-modeling
Sep 21, 2026
LIT-500: Vector-quantized Image Modeling with Improved VQGAN
Sep 21, 2026
LIT-501: The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation
Sep 21, 2026
LIT-502: MemGraphRAG: Memory-based Multi-Agent System for Graph Retrieval-Augmented Generation
Sep 21, 2026
LIT-503: First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization
Sep 21, 2026
LIT-504: Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?
- analysis-and-evaluation
Sep 21, 2026
LIT-505: Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences
Sep 21, 2026
LIT-506: Interpretable RNA Foundation Model from Unannotated Data for Highly Accurate RNA Structure and Function Predictions
Sep 21, 2026
LIT-507: Adaptive Estimators Show Information Compression in Deep Neural Networks
- analysis-and-evaluation
- representation-and-encoding
Sep 21, 2026
LIT-508: Opening the Black Box of Deep Neural Networks via Information
Sep 21, 2026
LIT-509: On the Information Bottleneck Theory of Deep Learning
- analysis-and-evaluation
- representation-and-encoding
Sep 21, 2026
LIT-510: Gemini: A Family of Highly Capable Multimodal Models
Sep 21, 2026
LIT-511: Image-GS: Content-Adaptive Image Representation via 2D Gaussians
Sep 21, 2026
LIT-512: SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
Sep 21, 2026
LIT-513: mHC-lite: You Don't Need 20 Sinkhorn-Knopp Iterations
- model-architecture
Sep 21, 2026
LIT-514: Rethinking Early Stopping: Refine, Then Calibrate
- analysis-and-evaluation
- training-optimization
Sep 21, 2026
LIT-515: Exploiting Block Coordinate Descent for Cost-Effective LLM Model Training
- systems-optimization
- training-optimization
Sep 20, 2026
LIT-443: Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training
- training-optimization
Sep 20, 2026
LIT-444: Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful
- training-optimization
Sep 20, 2026
LIT-445: How Does Critical Batch Size Scale in Pre-training?
- training-optimization
Sep 20, 2026
LIT-446: Variational Diffusion Models
- generative-modeling
Sep 20, 2026
LIT-447: SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers
- generative-modeling
- flows-and-transport
Sep 20, 2026
LIT-448: Scalable Diffusion Models with Transformers
- generative-modeling
- model-architecture
Sep 20, 2026
LIT-449: Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
Sep 20, 2026
LIT-450: How do language models learn facts? Dynamics, curricula and hallucinations
Sep 20, 2026
LIT-451: Data Mixing Can Induce Phase Transitions in Knowledge Acquisition
- data-pipeline
- training-optimization
Sep 20, 2026
LIT-452: Scaling Laws for Fact Memorization of Large Language Models
- analysis-and-evaluation
- training-optimization
Sep 20, 2026
LIT-453: Understanding Optimization in Deep Learning with Central Flows
- training-optimization
Sep 20, 2026
LIT-454: The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion
- training-optimization
Sep 20, 2026
LIT-455: Hidden Breakthroughs in Language Model Training
- analysis-and-evaluation
- training-optimization
Sep 20, 2026
LIT-456: Muon is Not That Special: Random or Inverted Spectra Work Just as Well
- training-optimization
Sep 20, 2026
LIT-457: When do spectral gradient updates help in deep learning?
- training-optimization
- signal-structure
Sep 20, 2026
LIT-458: The Platonic Representation Hypothesis
Sep 20, 2026
LIT-459: Contextures: The Mechanism of Representation Learning
- representation-and-encoding
- training-optimization
Sep 20, 2026
LIT-460: The Lattice Representation Hypothesis of Large Language Models
- concept-geometry
- representation-and-encoding
Sep 20, 2026
LIT-461: Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability
- training-optimization
Sep 20, 2026
LIT-462: Spectral Condition for μP under Width-Depth Scaling
- training-optimization
Sep 19, 2026
LIT-432: Finetuned Language Models Are Zero-Shot Learners
- adaptation-and-tuning
- data-pipeline
Sep 19, 2026
LIT-433: Learning to summarize from human feedback
- adaptation-and-tuning
- analysis-and-evaluation
Sep 19, 2026
LIT-434: Deep reinforcement learning from human preferences
- adaptation-and-tuning
Sep 19, 2026
LIT-435: NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
- vision-and-graphics
- representation-and-encoding
Sep 19, 2026
LIT-436: Scalable Optimization in the Modular Norm
- training-optimization
Sep 19, 2026
LIT-437: A Spectral Condition for Feature Learning
- training-optimization
Sep 19, 2026
LIT-438: Modular Duality in Deep Learning
- training-optimization
Sep 19, 2026
LIT-439: Deep Unsupervised Learning using Nonequilibrium Thermodynamics
- generative-modeling
Sep 19, 2026
LIT-440: How much do language models memorize?
- analysis-and-evaluation
- training-optimization
Sep 19, 2026
LIT-441: Pre-training under infinite compute
- training-optimization
- data-pipeline
Sep 19, 2026
LIT-442: Diffusion Beats Autoregressive in Data-Constrained Settings
- generative-modeling
- data-pipeline
Sep 18, 2026
LIT-404: HellaSwag: Can a Machine Really Finish Your Sentence?
- analysis-and-evaluation
Sep 18, 2026
LIT-405: On the Measure of Intelligence
- analysis-and-evaluation
Sep 18, 2026
LIT-406: Qwen2.5 Technical Report
- model-architecture
Sep 18, 2026
LIT-407: Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
- analysis-and-evaluation
Sep 18, 2026
LIT-408: Program Synthesis with Large Language Models
- analysis-and-evaluation
Sep 18, 2026
LIT-411: Measuring Mathematical Problem Solving With the MATH Dataset
- analysis-and-evaluation
Sep 18, 2026
LIT-415: Stream of Search (SoS): Learning to Search in Language
- analysis-and-evaluation
Sep 18, 2026
LIT-416: GPQA: A Graduate-Level Google-Proof Q&A Benchmark
- analysis-and-evaluation
Sep 18, 2026
LIT-418: HybridFlow: A Flexible and Efficient RLHF Framework
- distributed-optimization
- systems-optimization
Sep 18, 2026
LIT-419: OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
- analysis-and-evaluation
Sep 18, 2026
LIT-420: Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
- in-context-learning
- analysis-and-evaluation
Sep 18, 2026
LIT-421: Gradient-based learning applied to document recognition
- analysis-and-evaluation
- vision-and-graphics
Sep 18, 2026
LIT-422: ImageNet: A large-scale hierarchical image database
- analysis-and-evaluation
- vision-and-graphics
Sep 18, 2026
LIT-423: ImageNet Large Scale Visual Recognition Challenge
- analysis-and-evaluation
- vision-and-graphics
Sep 18, 2026
LIT-424: Learning Multiple Layers of Features from Tiny Images
- analysis-and-evaluation
- vision-and-graphics
Sep 18, 2026
LIT-425: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Sep 18, 2026
LIT-426: The Pile: An 800GB Dataset of Diverse Text for Language Modeling
- data-pipeline
- analysis-and-evaluation
Sep 18, 2026
LIT-427: SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
Sep 18, 2026
LIT-428: Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Sep 18, 2026
LIT-429: Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
- model-architecture
- attention-techniques
Sep 18, 2026
LIT-430: RWKV: Reinventing RNNs for the Transformer Era
- model-architecture
- attention-techniques
Sep 18, 2026
LIT-431: An Attention Free Transformer
- attention-techniques
- model-architecture
Sep 17, 2026
LIT-378: QLoRA: Efficient Finetuning of Quantized LLMs
Sep 17, 2026
LIT-379: The Surprising Effectiveness of Test-Time Training for Few-Shot Learning
Sep 17, 2026
LIT-380: The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Sep 17, 2026
LIT-381: Scaling Diffusion Language Models via Adaptation from Autoregressive Models
- generative-modeling
- adaptation-and-tuning
Sep 17, 2026
LIT-382: Large Language Models Cannot Self-Correct Reasoning Yet
- analysis-and-evaluation
- adaptation-and-tuning
Sep 17, 2026
LIT-383: Training Language Models to Self-Correct via Reinforcement Learning
- adaptation-and-tuning
Sep 17, 2026
LIT-384: VGGT: Visual Geometry Grounded Transformer
- vision-and-graphics
- model-architecture
Sep 17, 2026
LIT-385: DUSt3R: Geometric 3D Vision Made Easy
- vision-and-graphics
Sep 17, 2026
LIT-386: SuperGlue: Learning Feature Matching with Graph Neural Networks
- vision-and-graphics
- graphs-and-networks
Sep 17, 2026
LIT-387: LoFTR: Detector-Free Local Feature Matching with Transformers
- vision-and-graphics
- attention-techniques
Sep 17, 2026
LIT-388: Evaluating Large Language Models Trained on Code
- analysis-and-evaluation
Sep 17, 2026
LIT-389: Training Verifiers to Solve Math Word Problems
- analysis-and-evaluation
Sep 17, 2026
LIT-390: Measuring Massive Multitask Language Understanding
- analysis-and-evaluation
Sep 17, 2026
LIT-391: DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
- data-pipeline
Sep 17, 2026
LIT-392: Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models
- data-pipeline
Sep 17, 2026
LIT-393: Understanding Dropout
- model-stability
Sep 17, 2026
LIT-394: Improving neural networks by preventing co-adaptation of feature detectors
- model-stability
Sep 17, 2026
LIT-395: Dropout: A Simple Way to Prevent Neural Networks from Overfitting
- model-stability
Sep 17, 2026
LIT-396: Dropout Training as Adaptive Regularization
- model-stability
Sep 17, 2026
LIT-397: Deep Learning on a Data Diet: Finding Important Examples Early in Training
- data-pipeline
Sep 17, 2026
LIT-398: DataComp-LM: In search of the next generation of training sets for language models
- data-pipeline
- analysis-and-evaluation
Sep 17, 2026
LIT-399: Beyond neural scaling laws: beating power law scaling via data pruning
- data-pipeline
- training-optimization
Sep 17, 2026
LIT-400: Estimating Training Data Influence by Tracing Gradient Descent
- analysis-and-evaluation
- data-pipeline
Sep 17, 2026
LIT-401: Studying Large Language Model Generalization with Influence Functions
- analysis-and-evaluation
- data-pipeline
Sep 17, 2026
LIT-402: If Influence Functions are the Answer, Then What is the Question?
- analysis-and-evaluation
Sep 17, 2026
LIT-403: Understanding Black-box Predictions via Influence Functions
- analysis-and-evaluation
- data-pipeline
Sep 17, 2026
LIT-409: Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
- representation-and-encoding
- signal-structure
Sep 17, 2026
LIT-410: The English phrase-as-lemma construction: When a phrase masquerades as a word, people play along
- signal-structure
- representation-and-encoding
Sep 17, 2026
LIT-412: From Tokens to Words: On the Inner Lexicon of LLMs
- representation-and-encoding
Sep 17, 2026
LIT-413: Attention Is Off By One
- attention-techniques
Sep 17, 2026
LIT-414: Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing
- numerics-and-precision
- attention-techniques
Sep 17, 2026
LIT-417: The Remarkable Robustness of LLMs: Stages of Inference?
- model-architecture
Sep 16, 2026
LIT-375: Accelerating Large Language Model Decoding with Speculative Sampling
- inference-optimization
Sep 16, 2026
LIT-376: Fast Inference from Transformers via Speculative Decoding
- inference-optimization
Sep 16, 2026
LIT-377: Training language models to follow instructions with human feedback
- adaptation-and-tuning
Sep 15, 2026
LIT-226: MoEfication: Transformer Feed-forward Layers are Mixtures of Experts
- model-architecture
Sep 15, 2026
LIT-227: Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
- model-architecture
Sep 15, 2026
LIT-228: Emergent Modularity in Pre-trained Transformers
- analysis-and-evaluation
Sep 15, 2026
LIT-229: Evolution Strategies at the Hyperscale
- training-optimization
Sep 15, 2026
LIT-230: Understanding Evolution Strategies for LLM Reasoning: Broader Reasoning Coverage than GRPO
Sep 15, 2026
LIT-231: Hyper-ES: Effective Evolution Strategies for LLM Reasoning via Descent Direction Merging
- adaptation-and-tuning
- training-optimization
Sep 15, 2026
LIT-232: EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scale
- training-optimization
Sep 15, 2026
LIT-233: Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights
Sep 15, 2026
LIT-234: Beyond the Best Guess: Improving LLM Solution Coverage with Evolution Strategies
- adaptation-and-tuning
- training-optimization
Sep 15, 2026
LIT-235: Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training
Sep 15, 2026
LIT-236: The Blessing of Dimensionality in LLM Fine-tuning: A Variance-Curvature Perspective
Sep 15, 2026
LIT-237: Evolutionary Strategies lead to Catastrophic Forgetting in LLMs
- adaptation-and-tuning
- training-optimization
Sep 15, 2026
LIT-238: Overcoming Forgetting in LLM Fine-Tuning with Evolution Strategies
- adaptation-and-tuning
- training-optimization
Sep 15, 2026
LIT-239: ESSA: Evolutionary Strategies for Scalable Alignment
- adaptation-and-tuning
- training-optimization
Sep 15, 2026
LIT-240: ESSAM: A Novel Competitive Evolution Strategies Approach to Reinforcement Learning for Memory Efficient LLMs Fine-Tuning
- adaptation-and-tuning
- training-optimization
Sep 15, 2026
LIT-241: Proximal Policy Optimization Algorithms
- training-optimization
Sep 15, 2026
LIT-242: Evolution Strategies as a Scalable Alternative to Reinforcement Learning
- training-optimization
Sep 15, 2026
LIT-245: Bayesian Learning via Stochastic Gradient Langevin Dynamics
- training-optimization
Sep 15, 2026
LIT-246: Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts
- distributed-optimization
Sep 15, 2026
LIT-248: Topics in propagation of chaos
- analysis-and-evaluation
Sep 15, 2026
LIT-251: The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks
- model-stability
Sep 15, 2026
LIT-252: OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training
- distributed-optimization
Sep 15, 2026
LIT-254: Stochastic Gradient Push for Distributed Deep Learning
- distributed-optimization
Sep 15, 2026
LIT-257: Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
- distributed-optimization
Sep 15, 2026
LIT-261: The mean field analysis of the Kuramoto model on graphs II. Asymptotic stability of the incoherent state, center manifold reduction, and bifurcations
- analysis-and-evaluation
Sep 15, 2026
LIT-262: Consensus Based Sampling
- analysis-and-evaluation
Sep 15, 2026
LIT-265: Large Batch Optimization for Deep Learning: Training BERT in 76 minutes
- training-optimization
Sep 15, 2026
LIT-267: Mean Field Analysis of Neural Networks: A Law of Large Numbers
- analysis-and-evaluation
Sep 15, 2026
LIT-271: A Mean Field View of the Landscape of Two-Layers Neural Networks
- analysis-and-evaluation
Sep 15, 2026
LIT-273: Distributed Optimization with Gradient Tracking over Heterogeneous Delay-Prone Directed Networks
- distributed-optimization
Sep 15, 2026
LIT-275: The mean field analysis for the Kuramoto model on graphs I. The mean field equation and transition point formulas
- analysis-and-evaluation
Sep 15, 2026
LIT-276: Smoothing DiLoCo with Primal Averaging for Faster Training of LLMs
- distributed-optimization
Sep 15, 2026
LIT-278: 1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed
- distributed-optimization
Sep 15, 2026
LIT-279: HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent
- distributed-optimization
Sep 15, 2026
LIT-280: signSGD: Compressed Optimisation for Non-Convex Problems
- distributed-optimization
Sep 15, 2026
LIT-282: Convex Analysis of the Mean Field Langevin Dynamics
- analysis-and-evaluation
Sep 15, 2026
LIT-285: Coordination of groups of mobile autonomous agents using nearest neighbor rules
- distributed-optimization
- analysis-and-evaluation
Sep 15, 2026
LIT-286: The Kuramoto model: A simple paradigm for synchronization phenomena
- analysis-and-evaluation
Sep 15, 2026
LIT-288: Achieving Tighter Finite-Time Rates for Heterogeneous Federated Stochastic Approximation under Markovian Sampling
- distributed-optimization
Sep 15, 2026
LIT-292: Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms
- distributed-optimization
Sep 15, 2026
LIT-293: Emergent Behavior in Flocks
- analysis-and-evaluation
Sep 15, 2026
LIT-296: The Variational Formulation of the Fokker-Planck Equation
- analysis-and-evaluation
Sep 15, 2026
LIT-297: Reconciling modern machine learning practice and the bias-variance trade-off
- analysis-and-evaluation
Sep 15, 2026
LIT-298: On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport
- analysis-and-evaluation
Sep 15, 2026
LIT-301: Products of indecomposable, aperiodic, stochastic matrices
- analysis-and-evaluation
Sep 15, 2026
LIT-302: Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent
- distributed-optimization
Sep 15, 2026
LIT-303: SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
- distributed-optimization
Sep 15, 2026
LIT-304: Consensus Problems in Networks of Agents With Switching Topology and Time-Delays
- analysis-and-evaluation
Sep 15, 2026
LIT-305: A Bayesian Perspective on Generalization and Stochastic Gradient Descent
- training-optimization
Sep 15, 2026
LIT-307: Communication-Efficient Learning of Deep Networks from Decentralized Data
- distributed-optimization
Sep 15, 2026
LIT-310: Phase transitions, logarithmic Sobolev inequalities, and uniform-in-time propagation of chaos for weakly interacting diffusions
- analysis-and-evaluation
Sep 15, 2026
LIT-311: An effective theory of collective deep learning
- analysis-and-evaluation
Sep 15, 2026
LIT-312: Exact Solution for On-Line Learning in Multilayer Neural Networks
- analysis-and-evaluation
Sep 15, 2026
LIT-314: Proving the Limited Scalability of Centralized Distributed Optimization via a New Lower Bound Construction
- distributed-optimization
Sep 15, 2026
LIT-315: Epidemic Learning: Boosting Decentralized Learning with Randomized Communication
- distributed-optimization
Sep 15, 2026
LIT-316: Distributed Deep Learning in Open Collaborations
- distributed-optimization
Sep 15, 2026
LIT-323: Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD
- distributed-optimization
Sep 15, 2026
LIT-324: Broken Neural Scaling Laws
- analysis-and-evaluation
Sep 15, 2026
LIT-325: Distributed asynchronous deterministic and stochastic gradient optimization algorithms
- distributed-optimization
Sep 15, 2026
LIT-326: A Necessary and Sufficient Condition for Consensus Over Random Networks
- analysis-and-evaluation
Sep 15, 2026
LIT-328: Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networks
- analysis-and-evaluation
- signal-structure
Sep 15, 2026
LIT-333: Git Re-Basin: Merging Models modulo Permutation Symmetries
- model-stability
Sep 15, 2026
LIT-336: Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks
- distributed-optimization
Sep 15, 2026
LIT-337: PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization
- distributed-optimization
Sep 15, 2026
LIT-340: Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup
- analysis-and-evaluation
Sep 15, 2026
LIT-341: A Growing Neural Gas Network Learns Topologies
- representation-and-encoding
Sep 15, 2026
LIT-344: Byzantine-Tolerant Machine Learning
- distributed-optimization
Sep 15, 2026
LIT-345: Generalized EXTRA stochastic gradient Langevin dynamics
- distributed-optimization
Sep 15, 2026
LIT-346: On-line learning in soft committee machines
- analysis-and-evaluation
Sep 15, 2026
LIT-350: Subspace Networks: Scaling Decentralized Training with Communication-Efficient Model Parallelism
- distributed-optimization
Sep 15, 2026
LIT-353: Emergence of stochastic flocking for the discrete Cucker-Smale model with randomly switching topologies
- analysis-and-evaluation
Sep 15, 2026
LIT-354: Asynchronous Stochastic Gradient Descent with Delay Compensation
- distributed-optimization
Sep 15, 2026
LIT-360: Neural Tangent Kernel: Convergence and Generalization in Neural Networks
- analysis-and-evaluation
- training-optimization
Sep 15, 2026
LIT-361: Local SGD Converges Fast and Communicates Little
- distributed-optimization
Sep 15, 2026
LIT-362: Don't Use Large Mini-Batches, Use Local SGD
- distributed-optimization
Sep 15, 2026
LIT-363: 'Neural-gas' network for vector quantization and its application to time-series prediction
- representation-and-encoding
Sep 15, 2026
LIT-364: Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed
- analysis-and-evaluation
Sep 15, 2026
LIT-365: Trainability and Accuracy of Neural Networks: An Interacting Particle System Approach
- analysis-and-evaluation
Sep 15, 2026
LIT-366: Asynchronous Local Computations in Distributed Bayesian Learning
- distributed-optimization
Sep 15, 2026
LIT-367: ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
- systems-optimization
Sep 15, 2026
LIT-370: Deep learning with Elastic Averaging SGD
- distributed-optimization
Sep 15, 2026
LIT-372: Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices
- distributed-optimization
Sep 15, 2026
LIT-373: SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
- distributed-optimization
Sep 15, 2026
LIT-374: LAYUP: Asynchronous decentralized gradient descent with LAYer-wise UPdates
- distributed-optimization
Sep 13, 2026
LIT-225: Generating Long Sequences with Sparse Transformers
- attention-techniques
Sep 09, 2026
LIT-219: PyTorch Distributed: Experiences on Accelerating Data Parallel Training
- distributed-optimization
Sep 09, 2026
LIT-220: Horovod: fast and easy distributed deep learning in TensorFlow
- distributed-optimization
Sep 09, 2026
LIT-221: Train With Mixed Precision — NVIDIA deep learning performance guide
- numerics-and-precision
- training-optimization
Sep 09, 2026
LIT-222: CoreWeave Training Benchmarks Whitepaper
- distributed-optimization
Sep 09, 2026
LIT-223: How Does Batch Normalization Help Optimization?
- model-stability
Sep 09, 2026
LIT-224: Orca: A Distributed Serving System for Transformer-Based Generative Models
- inference-optimization
Sep 07, 2026
LIT-187: GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
- distributed-optimization
- model-architecture
Sep 07, 2026
LIT-188: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
- model-architecture
Sep 07, 2026
LIT-189: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- model-architecture
Sep 07, 2026
LIT-190: Massive Activations in Large Language Models
- analysis-and-evaluation
- attention-techniques
Sep 07, 2026
LIT-191: Efficient Streaming Language Models with Attention Sinks
- attention-techniques
Sep 07, 2026
LIT-192: Extending Context Window of Large Language Models via Positional Interpolation
- representation-and-encoding
- attention-techniques
Sep 07, 2026
LIT-193: YaRN: Efficient Context Window Extension of Large Language Models
- representation-and-encoding
- attention-techniques
Sep 07, 2026
LIT-194: Gated Linear Attention Transformers with Hardware-Efficient Training
- attention-techniques
Sep 07, 2026
LIT-195: Parallelizing Linear Transformers with the Delta Rule over Sequence Length
- attention-techniques
Sep 07, 2026
LIT-196: LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts
- model-architecture
Sep 07, 2026
LIT-197: Microscaling Data Formats for Deep Learning
- numerics-and-precision
- systems-optimization
Sep 07, 2026
LIT-198: Why Low-Precision Transformer Training Fails: An Analysis on Flash Attention
- numerics-and-precision
- model-stability
Sep 07, 2026
LIT-199: Language Modeling with Gated Convolutional Networks
- model-architecture
Sep 07, 2026
LIT-200: PowLU: An Activation Function for Stable Pre-Training of LLMs
Sep 07, 2026
LIT-201: Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance
- data-pipeline
Sep 07, 2026
LIT-202: Deduplicating Training Data Makes Language Models Better
- data-pipeline
Sep 07, 2026
LIT-203: Fewer Truncations Improve Language Modeling
- data-pipeline
Sep 07, 2026
LIT-204: Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
- attention-techniques
Sep 07, 2026
LIT-205: How to Synthesize Text Data without Model Collapse?
- data-pipeline
- signal-structure
Sep 07, 2026
LIT-206: Ring Attention with Blockwise Transformers for Near-Infinite Context
- distributed-optimization
Sep 07, 2026
LIT-207: The Impact of Positional Encoding on Length Generalization in Transformers
- representation-and-encoding
- attention-techniques
Sep 07, 2026
LIT-208: Rope to Nope and Back Again: A New Hybrid Attention Strategy
- representation-and-encoding
- attention-techniques
Sep 07, 2026
LIT-209: SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling
- representation-and-encoding
- attention-techniques
Sep 07, 2026
LIT-210: Round and Round We Go! What makes Rotary Positional Encodings useful?
- representation-and-encoding
- attention-techniques
Sep 07, 2026
LIT-211: Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning
- adaptation-and-tuning
- training-optimization
Sep 07, 2026
LIT-212: DiLoCo: Distributed Low-Communication Training of Language Models
- distributed-optimization
Sep 07, 2026
LIT-213: The Road Less Scheduled
- training-optimization
Sep 07, 2026
LIT-214: Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch
- distributed-optimization
Sep 07, 2026
LIT-215: V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
- vision-and-graphics
Sep 07, 2026
LIT-216: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
- vision-and-graphics
- representation-and-encoding
Sep 07, 2026
LIT-217: Large Language Diffusion Models
- generative-modeling
Sep 07, 2026
LIT-218: Video models are zero-shot learners and reasoners
- vision-and-graphics
Sep 05, 2026
LIT-119: Falcon-H1-Tiny: A series of extremely small, yet powerful language models redefining capabilities at small scale
Sep 05, 2026
LIT-120: Falcon-H1: A Family of Hybrid-Head Language Models Redefining Efficiency and Performance
- model-architecture
- data-pipeline
Sep 05, 2026
LIT-121: Learnable Multipliers: Freeing the Scale of Language Model Matrix Layers
- training-optimization
Sep 05, 2026
LIT-122: Muon is Scalable for LLM Training
- training-optimization
Sep 05, 2026
LIT-123: Wait, Wait, Wait... Why Do Reasoning Models Loop?
- analysis-and-evaluation
- tiny-models
Sep 05, 2026
LIT-124: Efficient Training of Language Models to Fill in the Middle
- data-pipeline
Sep 05, 2026
LIT-125: Qwen2.5-Coder Technical Report
- data-pipeline
Sep 05, 2026
LIT-126: Structure-Aware Fill-in-the-Middle Pretraining for Code
- data-pipeline
Sep 05, 2026
LIT-127: DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Sep 05, 2026
LIT-128: Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scaling
- adaptation-and-tuning
Sep 05, 2026
LIT-129: Can Tiny Language Models Reason?
Sep 05, 2026
LIT-130: Olmo 3
- adaptation-and-tuning
Sep 05, 2026
LIT-131: Kimi K3: Open Frontier Intelligence
- model-architecture
Sep 05, 2026
LIT-132: Kimi K2: Open Agentic Intelligence
- training-optimization
Sep 05, 2026
LIT-133: Kimi Linear: An Expressive, Efficient Attention Architecture
- attention-techniques
- representation-and-encoding
Sep 05, 2026
LIT-134: Attention Residuals
- model-architecture
Sep 05, 2026
LIT-135: Qwen3.8-27B
- model-architecture
- multimodal-learning
Sep 05, 2026
LIT-136: Qwen3-Next: Towards Ultimate Training & Inference Efficiency
- model-architecture
Sep 05, 2026
LIT-137: Gated Delta Networks: Improving Mamba2 with Delta Rule
- attention-techniques
- model-architecture
Sep 05, 2026
LIT-138: Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free
- attention-techniques
Sep 05, 2026
LIT-139: DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Sep 05, 2026
LIT-140: mHC: Manifold-Constrained Hyper-Connections
- model-architecture
Sep 05, 2026
LIT-141: Hyper-Connections
- model-architecture
Sep 05, 2026
LIT-142: DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- attention-techniques
Sep 05, 2026
LIT-143: Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
- attention-techniques
Sep 05, 2026
LIT-144: MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
- training-optimization
Sep 05, 2026
LIT-145: Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
- training-optimization
Sep 05, 2026
LIT-146: Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler
- training-optimization
Sep 05, 2026
LIT-147: Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs
- training-optimization
Sep 05, 2026
LIT-148: Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer
- training-optimization
Sep 05, 2026
LIT-149: u-µP: The Unit-Scaled Maximal Update Parametrization
- training-optimization
- numerics-and-precision
Sep 05, 2026
LIT-150: Don't be lazy: CompleteP enables compute-efficient deep transformers
- training-optimization
Sep 05, 2026
LIT-151: oHC: Orthogonal Hyper-Connections on SO(4) via Quaternions
- model-architecture
Sep 05, 2026
LIT-152: On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability
- model-architecture
Sep 05, 2026
LIT-153: Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization
- training-optimization
Sep 05, 2026
LIT-154: 2 OLMo 2 Furious
- model-stability
Sep 05, 2026
LIT-155: Small-scale proxies for large-scale Transformer training instabilities
- model-stability
- training-optimization
Sep 05, 2026
LIT-156: Fantastic Pretraining Optimizers and Where to Find Them
- training-optimization
Sep 05, 2026
LIT-157: SOAP: Improving and Stabilizing Shampoo using Adam
- training-optimization
Sep 05, 2026
LIT-158: A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
- distributed-optimization
- training-optimization
Sep 05, 2026
LIT-159: Muon: An optimizer for hidden layers in neural networks
- training-optimization
Sep 05, 2026
LIT-160: DeepSeek-V3 Technical Report
- model-architecture
Sep 05, 2026
LIT-161: Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- model-architecture
- attention-techniques
Sep 05, 2026
LIT-162: Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
- model-architecture
- attention-techniques
Sep 05, 2026
LIT-163: Better & Faster Large Language Models via Multi-token Prediction
- training-optimization
Sep 05, 2026
LIT-164: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- adaptation-and-tuning
Sep 05, 2026
LIT-165: Mamba-3: Improved Sequence Modeling using State Space Principles
- model-architecture
- attention-techniques
Sep 05, 2026
LIT-166: Scaling Data-Constrained Language Models
- data-pipeline
Sep 05, 2026
LIT-167: Understanding R1-Zero-Like Training: A Critical Perspective
- analysis-and-evaluation
- adaptation-and-tuning
Sep 05, 2026
LIT-168: DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- adaptation-and-tuning
Sep 05, 2026
LIT-169: Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- adaptation-and-tuning
Sep 05, 2026
LIT-170: DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
- model-architecture
Sep 05, 2026
LIT-171: Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts
- model-architecture
Sep 05, 2026
LIT-172: Tulu 3: Pushing Frontiers in Open Language Model Post-Training
- adaptation-and-tuning
Sep 05, 2026
LIT-173: RWKV-7 "Goose" with Expressive Dynamic State Evolution
- model-architecture
Sep 05, 2026
LIT-174: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
- attention-techniques
Sep 05, 2026
LIT-175: Demystifying Training-Time Augmentation for Data-Constrained Language Model Pretraining
- data-pipeline
Sep 05, 2026
LIT-176: Native Hybrid Attention for Efficient Sequence Modeling
- attention-techniques
Sep 05, 2026
LIT-177: Erase-then-Delta Attention: Decoupling Erase and Write Addresses in Delta-Rule Linear Attention
- attention-techniques
Sep 05, 2026
LIT-178: Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset
- data-pipeline
Sep 05, 2026
LIT-179: The Llama 3 Herd of Models
- model-architecture
Sep 05, 2026
LIT-180: Group Sequence Policy Optimization
- adaptation-and-tuning
Sep 05, 2026
LIT-181: Beyond the Birkhoff Polytope: Spectral-Sphere-Constrained Hyper-Connections
- model-architecture
Sep 05, 2026
LIT-182: Qwen3 Technical Report
- model-architecture
Sep 05, 2026
LIT-183: Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
- model-architecture
Sep 05, 2026
LIT-184: The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
- data-pipeline
Sep 05, 2026
LIT-185: EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
- inference-optimization
Sep 05, 2026
LIT-186: Scaling Laws for Precision
- numerics-and-precision
- training-optimization
Aug 24, 2026
LIT-001: Adam: A Method for Stochastic Optimization
- training-optimization
Aug 24, 2026
LIT-002: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- model-stability
Aug 24, 2026
LIT-003: Neural Machine Translation of Rare Words with Subword Units
- representation-and-encoding
Aug 24, 2026
LIT-004: Training Deep Nets with Sublinear Memory Cost
- distributed-optimization
- systems-optimization
Aug 24, 2026
LIT-005: Layer Normalization
- model-stability
Aug 24, 2026
LIT-006: SGDR: Stochastic Gradient Descent with Warm Restarts
- training-optimization
Aug 24, 2026
LIT-007: Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
- distributed-optimization
- training-optimization
Aug 24, 2026
LIT-008: Attention Is All You Need
- model-architecture
- attention-techniques
Aug 24, 2026
LIT-009: Large Batch Training of Convolutional Networks
- training-optimization
Aug 24, 2026
LIT-010: Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
- training-optimization
Aug 24, 2026
LIT-011: Mixed Precision Training
- numerics-and-precision
- training-optimization
Aug 24, 2026
LIT-012: Decoupled Weight Decay Regularization
- training-optimization
Aug 24, 2026
LIT-013: Population Based Training of Neural Networks
- training-optimization
Aug 24, 2026
LIT-014: Visualizing the Loss Landscape of Neural Nets
- analysis-and-evaluation
- model-stability
Aug 24, 2026
LIT-015: Understanding Batch Normalization
- model-stability
Aug 24, 2026
LIT-016: GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
- distributed-optimization
Aug 24, 2026
LIT-017: An Empirical Model of Large-Batch Training
- training-optimization
Aug 24, 2026
LIT-018: On the Convergence of Adam and Beyond
- training-optimization
Aug 24, 2026
LIT-019: The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
- analysis-and-evaluation
Aug 24, 2026
LIT-020: Pay Less Attention with Lightweight and Dynamic Convolutions
- attention-techniques
- model-architecture
Aug 24, 2026
LIT-021: Mish: A Self Regularized Non-Monotonic Activation Function
- model-architecture
Aug 24, 2026
LIT-022: Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
- distributed-optimization
- model-stability
Aug 24, 2026
LIT-023: Root Mean Square Layer Normalization
- model-stability
- model-architecture
Aug 24, 2026
LIT-024: Fast Transformer Decoding: One Write-Head is All You Need
Aug 24, 2026
LIT-025: Understanding and Improving Layer Normalization
- model-stability
Aug 24, 2026
LIT-026: Advances and Open Problems in Federated Learning
- distributed-optimization
Aug 24, 2026
LIT-027: ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
- distributed-optimization
Aug 24, 2026
LIT-028: Scaling Laws for Neural Language Models
- training-optimization
Aug 24, 2026
LIT-029: On Layer Normalization in the Transformer Architecture
- model-stability
Aug 24, 2026
LIT-030: GLU Variants Improve Transformer
- model-architecture
Aug 24, 2026
LIT-031: Pruned Neural Networks are Surprisingly Modular
- analysis-and-evaluation
Aug 24, 2026
LIT-032: PowerNorm: Rethinking Batch Normalization in Transformers
- model-stability
Aug 24, 2026
LIT-033: Longformer: The Long-Document Transformer
- attention-techniques
Aug 24, 2026
LIT-034: Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
- adaptation-and-tuning
Aug 24, 2026
LIT-035: Language Models are Few-Shot Learners
Aug 24, 2026
LIT-036: Denoising Diffusion Probabilistic Models
- generative-modeling
Aug 24, 2026
LIT-037: On the difficulty of training Recurrent Neural Networks
- model-stability
Aug 24, 2026
LIT-038: Denoising Diffusion Implicit Models
- generative-modeling
Aug 24, 2026
LIT-039: Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win
- analysis-and-evaluation
Aug 24, 2026
LIT-040: Scaling Laws for Autoregressive Generative Modeling
- training-optimization
Aug 24, 2026
LIT-041: Clusterability in Neural Networks
- analysis-and-evaluation
Aug 24, 2026
LIT-042: SGDR: Stochastic Gradient Descent with Warm Restarts
- training-optimization
Aug 24, 2026
LIT-043: Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM
- distributed-optimization
- model-stability
Aug 24, 2026
LIT-044: Contrastive Learning with Hard Negative Samples
- training-optimization
Aug 24, 2026
LIT-045: RoFormer: Enhanced Transformer with Rotary Position Embedding
- representation-and-encoding
- model-architecture
Aug 24, 2026
LIT-046: LoRA: Low-Rank Adaptation of Large Language Models
- adaptation-and-tuning
Aug 24, 2026
LIT-047: ReZero is All You Need: Fast Convergence at Large Depth
- model-stability
- model-architecture
Aug 24, 2026
LIT-048: Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
- representation-and-encoding
- model-architecture
Aug 24, 2026
LIT-049: Learning to Prompt for Vision-Language Models
Aug 24, 2026
LIT-050: Analyzing and Mitigating Data Stalls in DNN Training
- data-pipeline
Aug 24, 2026
LIT-051: A Unified Architecture for Accelerating Distributed DNN Training in Heterogeneous GPU/CPU Clusters
- distributed-optimization
Aug 24, 2026
LIT-052: Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
Aug 24, 2026
LIT-053: High Performance I/O For Large Scale Deep Learning
- data-pipeline
Aug 24, 2026
LIT-054: GLM-130B: An Open Bilingual Pre-trained Model
- model-stability
Aug 24, 2026
LIT-055: P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks
- adaptation-and-tuning
Aug 24, 2026
LIT-056: Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
- distributed-optimization
Aug 24, 2026
LIT-057: What Makes for Good Views for Contrastive Learning?
- analysis-and-evaluation
Aug 24, 2026
LIT-058: Measuring the Effects of Data Parallelism on Neural Network Training
- training-optimization
Aug 24, 2026
LIT-059: CheckFreq: Frequent, Fine-Grained DNN Checkpointing
- systems-optimization
- distributed-optimization
Aug 24, 2026
LIT-060: Improving Language Models by Retrieving from Trillions of Tokens
- model-architecture
- analysis-and-evaluation
Aug 24, 2026
LIT-061: Efficient Large Scale Language Modeling with Mixtures of Experts
- model-architecture
- training-optimization
Aug 24, 2026
LIT-062: High-Resolution Image Synthesis with Latent Diffusion Models
- generative-modeling
- signal-structure
Aug 24, 2026
LIT-063: TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
- systems-optimization
Aug 24, 2026
LIT-064: Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
- vision-and-graphics
Aug 24, 2026
LIT-065: Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
- distributed-optimization
- training-optimization
Aug 24, 2026
LIT-066: Data Movement Is All You Need: A Case Study on Optimizing Transformers
- systems-optimization
Aug 24, 2026
LIT-067: Progressive Distillation for Fast Sampling of Diffusion Models
- generative-modeling
- few-step-generation
Aug 24, 2026
LIT-068: Training Compute-Optimal Large Language Models
- training-optimization
Aug 24, 2026
LIT-069: PaLM: Scaling Language Modeling with Pathways
- model-architecture
- training-optimization
Aug 24, 2026
LIT-070: Hierarchical Text-Conditional Image Generation with CLIP Latents
- generative-modeling
Aug 24, 2026
LIT-071: CoCa: Contrastive Captioners are Image-Text Foundation Models
- multimodal-learning
- model-architecture
Aug 24, 2026
LIT-072: Simple Open-Vocabulary Object Detection with Vision Transformers
Aug 24, 2026
LIT-073: Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
- generative-modeling
Aug 24, 2026
LIT-074: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
- attention-techniques
- systems-optimization
Aug 24, 2026
LIT-075: Elucidating the Design Space of Diffusion-Based Generative Models
- generative-modeling
Aug 24, 2026
LIT-076: DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
- generative-modeling
- few-step-generation
Aug 24, 2026
LIT-077: Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
- analysis-and-evaluation
Aug 24, 2026
LIT-078: An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
- adaptation-and-tuning
Aug 24, 2026
LIT-079: DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
- adaptation-and-tuning
Aug 24, 2026
LIT-080: PaLI: A Jointly-Scaled Multilingual Language-Image Model
Aug 24, 2026
LIT-081: GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- numerics-and-precision
- inference-optimization
Aug 24, 2026
LIT-082: Constitutional AI: Harmlessness from AI Feedback
- adaptation-and-tuning
- data-pipeline
Aug 24, 2026
LIT-083: PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
- distributed-optimization
Aug 24, 2026
LIT-084: DeepNet: Scaling Transformers to 1,000 Layers
- model-stability
Aug 24, 2026
LIT-085: Progress measures for grokking via mechanistic interpretability
- analysis-and-evaluation
- capability-thresholds
Aug 24, 2026
LIT-086: K-Planes: Explicit Radiance Fields in Space, Time, and Appearance
- vision-and-graphics
Aug 24, 2026
LIT-087: Data Selection for Language Models via Importance Resampling
- data-pipeline
Aug 24, 2026
LIT-088: Scaling Vision Transformers to 22 Billion Parameters
- model-stability
Aug 24, 2026
LIT-089: Adding Conditional Control to Text-to-Image Diffusion Models
- adaptation-and-tuning
Aug 24, 2026
LIT-090: Symbolic Discovery of Optimization Algorithms
- training-optimization
Aug 24, 2026
LIT-091: T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models
- adaptation-and-tuning
Aug 24, 2026
LIT-092: Understanding Contrastive Learning Requires Incorporating Inductive Biases
- analysis-and-evaluation
Aug 24, 2026
LIT-093: Consistency Models
- generative-modeling
- few-step-generation
Aug 24, 2026
LIT-094: Neural Scene Graphs for Dynamic Scenes
- vision-and-graphics
Aug 24, 2026
LIT-095: The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- analysis-and-evaluation
Aug 24, 2026
LIT-096: Segment Anything
- vision-and-graphics
- data-pipeline
Aug 24, 2026
LIT-097: One Transformer Fits All Distributions in Multi-Modal Diffusion at Scale
- generative-modeling
Aug 24, 2026
LIT-098: PipeMare: Asynchronous Pipeline Parallel DNN Training
- distributed-optimization
Aug 24, 2026
LIT-099: PaLM 2 Technical Report
- training-optimization
- data-pipeline
Aug 24, 2026
LIT-100: GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
- attention-techniques
Aug 24, 2026
LIT-101: AdaNorm: Adaptive Gradient Norm Correction based Optimizer for CNNs
- training-optimization
Aug 24, 2026
LIT-102: ZeRO++: Extremely Efficient Collective Communication for Giant Model Training
- distributed-optimization
- numerics-and-precision
Aug 24, 2026
LIT-103: SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills
- inference-optimization
Aug 24, 2026
LIT-104: ReLoRA: High-Rank Training Through Low-Rank Updates
- training-optimization
- adaptation-and-tuning
Aug 24, 2026
LIT-105: Symbolic Discovery of Optimization Algorithms
- training-optimization
Aug 24, 2026
LIT-106: FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- attention-techniques
Aug 24, 2026
LIT-107: MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimation
- vision-and-graphics
Aug 24, 2026
LIT-108: 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- vision-and-graphics
Aug 24, 2026
LIT-109: NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction
- vision-and-graphics
Aug 24, 2026
LIT-110: Efficiently Scaling Transformer Inference
- inference-optimization
Aug 24, 2026
LIT-111: OnePose: One-Shot Object Pose Estimation without CAD Models
- vision-and-graphics
Aug 24, 2026
LIT-112: Efficient Memory Management for Large Language Model Serving with PagedAttention
- inference-optimization
- attention-techniques
Aug 24, 2026
LIT-113: Gaussian Shell Maps for Efficient 3D Human Generation
- vision-and-graphics
Aug 24, 2026
LIT-114: On Layer Normalization in the Transformer Architecture
- model-stability
Aug 24, 2026
LIT-115: Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
- attention-techniques
Aug 24, 2026
LIT-116: TensorRT-LLM: A Compiler and Runtime for LLMs
- systems-optimization
Aug 24, 2026
LIT-117: Efficient Online Data Mixing For Language Model Pre-Training
- data-pipeline
Aug 24, 2026
LIT-118: PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding
- adaptation-and-tuning

Created with Quartz v5.0.0 © 2026

Repository
Quartz