Your daily digest
Research papers, made simple to understand
The AI and machine learning papers people are reading most, each one rewritten in plain words: the problem, the idea, what was found and why it matters.
- Reinforcement Learning & Alignment
Direct Preference Optimization fine-tunes language models from human preferences without reinforcement learning
Direct Preference Optimization (DPO) trains language models to align with human preferences using a simple classification loss, matching or outperforming RLHF methods like PPO on sentiment, summarization, and dialogue tasks, while being more stable and efficient.
Paper: Direct Preference Optimization: Your Language Model is Secretly a Reward Model · Rafael Rafailov, Archit Sharma, Eric Mitchell and 3 others
Classic · 20237 min readRead the article
- Reinforcement Learning & Alignment
Proximal Policy Optimization improves policy gradient methods with simpler, efficient updates
PPO introduces a new surrogate objective that allows multiple minibatch updates per data sample, improving sample efficiency and simplicity over TRPO and other policy gradient methods, tested on robotic locomotion and Atari games.
Paper: Proximal Policy Optimization Algorithms · John Schulman, Filip Wolski, Prafulla Dhariwal and 2 others
Classic · 20174 min readRead the article
- Reinforcement Learning & Alignment
Training harmless AI assistants using AI-generated feedback and constitutional principles
This paper introduces Constitutional AI, a method that trains AI assistants to be harmless and non-evasive by using AI-generated critiques and preference labels based on a set of human-written principles, reducing reliance on human feedback labels for harmfulness.
Paper: Constitutional AI: Harmlessness from AI Feedback · Yuntao Bai, Saurav Kadavath, Sandipan Kundu and 48 others
Classic · 20226 min readRead the article
Browse earlier papers
18 days of papers in November 2026
- WeVisForensics jointly trains an investigator agent and evolving forensic inspectors for AI video detectionand 2 more papers
- Jeveration generates images and dense predictions in one fast forward passand 2 more papers
- Xing 4 advances large language models on Ascend chips with efficient architecture and trainingand 2 more papers
- Sage S2.1 Flash: A 320B-parameter multimodal model excelling in scientific reasoning and agentic tasksand 2 more papers
- Agentic robotics uses foundation models plus harnesses for versatile physical task controland 2 more papers
- Direct Preference Optimization fine-tunes language models from human preferences without reinforcement learningYou are reading this day
- Layer normalization stabilizes and speeds up training, especially in recurrent networksand 2 more papers
- SO-Arena benchmarks System One models' decision capabilities and reliability across domainsand 2 more papers
- Structured textbook-based supervision improves mathematical reasoning in large language modelsand 2 more papers
- Tree-Ising optimizes sparse bit-level edits to fix reasoning loops in language modelsand 2 more papers
- Unified tactile representation improves dexterous robot manipulation across sensor typesand 2 more papers
- Gaussian Spacing Normalization improves feed-forward 3D Gaussian Splatting generalizationand 2 more papers
- VFlash enables fast, accurate visual plan decoding by jointly predicting entire primitivesand 2 more papers
- TurnWise improves streaming video RL by assigning credit at each decision turnand 1 more paper
- Routing specialized AI models boosts efficiency and performance across tasksand 2 more papers
- Unified taxonomy and literature system organize agentic computer graphics researchand 2 more papers
- FrameForge adapts video models for diverse conditional image generation with input-adaptive temporal relationsand 2 more papers
- Diffusion models generate high-quality images by learning to reverse noise additionand 2 more papers