Reinforcement learning
- Reinforcement learningProject
OpenTinker provides RL-as-a-Service infrastructure for foundation models
OpenTinker is an open-source platform for agentic reinforcement learning as a service. It separates programming from execution and supports transition from training to inference.
The platform offers an alternative infrastructure for building and deploying reinforcement-learning workflows for foundation models.
- 10 months ago @alxfazio
Recursive language models use code to process long prompts
The post describes MIT’s recursive language models, which use a Python interface to search and filter long prompts and recursively process smaller chunks. It claims they handle inputs up to about 100× larger than usual and outperform base models and retrieval setups on information-heavy tasks.
The approach may help engineers process long inputs without relying on a larger context window.
- 10 months ago @HuggingPapers
Mask-Progressive RL Distillation for VLMs
The post describes Masters, a mask-progressive RL distillation framework for VLMs that it says avoids the computational cost of online RL. It reports scores rising from 75.7% to 80.4% for Qwen3-VL-8B and 75.4% to 80.0% for InternVL3.5-8B.
The reported results may interest engineers evaluating lower-compute approaches to improving VLM performance.
JustRL tests a simple RL recipe on 1.5B reasoning models
The paper presents JustRL, a single-stage training approach with fixed hyperparameters for two 1.5B reasoning models. Its preview reports average accuracies of 54.9% and 64.3%.
It offers a baseline for comparing simpler RL training recipes with more complex approaches.
- Reinforcement learningArticle
LoRA strategies for multi-domain RLVR
The author reports experiments on joint multi-domain RL training and expert LoRAs, finding interference with joint training. Averaging expert LoRA weights and continuing training outperformed a gated approach in their experiments.
The work compares practical PEFT strategies for RLVR and reports a simple baseline that beat the tested gating method.
- 10 months ago @aakashgupta
Self-play RL trains coding agents on bug injection and repair
Self-play SWE-RL trains one LLM agent to inject and repair bugs in real repositories. The post says the agent receives no natural-language problem description or human-labeled issues.
The approach explores training software agents without relying on human-curated coding tasks.
Advantage shaping for single-rollout multimodal RL
The post says a single-rollout approach for multimodal RL became stable after adding advantage shaping with an entropy bonus. The linked paper preview describes Single-stream Policy Optimization for LLMs.
The stabilization detail may be useful when designing low-variance policy-gradient methods.
- Reinforcement learningProject
OpenTinker separates RL programming from execution
OpenTinker is an open-source RL-as-a-Service platform for foundation models. Its page describes separating programming from execution and supporting a transition from training to inference.
The design may help engineers develop RL environments locally while using remote compute.
- Reinforcement learningArticle
A roundup of RL training systems and pass@k results
The post surveys on-policy rollout mismatch, rollout-system design, and studies of RL’s effects on pass@k. It reports mixed findings, including gains from relaxing reward assignment constraints and from OLMo 3 DPO training.
It highlights practical rollout challenges and evidence that RL’s effect on sampling performance depends on the training approach.
SA-RAG Uses Spreading Activation for Knowledge-Graph Retrieval
SA-RAG applies spreading activation to knowledge-graph-based RAG: activation spreads from query-matched entities through weighted connections, helping retrieve documents linked to highly activated entities.
It offers an alternative to LLM-guided iterative retrieval for finding connected evidence in multi-hop questions.
Stabilizing Reinforcement Learning with LLMs
A Qwen team paper examines instability in reinforcement learning for large language models. The post highlights training–inference and old–new policy gaps, and mentions importance sampling and routing replay.
The paper may help engineers understand practices for stabilizing reinforcement learning in large language models.
- 11 months ago @cwolferesearch
Olmo 3’s SFT, DPO, and RLVR training pipeline
Olmo 3 applies SFT on curated reasoning traces, followed by DPO preference tuning and RLVR using Group Relative Policy Optimization. The post says the combined SFT and DPO setup is a better starting point for RL than SFT alone or direct RL from the base model.
The pipeline offers a concrete example of how data curation and preference tuning can shape reasoning-model RL training.
- 1 year ago @gm8xx8
RLoop Uses Iterative Policy Initialization for RL
RLoop alternates reinforcement-learning exploration with RFT consolidation from filtered expert trajectories. The post reports gains over RL on Qwen-2.5-7B-Math with DAPO-17K, including about 9% higher Avg@32 and over 15% higher Pass@32.
The approach offers a policy-initialization strategy for reducing overfitting and forgetting in RLVR pipelines.
- Reinforcement learningArticle
Why RL Training Can Be More Fragile Than SFT
The post argues that SFT and RL can share a loss form but differ in practice because RL depends on noisy feedback and more complex rollout, log-probability, and reward-shaping infrastructure. It says these factors can destabilize training.
It highlights data-quality and infrastructure risks engineers should monitor in RL training systems.
- 1 year ago @gm8xx8
RLAC uses an adversarial critic for free-form generation
RLAC trains a generator and adversarial critic with an external validator, checking the most likely failure rather than everything. The post reports higher factuality with 5.7× fewer calls and better code scores using 9% of the data and 97.5% fewer test cases.
The approach may reduce validation costs while improving factuality and code scores.
Supervised Reinforcement Learning for Step-Wise Reasoning
The post introduces a framework called Supervised Reinforcement Learning, using expert trajectories to teach small language models to reason through difficult problems. The linked paper is titled “Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning.”
The paper may be relevant to engineers exploring trajectory-based training methods for language-model reasoning.
- 1 year ago @RidgerZhu
Scaling Latent Reasoning with Looped Language Models
The paper scales looped language models to 2.6 billion parameters and trains them on more than 7 trillion tokens. The author reports performance on par with state-of-the-art language models two to three times larger.
It offers engineers a result on scaling language models through a looped architecture.
Socratic-Zero co-evolves agents to generate reasoning data
Socratic-Zero uses Teacher, Solver, and Generator agents to create reasoning data from 100 seed questions. The post reports that Socratic-Solver-8B improves by 20.2 points over prior data-synthesis methods across seven math reasoning benchmarks.
The paper describes an approach to generating reasoning curricula without human-labeled data.
RE-Searcher uses goals and self-reflection for robust search
RE-Searcher is a goal-driven, self-reflective search agent for language models. It sets a search goal and checks whether retrieved evidence meets it, aiming to resist misleading cues in noisy search environments.
Its goal-setting and evidence checks offer an approach to making search agents more resilient to noisy results.
- 1 year ago @jiqizhixin
DeepMind studies autonomous discovery of reinforcement learning algorithms
A Nature paper titled “Discovering state-of-the-art reinforcement learning algorithms” reports that AI can autonomously discover better RL algorithms. The study was led by David Silver.
Algorithm discovery could offer engineers a way to develop new reinforcement learning methods.
arxiv.org
