xx18/Composition-RL-4B-Depth1_2

Composition-RL-8B

Composition-RL is a data-efficient Reinforcement Learning with Verifiable Rewards (RLVR) approach that addresses the challenge of "too-easy" prompts (those with a pass rate of 1) by automatically composing multiple verifiable problems into a single, harder yet still-verifiable prompt. This method ensures that training signals remain informative throughout the reinforcement learning process, leading to consistent improvements in reasoning capability.

This checkpoint is the 8B parameter version, initialized from Qwen3-8B-Base and trained on the MATH-Composition-199K dataset.

Resources

• Paper: Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models
• Repository: GitHub - XinXU-USTC/Composition-RL
• Collection: Composition-RL Collection

Key Features

• Automatic Composition: Composes multiple verifiable problems into new, more challenging prompts.
• Improved Reasoning: Consistently improves reasoning performance over RL trained on the original dataset across various model sizes.
• Curriculum Learning: Can be further boosted with a curriculum variant that gradually increases compositional depth over training.

Citation

If you find this work helpful for your research, please consider citing the paper:

@article{xu2026composition-rl,
  title={Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models},
  author={Xu, Xin and Bai, Clive and Yang, Kai and Chen, Tianhao and Chen, Yangkun and Liu, Weijie and Chen, Hao and Wang, Yang and Yang, Saiyong and Yang, Can},
  journal={arXiv preprint arXiv:2602.12036},
  year={2026}
}