Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
appvoid 
posted an update about 24 hours ago
Post
71
small reasoning models are overrated, these little ones just doom loop a lot by default. good data will always be the moat when training or finetuning small models and latest sota models like fable 5 and gpt 5.6 are increasingly making this a lot easier to do.

I think the emphasis on data quality is well placed. Smaller models seem much less forgiving of noisy or inconsistent training data, so improvements in curation may end up having a larger impact than simply adding more parameters.

That raises an interesting question: do you think the looping behavior is primarily caused by the training data, or do distillation and quantization play a larger role as models get smaller?

·

I think is the second thing. A small model that enters a strange context may produce a relatively flat distribution. It does not have a strong idea of what should come next but still it has to pick something up. Anti-repetition behavior from data consumes capacity, the less capacity, the more likely your model will suffer from this bias.

That's a simplified view I have on self-reinforcing degeneration mechanism which you can read about here btw: https://arxiv.org/abs/2109.08705