Feedback is welcome, especially on useful ablations: digit reversal, place_ids, role_ids, or whether auxiliary features should extend beyond arithmetic spans into ordinary text.
Maksymilian PRO
ucr-max
AI & ML interests
None yet
Recent Activity
new activity 34 minutes ago
MaliosDark/Nexus-Erebus-3M:Attribution and prior-work clarification new activity 2 days ago
UniversalComputingResearch/Atom2.7m:Kudos on Atom 2.7M liked a model 3 days ago
openbmb/MiniCPM5-1BOrganizations
replied to their post 7 days ago
posted an update 7 days ago
Post
117
Published under Universal Computing Research: Atom2.7m, a 2.74M-parameter causal LM experiment with arithmetic-aware numeric representation.
It keeps regular BPE-style text continuation, while adding structure for numeric spans: digit ordering, place_ids, and role_ids.
Atom2.7m reaches 69.24% accuracy on ArithMark2.0.
The question behind the project is narrow: how much of small-model arithmetic performance depends on representation, rather than scale alone?
Model: UniversalComputingResearch/Atom2.7m
Article: https://huggingface.co/blog/ucr-max/atom2-7m-arithmetic-representation
Demo: UniversalComputingResearch/Atom2.7-Demo
It keeps regular BPE-style text continuation, while adding structure for numeric spans: digit ordering, place_ids, and role_ids.
Atom2.7m reaches 69.24% accuracy on ArithMark2.0.
The question behind the project is narrow: how much of small-model arithmetic performance depends on representation, rather than scale alone?
Model: UniversalComputingResearch/Atom2.7m
Article: https://huggingface.co/blog/ucr-max/atom2-7m-arithmetic-representation
Demo: UniversalComputingResearch/Atom2.7-Demo
- 1 reply