TestGeniy 4K Context Reasoning Model

TestGeniy is a compact causal language model focused on mathematical reasoning, formal logic, and helpful text interaction.

This main release is the validated 4K-context anchor. It is the safe production checkpoint after context-extension and regression testing.

Release summary

• Context window: 4096 tokens.
• RoPE: extended from 2048 to 4096 positions using the original theta value 500000.
• Attention: sliding attention with block size 1024 and global attention in layers 3, 7, 11, 15, 19, and 23.
• Weights: validated logic_small_scope_step080 anchor, with context buffers extended to 4096.
• Evaluation questions were kept out of training.
• This main release does not include the rejected synthetic-CoT candidates.

Validation

The 4K model remained finite on full 4096-token forward passes and answered a 3157-token long-context probe correctly.

Fixed paired reasoning gate, 12 examples per dataset:

Benchmark	Anchor	4K main
GSM8K	2/12	2/12
MATH-500	2/12	2/12
ARC-Challenge	5/12	5/12
FOLIO	4/12	4/12

The release is a verified context-capability improvement with no measured regression on this gate. It is not presented as a benchmark-accuracy improvement.

Intended use

Use this checkpoint for compact English reasoning experiments, long-context prompting up to 4096 tokens, and further controlled fine-tuning.

Limitations

This is a small research model. It can produce incorrect reasoning or answers, especially on difficult mathematics and formal logic. The benchmark gate above is a regression gate, not a broad capability estimate.

Provenance

Base checkpoint: logic_small_scope_step080 from this project. The published weights contain no benchmark questions and no synthetic-CoT training data.