maxact-fast: a general residual-direction → text inverter

It's all just probes. SAE encoder columns, cluster classifiers, hand-crafted concept vectors — every one of them is a unit direction in a language model's residual stream. This project trains Qwen3-8B (LoRA) to take any such direction — injected activation-oracle-style into its own layer-1 residual stream — and write the text that maximally activates it at layer 27.

Prior work (our SAE line) showed a generator trained on one SAE's features beats exhaustive 500M-token corpus search on ~30% of held-out features at best-of-16, and — via a Sonnet-judged audit of all 65k features — that corpus max-activating examples alone mislabel or under-specify ~72% of features (56.8% refined, 3.6% outright contradicted, with 918 contradictions surviving adversarial re-testing). The bottleneck is conditioning diversity: one SAE = 65k directions.

The direction firehose

1. Embed UltraFineWeb (streamed) with BGE → k-means K clusters (pilot K=10k, target K=1M).
2. Sample cluster pairs (A,B), train a logistic-regression probe on the model's own layer-27 residuals; keep the direction only if it validates (held-out AUC ≥ 0.9 and cluster A actually projects high on it — no junk directions).
3. SFT target = corpus texts nearest A's centroid: "given direction w, write text like this."
4. Pretrain on millions of validated (direction, text) pairs → post-train on SAE-feature max-activating corpus examples → RL.

RL: Dr. GRPO, no KL

Advantage = r − group_mean (no /std — the Dr. GRPO fix), loss token-summed over a global batch normalizer, importance ratios TIS-capped, no KL leash (it blocked exactly the rare-feature specialization we want) — diversity comes from an explicit entropy bonus (maximize r + β·H) plus fluency/distinct gates. Rollouts via vLLM (vllm-lens per-request steering injection). Reward = max-over-positions dot of the standalone re-tokenized generation through the clean base model. Fair by construction.

Why

If the cross-uplift bet lands — cluster-probe pretraining transferring zero-shot to SAE features — you get a universal activation-maximization oracle: hand it any probe you trained, get back legible text showing what that probe actually fires on. Feature auditing, probe debugging, and "what did my classifier really learn" become one generate() call.

Code in this repo; SAE-line results: qwen3-8b-sae-maxact-lora. MATS interpretability project (Neel Nanda stream).