Heatmap Leaderboard
๐ฅ
21
Organization Activity Heatmap
master PRO
fantos
AI & ML interests
None yet
Recent Activity
liked a Space 1 day ago
ginigen-ai/Heatmap-Leaderboard liked a model 1 day ago
FINAL-Bench/AETHER-7B-7Attn-base liked a dataset 1 day ago
FINAL-Bench/Aether-7B-5Attn-checkpointsOrganizations
liked a
Space 1 day ago
liked a
model 1 day ago
liked a
dataset 1 day ago
liked a
model 1 day ago
liked a
Space 1 day ago
liked 2
models 1 day ago
upvoted a collection 1 day ago
reacted to SeaWolf-AI's post with ๐ 1 day ago
Post
4747
A small gift for anyone building or studying foundation models.
Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing โ so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.
What you can do with it:
๐ Rebuild it from scratch, or fork the recipe for your own model
๐ฌ Study a real heterogeneous-attention MoE โ 49 layers place 5 attention mechanisms on a 7ร7 Latin square, arranged as a clean, attributable ablation
๐ Trace training dynamics across the released checkpoints (110k / 115k / 162k)
It's a modest 6.59B model, and an honest one โ the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. ๐ค
๐ Full write-up โ
[blog] ยท https://huggingface.co/blog/FINAL-Bench/opensource-llm
๐ฆ 5 Attention Base ยท FINAL-Bench/Aether-7B-5Attn
๐ฏ 5 Attention Instruct ยท FINAL-Bench/Aether-7B-5Attn-it
๐ 5 Attention Live demo ยท FINAL-Bench/Aether-Sovereign-AI
๐ฆ 7 Attention Base ยท https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
๐ฆ 11 Attention Base ยท FINAL-Bench/Aether-6B-11Attn-base
๐งฌ Collection ยท https://huggingface.co/collections/FINAL-Bench/aether-foundation-model
#opensource #LLM #MoE #reproducibility #Apache2
Most "open" models hand you the weights and stop there. With Aether-7B-5Attn we wanted to hand over the whole thing โ so you can actually learn from it, reproduce it, and build on it: the data recipe, the training code, every hyperparameter, the complete logs, and the intermediate checkpoints. All Apache-2.0, reproducible byte-for-byte.
What you can do with it:
๐ Rebuild it from scratch, or fork the recipe for your own model
๐ฌ Study a real heterogeneous-attention MoE โ 49 layers place 5 attention mechanisms on a 7ร7 Latin square, arranged as a clean, attributable ablation
๐ Trace training dynamics across the released checkpoints (110k / 115k / 162k)
It's a modest 6.59B model, and an honest one โ the limitations (no KV-cache in this build, small scale) are written right in the card. We're not claiming it's special. If any piece of it saves you time or teaches you something, that's exactly what we hoped for. ๐ค
๐ Full write-up โ
[blog] ยท https://huggingface.co/blog/FINAL-Bench/opensource-llm
๐ฆ 5 Attention Base ยท FINAL-Bench/Aether-7B-5Attn
๐ฏ 5 Attention Instruct ยท FINAL-Bench/Aether-7B-5Attn-it
๐ 5 Attention Live demo ยท FINAL-Bench/Aether-Sovereign-AI
๐ฆ 7 Attention Base ยท https://huggingface.co/FINAL-Bench/Aether-7B-7Attn-base
๐ฆ 11 Attention Base ยท FINAL-Bench/Aether-6B-11Attn-base
๐งฌ Collection ยท https://huggingface.co/collections/FINAL-Bench/aether-foundation-model
#opensource #LLM #MoE #reproducibility #Apache2
- 5 replies
liked a
model 2 days ago
liked a
model 9 days ago
liked a
Space 9 days ago
reacted to SeaWolf-AI's post with ๐ฅ 9 days ago
Post
5389
๐ต VKUE โ No GPU? Runs anyway.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound โ sweep 34B params/token and an 8 GB card dies at 1โ2 tok/s.
So we ran ONE 34.7B reasoning model โ Ourbox-35B-JGOS, a sparse Mixture-of-Experts โ as the identical weights across the whole hardware spectrum. All measured:
โข B200: 18,057 tok/s (aggregate)
โข 1ร A10G: 126 tok/s
โข 8 GB laptop (RTX 5060): 20 tok/s
โข GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB โ ~11ร less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop โ or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 โ 3.7ร from sparsity alone, ~2ร the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
๐ Article: https://huggingface.co/blog/FINAL-Bench/vkue
๐ต GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
๐ต CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
๐ VKUE leaderboard: FINAL-Bench/VKUE
๐ค Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
โก VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
"Frontier models need a datacenter GPU" rests on a hidden assumption: that the model reads ALL its parameters every token. Decode is memory-bandwidth bound โ sweep 34B params/token and an 8 GB card dies at 1โ2 tok/s.
So we ran ONE 34.7B reasoning model โ Ourbox-35B-JGOS, a sparse Mixture-of-Experts โ as the identical weights across the whole hardware spectrum. All measured:
โข B200: 18,057 tok/s (aggregate)
โข 1ร A10G: 126 tok/s
โข 8 GB laptop (RTX 5060): 20 tok/s
โข GPU-less CPU: 17 tok/s
Why it works: Ourbox holds 34.7B params but only ~3B are active per token (256 experts, top-8). Since decode is bandwidth-bound, a dense 34B moves ~16.7 GB/token while Ourbox moves ~1.45 GB โ ~11ร less traffic. Put the experts in system RAM, keep attention/router/shared on the GPU, and a 34.7B reasoner runs on an 8 GB laptop โ or no GPU at all.
Sparsity alone, proven (same laptop, same quant, ~same footprint): Ourbox-35B (A3B) 20.01 tok/s vs Qwen2.5-32B (dense) 5.36 โ 3.7ร from sparsity alone, ~2ร the best dense-32B on any 8 GB machine. Not a toy: GPQA Diamond 86.4% (maj@8).
Try it live (same prompt, GPU vs GPU-less CPU, live tok/s). Honest scope: one machine's measurements; the CPU path proves it RUNS without a GPU, not that it beats one.
๐ Article: https://huggingface.co/blog/FINAL-Bench/vkue
๐ต GPU vs CPU demo: https://final-bench-ourbox-35b-vkue-demo.hf.space/
๐ต CPU-only demo: https://final-bench-ourbox-35b-vkue-cpu.hf.space
๐ VKUE leaderboard: FINAL-Bench/VKUE
๐ค Model: FINAL-Bench/Ourbox-35B-JGOS-GGUF
โก VKAE (speed): VIDraft/vkae
VKUE is the "runs anywhere" side of our serving line; VKAE the "fast on datacenter GPUs" side. VKAE is fast; VKUE is everywhere.
- 7 replies
liked 2
models 18 days ago
liked a
model 20 days ago
liked a
Space 20 days ago
reacted to ginigen-ai's post with ๐โค๏ธ๐ฅ 20 days ago
Post
10456
๐ง Does your LLM know when it's about to be wrong?
Most leaderboards measure accuracy. We measure metacognition โ whether a model catches its own errors. Benchmark + leaderboard + adapters, all open. ๐
The surprise: even a K-AI #1 model (JGOS-31B-Citizen) is the strongest on multiple-choice traps (trap_rate 0.005 โ ~2 misses in 400) yet blind to its own free-form mistakes (self-confidence AUROC = 0.5, pure random). A tiny base-frozen adapter recovers that signal.
Two independent axes (never compared across a row): โ trap_rate โ does it fall for tempting trap options? (lower = stronger) โก adapter gain ฮ โ how much a lightweight adapter catches errors the model itself misses. (higher = more adapter value)
What's open: ๐ 300+100 trap problems (each with a hidden trap + TICOS type) ๐ 24-model leaderboard ๐งฉ 11 per-model adapters โ adapters, NOT fine-tunes (base stays frozen; the adapter just reads the hidden state โ P(wrong))
Submit any HF model โ auto-scored daily at 09:00 KST and added to the board.
๐ Leaderboard โ ginigen-ai/Metacognition-Leaderboard-Space
๐ Benchmark โ ginigen-ai/Metacognition-Bench
๐งฉ Adapters โ FINAL-Bench/metacognition-adapters-6a42c032e6beb803dd032961
๐ Article โ https://huggingface.co/blog/ginigen-ai/metacognition
Benchmark by ginigen-ai ยท Adapters by FINAL-Bench (Darwin/Chimera platform + AETHER metacognition tech).
Most leaderboards measure accuracy. We measure metacognition โ whether a model catches its own errors. Benchmark + leaderboard + adapters, all open. ๐
The surprise: even a K-AI #1 model (JGOS-31B-Citizen) is the strongest on multiple-choice traps (trap_rate 0.005 โ ~2 misses in 400) yet blind to its own free-form mistakes (self-confidence AUROC = 0.5, pure random). A tiny base-frozen adapter recovers that signal.
Two independent axes (never compared across a row): โ trap_rate โ does it fall for tempting trap options? (lower = stronger) โก adapter gain ฮ โ how much a lightweight adapter catches errors the model itself misses. (higher = more adapter value)
What's open: ๐ 300+100 trap problems (each with a hidden trap + TICOS type) ๐ 24-model leaderboard ๐งฉ 11 per-model adapters โ adapters, NOT fine-tunes (base stays frozen; the adapter just reads the hidden state โ P(wrong))
Submit any HF model โ auto-scored daily at 09:00 KST and added to the board.
๐ Leaderboard โ ginigen-ai/Metacognition-Leaderboard-Space
๐ Benchmark โ ginigen-ai/Metacognition-Bench
๐งฉ Adapters โ FINAL-Bench/metacognition-adapters-6a42c032e6beb803dd032961
๐ Article โ https://huggingface.co/blog/ginigen-ai/metacognition
Benchmark by ginigen-ai ยท Adapters by FINAL-Bench (Darwin/Chimera platform + AETHER metacognition tech).
- 11 replies