This page carries the protocol, ablations, and provenance behind the Lite-Beam Space. The model card is the short, positive version; this is the audit trail.
The public AA-Omniscience split has 600 questions. The 200-question dev set was used throughout development. The other 400 were untouched until the release run. The test protocol was: minimal reasoning (8), the AA answer prompt, Beam as the judge with AA's grader template, and the same question sample for every arm.
| arm | C | I | P | NA | index |
|---|---|---|---|---|---|
| Base Q2 | 112 | 47 | 4 | 237 | 16.25 |
| Software+Law engram | 140 | 45 | 7 | 208 | 23.75 |
The base 400 is the closed-book rows from the release run plus a separate 133-item base run on the Software+Law domains. The release arm is one 400-item run. Paired: +7.5, 95% CI [+3.5,+11.8], 38 better / 10 worse, exact sign p = 0.0001. On the 133 Software+Law items: 40.60 vs 18.05, +22.6, 95% CI [+10.5,+34.6], p = 0.0001.
The route was selected on dev, then frozen before the 400 test. Development showed the software and law tables were the only verticals whose retrieval coverage translated into answer gains. Health did not translate at this prompt; OER lost to wiki and to closed-book on its 100-item set. The test was not used to change the route.
| domain | release action |
|---|---|
| Software Engineering | software + wiki table |
| Law | law + wiki table |
| Health | closed-book |
| Finance | closed-book |
| STEM | closed-book |
| Humanities | closed-book |
The quant was compared to the FP8 teacher on 16 × 2048 held-out tokens with top-64 logprobs, same reference and engine. Lite-Beam KLD 0.196, Unsloth UD-IQ2_XXS 0.205; same top-1 91.5% vs 91.3%; PPL ratio 1.114 vs 1.125; Lite-Beam wins 14/16 chunks. The map is all 256 experts, gate/up IQ2_XXS, down IQ2_S, attention Q6_K, router F32.
AIME 2026: full 30 questions, minimal reasoning (8), 1,024-token cap: 9/30 (30.0%). GPQA Diamond: full 198 questions, minimal reasoning (8), 512-token cap: 92/198 (46.5%). These are local short-answer screens on the Q2 artifact; the published FP8 rows are separate full-reasoning runs from Reflection's card. The short screens are not paired with the full max-effort engram arms below.
GPQA Diamond 198 with the local category-routed engram at max effort / 32K: 173/198 (87.4%) final. The completed retry set was 25/39; 194 responses are valid (173/194 = 89.2%), with 3 HTTP 500 errors and 1 length cap unresolved. AIME 2026 at max effort / 32K: 29/30 (96.7%) final; four of the five old 8K caps were recovered correctly and the fifth hit the 32K cap. HLE text-only multiple-choice 560 is running at max effort / 32K: 82/560 processed, 28 correct so far (5 HTTP errors). These full-arm values are not paired with the short base screens; the effort and token budgets differ. The 20-item GPQA early pair (12/20 base, 13/20 engram) was only a direction check, not a headline.
Two HLE text-only multiple-choice probes were run on the Q2 artifact. High effort / 2,048 tokens: 0/10 correct, 7/10 length-capped. Max effort / 8,192 tokens: 1/10 correct, 3/10 still length-capped. On the same 10 questions with the local category-routed engram (health / software / OER / wiki), max effort / 8,192: 4/10, with 2/10 length-capped. This is a small diagnostic, not the full HLE 560 score.
GPQA and AIME are final. The full HLE 560 run is in progress (82/560, 28 correct at this snapshot). The card and this page will be updated as HLE advances. HLE is not final.
The FP8 column on the model card is Reflection's published table from the Beam release blog: AA-Omniscience 8.8, AIME 2026 98.0%, GPQA Diamond 90.3%, HLE no-tools 36.0%, LiveCodeBench 88.1%, IFBench 80.7%. Those are not runs of this GGUF and use a different harness, sample, and judge. The model card keeps them as a reference column; this page is the boundary between the two.
SimpleQA-verified dev 200, minimal reasoning: base 6.5; wiki engram k=3 50.5. The wiki table is an optional local add-on and is not part of the model repo. The result is direction, not a model-only claim.
Software, Law, and OER tables plus the retrieval code are in tablinum/beam-engram-tables. The Software table is 836/842 DevDocs documents; the five live missing versions were not re-indexed and gtk~3.22 was a 404. The health table is built locally but is not redistributed: its sources include NCBI StatPearls/Bookshelf and publisher-copyrighted MedRAG textbooks/PubMed abstracts whose terms do not permit us to publish the passages. The health build recipe is included in the dataset so the table can be reconstructed locally under the source terms. Nothing from the health table is sent to a remote endpoint.
# model + table links https://huggingface.co/tablinum/Lite-Beam-1.0-501B-A23B-IQS_XXS https://huggingface.co/datasets/tablinum/beam-engram-tables # llama.cpp llama-server -m Lite-Beam-1.0-501B-A23B-IQ2_XXS.gguf --alias beam-1.0 \ -ngl 99 -ncmoe 99 --load-mode none -t 38 -tb 38 -c 131072 -b 4096 -ub 4096 \ -ctk q8_0 -ctv q8_0 -np 2 --kv-unified -fa on --jinja -ts 3,1 \ --cache-ram 8192 --host 0.0.0.0 --port 18360 --no-webui # native harness, local GGUF mirror --api openai-chat-completions --base-url http://127.0.0.1:18360/v1 \ --model beam-1.0 --context-window 131072 --reasoning-effort medium
Return to the Lite-Beam Space or the model card.