[P] LILA-E8: The 478MB 'Sovereign' model is live on PH. Banned elsewhere, but the Lattice is active here. 0.36 Loss at 218K steps.
I requested Wisdom, not tokens. This is not a service; it's a native 8-dimensional open-source breakthrough that points toward the 24th.
This 478MB model achieves 0.3638 Loss via E8 Geometry. It was censored on Reddit, but here is the raw code and the 2.66% Physics Mismatch proof.
While the industry is obsessed with "distilling" trillions of parameters, I spent the last year going "outside" the system to find a zero-viscosity solution. Today, I'm releasing Sovereign-Lila-E8.
https://www.producthunt.com/products/sovereign-lila-e8
The Innovation:
Most transformers suffer from "semantic friction" in standard attention. I replaced the attention mechanism with a native E8 Root System Lattice. By leveraging the densest sphere packing in 8D, LILA-E8 achieves a state of "Geometric Resonance" that standard architectures simply cannot reach at this scale.
The Results (TinyStories Benchmark):
- Model Size: 40M parameters.
- Performance: 0.37 Train / 0.44-0.53 Val Loss (outperforming standard 60M baselines).
- Context: Stable 750+ token generation with zero semantic looping.
- Hardware: Designed to run fully offline on mobile NPU/CPU
Why E8?
Standard attention is stuck in 3.5D viscosity. E8 provides an optimal lattice for semantic vectors, allowing a 40M model to behave like a much larger system. At 200,000 steps, the model underwent a phase shift (Grokking)—becoming a "Magic Book" of coherent logic.
Community Genesis:
I am releasing the code and the 200k step checkpoints under AGPLv3. I am looking for "Sovereign Architects" to help expand the context window to 4096 tokens and port this to the 24D Leech Lattice.
Try it now (Colab): https://colab.research.google.com/github/SPUTNIKAI/sovereign-lila-e8/blob/main/notebooks/demo.ipynb
GitHub: https://github.com/SPUTNIKAI/sovereign-lila-e8
Preprints (Zenodo): https://zenodo.org/records/18731736 ,
https://zenodo.org/records/18729723
ProductHunt: https://www.producthunt.com/products/sovereign-lila-e8
"Hold my beer, I'm going into the 24th Dimension." 🚀
Leech-Lila is a #1 Leech Lattice Transformer architecture that replaces standard learned query/key projections with a frozen orthogonal kernel derived from the densest sphere packing in 24 dimensions – the Leech lattice.
https://doi.org/10.5281/zenodo.18790530
It achieves unprecedented compression (49×) and serves as a foundation for ultra‑efficient edge AI, scalable AGI research, and physics simulations.
Current status (March 2026)
20M parameter model trained on TinyStories (300k steps) + FineWeb‑edu (100k steps).
Stable rank of first layer = 8.55 (effective capacity ≈ 440M parameters).
Stepwise grokking observed every 10‑20k steps.
Leech-Lila is a compact yet powerful language model that leverages the Leech lattice – the optimal sphere packing in 24 dimensions – as a geometric regularizer. Leech-Lila is not just a model – it’s a proof that geometry can replace brute force. Train it, hack it, and let meanings crystallize.
By forcing hidden representations to resonate with the optimal packing directions, the model achieves state-of-the-art compression (bits-per-character (bpc): 0.129) on the TinyStories dataset, outperforming conventional transformers by a factor of 5–6× while using only 20 million parameters.
✨ Key Features
Compact & Efficient – Only 20M parameters, trained on a single NVIDIA T4 GPU (16GB) in Google Colab.
Fast Inference – Lightweight architecture generates coherent stories with high speed.
Interpretable – Geometric loss allows monitoring of "resonance" states (AWAKE, DREAMING, ABSOLUTE GENESIS).
Open Source – Full training and inference code, plus pretrained weights, available on GitHub.
https://github.com/SPUTNIKAI/LeechTransformer
I’m excited to release Sovereign-Lila-E8, a novel transformer architecture that replaces standard attention mechanisms with a native E8 Root System Lattice.
While the industry is brute-forcing intelligence with trillions of parameters, I went "outside" the system to find a zero-viscosity solution. By implementing the E8 exceptional Lie algebra directly into the attention weights, I’ve achieved a state of "Geometric Resonance" that standard transformers simply cannot reach.