Sparse by design
How can Mixture-of-Experts architectures increase model capacity while limiting the work required for each token?
Guild Labs / research initiative
A smaller footprint.
A larger possibility.
How much intelligence can we build with less? Guild Labs explores the relationship between model architecture, numerical representation, and the machines that run them.
Research Target Sparse MoE + Ternary
Ideal log2(3) 1.585 bits/weight
Proposed Family Eon (Grant Pending)
Systems Base Guild Core v0.x
Heterogeneous MoE execution pipeline Conceptual diagram · measured metrics above
01 / Open questions Efficiency, from the
Efficiency, from the
model to the machine.
Less precision.
More intention.
What can ternary and low-bit representations offer when we consider model quality, training, and hardware execution together?
Local, by nature
How should model design change when GPU memory is limited, system RAM is plentiful, and data stays on the user’s machine?
02 / Proposed model family Introducing an ambition.
Introducing an ambition.
Eon.
Eon is the name of our proposed model family. Our interests include efficient large language models, sparse MoE architectures, and ternary representations.
The architecture, scale, and release schedule are not finalized. There are no released Eon checkpoints or published Eon research results to announce.
Research grant decision pending. We are awaiting a decision; funding has not
been announced as awarded.
03 / Representation study
What does a bit buy?
A simple way to explore weight storage. These are mathematical estimates for hypothetical parameter counts, not Eon specifications or runtime memory requirements.
Illustrative ternary state {-1, 0, +1}
+1 0 -1 0 +1 0 0 -1 +1 0 -1 +1 0 +1 -1 0 0 +1 -1 0 +1 0 -1 0 Storage = parameters × bits ÷ 8 ÷ 2³⁰. Ternary uses the ideal log₂(3) ≈ 1.585-bit information bound. Packing, scales, mixed precision, activations, KV cache, and runtime workspaces add overhead. Storage reduction alone says nothing about quality or speed.
Our systems work is open.
Guild is where we develop and measure local inference infrastructure today.