Cerbeto: a little brain that goes to school
Disclaimer: this article was written with the help of AI. Cerbeto is a real project I am actively working on; the numbers below describe its state on 31 October 2026.
Cerbeto (Esperanto: cerbo, "brain", plus the diminutive -et-, so "little brain") is my attempt to build a small language model entirely from scratch and then raise it rather than just train it. The student network starts at about 8 million parameters with randomly initialised weights, its own tokenizer and its own architecture. No pre-trained model is used as a base. It is meant to learn English and simple reasoning the way a child goes through school: nursery, preschool, first grade, second grade. On the way it grows, up to a hard limit of 30 million parameters.
The school metaphor is taken literally. In the nursery the student reads a "school library" of ready-made children's corpora (BabyLM, TinyStories, Simple English Wikipedia and a slice of FineWeb-Edu filtered to an early-grade reading level), about 628 million tokens for stage 0 alone. After that, lessons begin. A large LLM running locally in Ollama acts as the homeroom teacher: it asks questions, explains mistakes and raises the difficulty. Every day follows a rhythm. During the day the student takes lessons and its experiences go into a fast "hippocampus" memory. During sleep they are consolidated into the weights, with replay of older material and synaptic consolidation (EWC) to limit forgetting. Then comes an exam. A new version of the student is promoted only if it passes; otherwise the system rolls back automatically. Every weight change produces an immutable, reproducible version.
A few rules keep the experiment honest. Truth comes from code, not from the LLM: lesson tasks come from generators that know the correct answer, and the teacher's explanations enter the training data only after they are checked against that answer. Exams are generated from reserved seeds and never reach the teacher or the training set. They come in three variants: known task types, harder ones than in training, and new combinations. The goal of the MVP is a measurable improvement on those new tasks after automatic learning cycles, with forgetting of earlier skills kept within 2 percentage points. A control student, fixed at 30M parameters from day one, goes through the same school to show whether growing the network actually helps. The "biological" mechanisms (growth, pruning, sleep, neuron renewal) are modelled at the system level, and each one has to point to a known technique in the ML literature and have its own metric.
Everything runs locally and offline. The brain is Python with PyTorch on Apple's GPU via MPS (a MacBook Pro M1 Pro with 32 GB is the reference machine). It uses fp32 or fp16, because the M1 has no hardware bf16. A Ruby on Rails 8 panel will let you configure teachers, watch lessons live, compare and roll back versions, and chat with the student to correct it yourself. The nursery is done once, centrally. After that, the plan is a package that lets anyone continue raising their own independent copy on a 16 GB laptop, with no internet and no API bills. So far, specification 0001 is delivered: the library is split into shelves by difficulty, and the student has its own byte-level BPE tokenizer with 8,192 tokens (1.24 tokens per word on TinyStories), backed by 27 tests. Next comes the student architecture: a transformer with a recurrent core for "thinking in a loop", plus operators for growing and pruning it.
Cerbeto has a second role: it is the public reference project for Rondo, the methodology I describe in Rondo with Context Driven Development. The repository is built the CDD way. The README describes only the "here and now": domain, architecture and critical path. Every change starts as a numbered specification in specs/ with Rondo's nine sections, from TL;DR to control gates. CHANGELOG.md records what was actually delivered, and the tests are the proof, derived from each specification's gates. An engineering manifesto sets the rules every specification must follow. If you want to see what Rondo looks like on a real, non-trivial project, rather than on a slide, this is the place to watch. I will write follow-ups as the student moves from nursery to its first exams.