Source
Falcon-LLM Team (2026), LIT-119 — the Falcon-H1-Tiny technical blogpost.
The failure it fixes
When the tool-calling mix included sources carrying long chain-of-thought traces between the calls, the 90M model produced repetition loops instead of function calls — stuck on a phrase or a partial trace — visible directly in BFCL-v3 generations. Removing all reasoning and thinking content from those sources and keeping only the direct tool-calling examples fixed it immediately. The authors’ reading, following Pipis et al. (2025, LIT-123) on why distilled small models loop more than their teachers: when the training distribution carries reasoning patterns beyond the model’s capacity, the model learns the cheaper pattern, which is repetition. The same failure appeared in the 90M reasoning model, which is more prone to the repetition trap than the 0.6B one.
The observation and the fix both come from LIT-119, the Falcon-H1-Tiny blogpost, which states the lesson outright for its tool-calling model — chain-of-thought traces cause repetition loops rather than better reasoning at this scale, so train on pure function-calling data. It reports the loops and their disappearance qualitatively, not as a before-and-after score.
Conditions
This is about models around 100M trained for a structured target other than reasoning. It is not a claim against reasoning traces as such — the reasoning models in the same release are trained on nothing else — but a claim that a tiny model cannot carry both, and that the traces are what to drop when the target is the call.
Known implementations
- Falcon-H1-Tiny-Tool-Calling