#LFM3-1.7B launch: which benchmark numbers to trust, and which jobs justify a small model

Liquid AI released the LFM3 family on Oct 27, 2026. The text model, LFM3-1.7B, has 1.73B parameters, takes about 1 GB at 4-bit, and averages 49.37 on the company's benchmark table. That beats the 9B-parameter Qwen3.5 9B at 47.24. A number like that is easy to repost. It is cheaper to check how the table was built before you do.

I read the full benchmark table row by row. This post says which rows you can use for a decision, which ones need a discount, and ends with a checklist for whether your own workload should move to a small model.

One scope note first. I tried to build llama.cpp locally and download the GGUF to run it, but the execution environment did not allow it. This post contains no speed, memory or output that I measured myself. Every figure comes from Liquid AI's launch post and the Hugging Face model card. Anything else is my own recalculation from the same table, and I label it as such.

#What shipped

LFM3 is a family. The official post lists six 2B-class models in the first batch, all branched from one LFM3-1.7B-Base checkpoint:

The weights are open (base and post-trained versions) on Hugging Face. When I read the LFM3-1.7B page on Oct 28, it showed 6,311 downloads and 212 likes. Upstream llama.cpp already has an LFM3 implementation file, and the GGUF repo has Q4_K_M, Q8_0 and other quantizations, so you should not need to convert anything yourself.

Text model specs: 24 layers, of which 13 are gated short convolutions, 5 are sliding-window attention with a 256-token window, and 6 are global attention. Vocabulary is 128,000, context is 256K, pretraining used 22T tokens, and a 3.5T-token mid-training stage stretched the context to 256K. Liquid says this hybrid layout came from a hardware-in-the-loop architecture search. The number of devices measured during the search went from two for LFM2 to 14 device and backend combinations, including phones, a Raspberry Pi 5, laptops and a Qualcomm Hexagon NPU.

#How I read the official benchmark table

The table has 29 rows. The comparison models are LFM2.5 2.6B, MiniCPM5 2B, Gemma 4 E4B, Qwen3.5 4B and Qwen3.5 9B. Four things to know before looking at any single row.

The conditions are published. Every model ran on vLLM in BF16 at its full context length (262,144 for LFM3 and Qwen3.5 4B and 9B, 131,072 for the rest). Unless noted, temperature was 0.6 and top_p 0.95. BFCL, ToolSandbox and Claw-Eval used greedy decoding, and agent tasks had a turn cap. That is enough detail to reproduce a run, which is more honest than a bare bar chart.

But the vendor ran it. The launch post has no third-party reproduction. Liquid AI chose the opponents and the benchmarks. No small Llama or Phi model is in the lineup. I noticed that while reading; it does not mean those models would win.

"49.37 beats a 9B model" needs unpacking. The average is an unweighted mean of 29 scores. I recomputed it, and both 49.37 and 47.24 check out. But within that same table, LFM3 is higher than Qwen3.5 9B on 14 rows and lower on 15. The wins cluster in a few categories:

The biggest leads are AutomationBench (26 points) and WebArena-Verified and LiveClaw (about 10 points each). Those are the tasks the final training stage targets directly, by running tasks inside real agent harnesses. My read is that these wins come from the training direction, not from the model suddenly being smarter overall. Liquid itself concedes that agentic coding and knowledge-heavy tests (SWE-bench, GPQA Diamond) favor the larger models.

Numbers I would discount.

First, on RULER 128K, LFM2.5 2.6B scores 1.15 while LFM3 scores 65.56. A 1.15 looks like the old model simply broke on that test. I can't tell why from public data, so I won't call it a 57x long-context improvement. The more meaningful comparison is Qwen3.5 4B at 84.63 and 9B at 87.11, both almost 20 points above LFM3.

Second, the three AA-Omniscience rows should be read separately. LFM3's accuracy is 7.07, the lowest of the six models. Its non-hallucination score is 92.93, the highest. The composite index of 0.50 therefore beats Qwen3.5 4B at -57.17.

My interpretation is that it declines to answer more often instead of guessing, but that is inferred from the column names; check the metric definition. If your app needs a model that answers when it knows and says so when it doesn't, this row helps. If you want more correct answers, it is telling you not to rely on the model's memory.

Third, all three agentic-coding rows (SWE-bench Verified, SWE-bench Multilingual, Terminal-Bench 2.1) are below Qwen3.5 9B, and Terminal-Bench 2.1 is only 12.92. The model card also says outright that the model is not recommended for agentic coding or complex multi-step reasoning.

#The edge-device numbers

Liquid measured on three devices: AMD Ryzen AI Max+ 395, Apple M5 Max and iPhone 17 Pro. Setup was llama.cpp, Q4_0 GGUF, 32K context, batch size 1, 100 output tokens. The claim is that LFM3-1.7B decodes 1.8x to 2.8x faster than the runner-up, prefills at least 2.5x faster, and uses the least memory.

Two caveats. LFM3 ran on a llama.cpp build with LFM3 support, while the other models ran on release b11744, so the two sides are not the same build. And an advantage at 32K context does not guarantee the same gap when you stuff in a 100K-token document.

A number I find more practical is KV cache: 1.6 GB at 128K context, against 4.3 GB for Qwen3.5 4B and 5.6 GB for MiniCPM5 2B. Weights of about 1 GB plus 1.6 GB of cache puts a long-context process near 2.6 GB. That is my arithmetic, not a measurement, and real usage will vary by framework. If it holds, an 8 GB laptop can keep the model loaded next to your normal work. A 4B-class competitor under the same conditions spends 4.3 GB on KV cache alone.

For servers, Liquid reports roughly 3.6 billion tokens per day on a single NVIDIA B300 with SGLang, BF16 and 1,024 concurrent requests. That figure means little for an individual. It matters for a team that wants to self-host a lot of background agents.

#License: can you put it in a product

The model card names LFM Open License v1.0, and the Hugging Face metadata says other. I read the full LICENSE. The key part is section 5: commercial use is licensed only if you, or your legal entity, have annual revenue not above US$10 million. Above that, commercial use falls outside the license. Qualifying nonprofits doing non-commercial or research work are exempt from the cap.

In practice, individuals, startups and small companies can use it directly. A company above US$10 million in revenue needs a separate agreement with Liquid AI. I am not a lawyer; have your counsel read the original text.

#Which jobs justify a small model

Where I think it is worth it:

• Workflows built on tool calling and structured output. BFCL V4 at 71.39 and IFStruct at 93.72 both lead every opponent in the table, and these tasks have fixed output formats that are easy to check automatically.
• Anything that has to run locally. If data can't leave the machine, or the model has to work offline on a phone or laptop, a roughly 1 GB weight file makes that plausible.
• Running many background agents at once. Liquid's pitch is that tokens are effectively free, so you can leave agents running. Whether you save money depends on what you use today, and I have no numbers for that.
• A clear task you are willing to fine-tune for. Liquid says each model can be fine-tuned on a single GPU and links Colab notebooks for SFT, DPO and GRPO. Their wind-turbine blade defect example went from under 10% classification accuracy before fine-tuning to 67% after. That is an official example, and I did not reproduce it.

Where I would not switch:

• Coding agents, especially ones that edit a real repo.
• Question answering that needs a lot of factual knowledge. Liquid's approach is to pair the model with search: HLE goes from 7.17 without tools to 13.90 with them, nearly double, but the absolute score is still low and still below Qwen3.5 9B at 17.50.
• Retrieval and summarization over very long documents. On RULER 128K it trails the Qwen3.5 models by almost 20 points.

#What I don't know yet

• Real-world quality outside English. The model card lists Chinese among supported languages, but the benchmark table is English (Claw-Eval is marked EN) and has no non-English numbers.
• Speed on anything beyond the three test devices, especially ordinary laptops without a GPU and mid-range phones.
• How much agent scores drop on the Q4 quantization. The benchmarks are BF16, the edge speed tests are Q4_0, and the two sets of numbers never share a table.
• Third-party reproduction. The model is one day old, and no independent evaluation exists yet to confirm similar numbers.

#If you want to try it yourself

1. Download the Q4_K_M file from LiquidAI/LFM3-1.7B-GGUF and open it in a llama.cpp build with LFM3 support, or in LM Studio. An older build may not recognize the architecture; I did not verify which version added support.
2. Use the sampling settings from the model card: for general use temperature=0.1, top_k=50, repetition_penalty=1.1; for math and science temperature=1.0, top_p=0.95, top_k=20.
3. By default the model prints its reasoning first. You can turn that off with enable_thinking=False, though Liquid warns the behavior is unstable on tasks that need reasoning.
4. Take the three tasks you hand to a model most often and run each ten times. Count format errors and retries, then decide. None of the 29 rows in the official table is your task.

Sources: Liquid AI, "Introducing LFM3" (Oct 27, 2026); Hugging Face LiquidAI/LFM3-1.7B model card and LICENSE (read Oct 28, 2026).