I use to think small models needed more handholding but I think it’s more a question of age. Newer models do fine with less handholding regardless of size.
Yesterday for fun I tried Ornith 9G on my 16GB M1 with OpenCode. It took 3.5 hours do so what Kimi does in a minute (prompt: “analyze this codebase front end and back end and explain the auth mechanism”), but the subtlety and nuance it showed in its response was very surprising to me - I would have lost the bet if someone asked me if a 9G model could do this.
it is indeed surprising how much a small model can do! As you said, for specific tasks like coding, the training data quality and new architecture of newer models probably beats model size
I don't think so. Having build a custom agent with Anthrpic's SDK (in Go) and even running the agent on smaller quantized (non-frontier) models using Ollama, I seriously doubt you'd get good results. I read the same blog post, and I get the impression that the larger model with greater capabilities is able to infer more accurately without being constrained by large(r) system prompts, but I don't think this works so well for smaller less capable models.
yes. I think there is a threshold of some sort where if a model pass it can tolerate less system prompts. For example, same short prompt could work with deepseek v4 flash but not so much with Qwen 3.6 27B.
I use to think small models needed more handholding but I think it’s more a question of age. Newer models do fine with less handholding regardless of size.
Yesterday for fun I tried Ornith 9G on my 16GB M1 with OpenCode. It took 3.5 hours do so what Kimi does in a minute (prompt: “analyze this codebase front end and back end and explain the auth mechanism”), but the subtlety and nuance it showed in its response was very surprising to me - I would have lost the bet if someone asked me if a 9G model could do this.
it is indeed surprising how much a small model can do! As you said, for specific tasks like coding, the training data quality and new architecture of newer models probably beats model size
I don't think so. Having build a custom agent with Anthrpic's SDK (in Go) and even running the agent on smaller quantized (non-frontier) models using Ollama, I seriously doubt you'd get good results. I read the same blog post, and I get the impression that the larger model with greater capabilities is able to infer more accurately without being constrained by large(r) system prompts, but I don't think this works so well for smaller less capable models.
yes. I think there is a threshold of some sort where if a model pass it can tolerate less system prompts. For example, same short prompt could work with deepseek v4 flash but not so much with Qwen 3.6 27B.
[flagged]