I know we have strong views on what a truly open model is (open weights, open training data, open training code etc.) but I really like how transparent they’ve been about the training of this model.
The realtime dashboard they shared during training (https://mimo.xiaomi.com/rl/) was an incredible learning and teaching tool for me, and they’ve been unusually comprehensive in sharing details about their methodology (check out that tech report - it's got lots of clever behind the scene tricks like Google or Deepseek writeups) and benchmark scores (even the stuff they didn’t do well on).
If you’re releasing an open model going forward, please consider offering the community more of this transparency!
I was absolutely mind blown when I saw how they were publishing that training dashboard while US models publish 100s of pages of reports (just provide a "copy as MD" button, folks, in the future). I was thinking about doing something similar but did not know how to show it, and this is a perfect example for someone who wants to show whatever they are training, for me it was local training on a consumer GPU.
My dream is to see this like a dashboard for a model trained across distributed machines, like Bitcoin mining, where minted coins are given to people whose machines were used for training. I don't know if they are worth it, but bragging rights alone, like a tag they can put on a website or social media, will be good enough for me.
For (C) -- I think it could additional create jobs funded by government, philanthropic and other private institutions. For example, a government funded museum may already be participating in Wikimedia GLAM projects (e.g. uploading historical images to Wikimedia Commons with complete metadata). Perhaps this type of open source contribution may increase if organisations realise their mission can be better accomplished by contributing this same open data into LLMs, in addition to Wikimedia Commons. If the museum's mission is to educate the public on the history of life in ACMEville, having LLMs be able to provide historical information and images to a prompt of "What is the history of ACMEville?" may be a good pursuit.
I'm sceptical though whether use of LLMs would encourage creation of data that doesn't already exist. For example, if you ask an LLM "What are the top 100 most prevalent flora endemic to ACME National Park", this data may not currently exist _at all_, and to collect, would require paying botanists to do an extensive field survey. If no one has done this work yet--why? Is it relevant to the scientific community, to making government decisions, etc, or just an obscure academic curiosity. There are certainly some journal articles on _other_ national parks describing some of their common endemic flora, but perhaps there was a reason for this. Such as a scientist funded by a one-off government program trying to determine how to preserve or even create habitat for a specific endangered species.
Consider for the prompt of: "What are the top 100 most prevalent flora endemic to ACME National Park"
An LLM may reply: "I couldn't find any journal article or other prior work that may answer this question. Typically such survey field work may cost $X to complete, require expert botanists, and take 6-12 months to complete. Let me know if you want further information on how to find and select a company to conduct such a botanical field survey."
Would this type of LLM response grow the industry of botanical field surveys, or do nothing, perhaps because anyone likely to fund botanical field surveys is already doing so regardless of whatever is happening with AI.
I think all of these can be addressed by being able to understand which machines contributed what to the model. I know that's messy, but if I can state "this information has come from Alice and the consensus is that it's good and right as it aligns with other information from Bob and others, meanwhile this information came from Mallory and stands out as being incongruent with the rest of the information I have in the model", we can identify malicious violations. If we're then able to state either Alice or Bob is one of the copyright holders, and one of them gets paid a little more, the other a little less as the confirmation agent, well, the economics of all of this changes a little.
At the moment we have Annas Archive being paid by frontier labs and rare/second hand books being destroyed in order to support the training regime. If instead we could just pay the publishers and they could distribute royalties to authors...
This dashboard is almost certainly built on verl (https://github.com/verl-project/verl), which comes with a bunch of dashboarding capabilities built in (that doesn't look too dissimiliar to these dashboards).
Not an expert on this but I think the RL runs need to work sequentially? I wonder what the opportunities for distributed execution would be... Maybe parallelizing the benchmark task or inference
> got lots of clever behind the scene tricks like Google or Deepseek writeups) and benchmark scores
Xiaomi MiMo is led by Luo Fuli, a former Alibaba & DeepSeek employee. Perhaps it is due to Luo just how similar Xiaomi's tech & GTM approach is to DeepSeek's.
Thanks so much for sharing this. As someone who mostly watches from the sideline, can you share what you can see in this dashboard that someone like me can't see? Is it the metrics themselves that they measure (the metrics tab is absurdly detailed), something in the notices, or something else I missed?
I might turn this into a blogpost if folks are interested, but my god there is so much clever info in that dashboard.
Here is one really neat bit:
A cutting edge training idea (for agents, it's been used elsewhere for ages) is on-policy RL, basically, it's not enough to say "here is an end to end agentic sequence (including tool calls etc.) that is perfect" you want to say "here is a sequence you might actually have generated that turns out to be correct".
Basically, it's more training efficient to improve models with small tweaks to do more of the right thing they are already doing sometimes than from some perfect oracular "this is the way" answer.
(if you've ever tried to teach humans new skills, you’ve probably noticed this too!)
When you do that, you care about how far the model you are updating (improving) has deviated from the one being used to generate rollouts (agentic rollouts for hard problems can take hours with lots of tool calls, so you can't keep redeploying every slight improvement).
Lo and behold, the dashboard literally has:
partial/avg_staleness (likely the measure of how many micro iterations the "generate answers" model is behind the "improving based on the occasional right answer" model)
train_infer_diff/new_infer/kl (a more direct KL divergence based way of measuring how differently the two models generate tokens)
How cool is that?!
And don't get me started on the clever ideas hiding behind dynsam/avg@n ...
Hold on, isn't that just standard practice for post-training LLMs for agentic use? Give task, generate n rollouts, grade rollouts (either at termination or after each tool call)? Or is the difference that the rollouts are generated ahead of time and then graded? (Of course, then it's not really on-policy.)
The best thing they did is being open about all the setbacks they had to deal with. They logged every restart with a reason, talked about dropping a cyber dataset after it degraded coding benchmarks. Also published real time training loss, benchmark scores after every checkpoint and running cost estimates.
Really the only thing missing was dataset descriptions, the dashboard only had random IDs like "dataset-zrso". I guess it's their lawyers fault.
It'd be great to see a description of even just a subset of training datasets. It feels very much under-reported how much expense is worth investing in preparing and selecting training datasets versus just using masses of random quality unprepared training data. This dashboard appears to be good though in showing the limits quickly reached when throwing parameters and compute at the problem.
For example, if they were to train on Wikipedia dumps, do they consider every article to be the same quality across each language, or have they done more work beyond Wikipedia's own article quality ratings to make training decisions such as "Ignore cebwiki it's machine-generated spam" and "Treat dewiki articles with coordinates within Germany as being higher quality (weight it higher) than their equivalent enwiki articles".
And let's say one of the datasets is all the source code of packages in the Gentoo package repository. Not every software package is a good example of how to write code. You perhaps wouldn't want to train your LLM on 1990s era PHP web application source code as an example of how to write code in 2026. Instead, you'd possibly want to use such PHP web application source code as a negative training example of what _not_ to write. But when training an LLM to detect software bugs, maybe outdated PHP source code is good for training.
Similarly for translation, perhaps UN treaty documents translated into 4+ languages are good translation examples because of high accuracy needed, professional translators being used, and bigger budgets. However this training data would perhaps be a negative training example towards translating chat messages, movie subtitles, etc because it doesn't use everyday slang and could result in output of nonsense such as "Pending Your Excellency's response, please accept, Your Excellency, my sincere greetings." for a prompt asking to write a birthday card for a child.
Preparing training data and deciding how to best use it for training I assume would be the largest expense (cost of labour -- mostly expert labour too) and also the greatest opportunity in the future for LLMs to improve. It seems to me somewhat irrelevant if the dashboard indicates a compute expense of $1m or $5m if good training datasets (prepared by experts in their fields) cost $10m/y to maintain. For example, hiring expert software developers to tag 1000's of open source software packages according to their quality, on different metrics, such as human readability, performance optimisation with choice of algorithms, reasonable trade-off between coherence and coupling in the software architecture, currency with state of the art programming trends/preferred dependencies/operating system APIs, etc. And keeping that metadata continually updated rather than a rapidly obsolete once off tagging project completed in 2005.
Because the world is conditioned to distrust chinese models (pick your reason here), I believe this is critical for them in order to kill any arguments outside the actual merits. They probably spent a lot of time making this call and might pay off on the long run.
Whatever well-founded/or not distrust people have in Chinese models, this dashboard proves/shows nothing that can make them trust it more or less. It's like providing the journalctl logs of your HTTP server on your website and claim this proves NSA isn't listening or something.
I wonder if this waters down the “distillation attack” claims by Anthropic. They have their own RL environments! I guess the caveat is that the RL datasets are still opaque, nothing is really proved.
I don't work with large language models, but out of curiosity: mimo's score on DeepSWE keeps going up, so why are they stopping training early? Is it due to budget constraints?
It looks like the curve is flattering, and the current state actually looks slightly cherry picked (it matches a previous spike that looks a bit of an outlier before the result went down). The longer they train, the more they risk getting scooped by another release by someone else. Etc etc it's a judgement call based on all of these factors (and more, including cost/occupying a big cluster as you mention)
+1 at some point, you need to expect to train a much better base model using everything you've learnt. At the least, you probably want to bring on line the next 10 clever RL environments and ideas your team has been cooking up (which will pipeline into v2.7 etc.)
maybe this is why Dario want to slow down AI development and all the big AI labs in the USA is singing the same song.
whey they all singing the same tune. it make me question what is their real motives.
they are afraid of Chinese good enough LLM model killing their margin. we already have story about US companies switch some task to use cheaper Chinese model hosted on Neoclouds.
The reason is money. They want regulation to make it harder for new competitors and competitors from other countries.
They invested billions into training the models but there is no competitive advantage, we see that within a couple of months everyone catches up. There is no way to profitability unless they get some policies to shields them against competitors that can't comply with the regulatory requirements.
That is also why there are things like Claude, Codex and Cursor. They are trying hard to build a customer relationship with a higher switching cost that hopefully sticks.
But the problem is that the AI buildout has become a large percentage of GDP. So obviously the government wants to keep it going because these companies are pumping enormous amounts of money into the economy.
> But the problem is that the AI buildout has become a large percentage of GDP. So obviously the government wants to keep it going because these companies are pumping enormous amounts of money into the economy.
They are pumping enormous amounts of money into each other. Hardly any of that is making its way to people, it's all going to highly automated construction and to energy use.
Seriously, how many jobs did the $1t in venture capital fund?
And, in this case, the dollar-amount increase in GDP serves as a virtual quantitative proxy for the increase in mowed lawns (and the value thereof). In other words, the participants in this economy are collectively ~$200 richer with their mowed lawns than they were without them.
If everyone goes around mowing lawns for each other, the economy is richer in lawn mowing at the expense of all the other things that would have been funded had everyone mowed their own lawns and purchased different services instead.
I am confused with this, if "everyone mowed their own lawns" then the net result will be exactly the same, everyone will be busy the same and not poorer, just without money movement.
This is not the same. If everyone wants mowed lawns, and everyone is busy working on that, there is no opportunity cost, everyone is working on their top priorities. The broken window fallacy is a fallacy because the headline gdp figure doesn't account for the destruction of the window which cancels out the benefit. In the grass mowing analogy nothing has been destroyed, useful and priority work has been done all around.
If the pricing is fair and at arms' length. What's happening in reality is as if they are mowing each others' lawns at wink wink nudge nudge $1000. Not a good proxy for actual value created.
In the real world, you have to pay taxes. So people are incentivized to claim less value for the lawns mowed, or even just do it themselves, instead of benefiting from the division of labour.
Leverage doesn't work that way. (If it were so easy, it would load up my investment portfolio with a lot more leverage than I currently do. And I don't live in the US where regulation T would keep me to a puny 2x leverage.)
But also importantly the government of the residents' country is about 39% ($78) richer, if say the participants are honest in reporting this and the country is the UK and the participants are people like you and me in the tech industry who frequent HN and would think to do something like this.
> Hardly any of that is making its way to people, it's all going to highly automated construction and to energy use.
How do we know that? How automated is the construction really?
In any case, the Fed and other central banks can print as much money as they want in order to hit any aggregate spending or inflation target they have for the economy.
I’m still not at all sure about the “billions” invested claim. How much of that is cloud running the models? How much is pre and post training (which may or may not be part of what we’d want to include in accounting). Etc. Does anyone have links to good reporting about this: not blind recitations of numbers, but analysis and thought mixes with investigation?
Well, if you are right, I just hope their protectionism will only affect the American market, and they leave us unAmericans free to get our models from wherever.
On the other hand (OTOH), China is desperate to keep up and keeps pushing open models (rightfully so), as they understand how far ahead from everyone the US is, and that whoever gets this right first basically is going to become an alien compared to others.
But even with all the open models the US is just insanely ahead in AI buildout and capital allocation (as usual).
Is it a bubble? Is it like the race for the-first-to-the-nuclear bomb? Both?
Unless something has shifted, “everyone catches up” is because these bleeding edge models are distilled. You don’t see this happening with other European and US labs and the problem isn’t something being ignored. I’m not convinced this pattern will continue indefinitely.
I did no such moral claim. I just noted that the foundation labs are working on technical hurdles to thwart distillation efforts and the cost and quality of Chinese models isn’t likely to keep up with the 6 month lag time everyone has assumed.
I don’t follow. Fair use is a copyright defense, and nobody is suggesting distillation attacks are just a copyright violation are they?
Aren’t they alleging these other companies directly entered into a contract and violated the terms, and in cases where question, answer pairs were obtained without such agreement, it was accomplished by outright wire fraud or theft?
I don't understand on what you are basing this reasoning? If a well educated workforce can produce Fable then why couldn't a well educated workforce produce MiMo?
Besides, the latter actually published and open-sourced its RL stack to make it reproducible, which would in fact make it more trustworthy than the models you are speculating were distilled.
No chinese lab has caught up yet. They've tried to fake it by distilling and overfitting on benchmarks to make their models look better than they are, the 'best' models available from chinese labs right now (GLM 5.3 and Kimi K3) fall apart completely when you try to do real work with them. K3 is especially embarrassing because it is larger than Mythos yet performs worse than opus 5 and 5.6 sol in benchmarks they haven't been able to fake yet.
Please explain how putting an upper bound on how good the strongest models can be prevents cheaper less strong models from catching up, rather than enabling it. I do not understand this argument at all.
The general idea is that Anthropic/OpenAI is pushing this narrative as an attempt at "Regulatory Capture"[1] which would allow them to make it prohibitively expensive for anyone but them to enter the market thus stifling competition.
I heard someone analogize token vendors to car manufacturers, where American companies only want to produce expensive options, the people want cheaper/better alternatives, and we ban BYD because those with enough money are more "persuasive"
The analogy is a good one, but your explanation is missing one aspect: the country (USA) does have a reasonable interest in having the capacity to build their own models. The “we need to slow down because it’s getting too dangerous” part is probably more related to “we need to slow our public facing development down so the US government can get the best and the American corporations can trickle out what we decide is safe”
It’s similar with cars. It’s not that American cars are better than Chinese cars on any tangible measurement. But America already shipped most of its manufacturing overseas. Everyone who built those factories is retired. The US should probably hold on to some capacity to make cars, seeing as their entire infrastructure depends on them.
American Ai/Car manufacturers could build cheaper/open models, some do, the big ones do not. It's not an either or, but a spectrum where they have chosen to build only in a subrange
That doesn’t explain the decline of German automotive industry which is now taken over by Chinese cars thanks to massive subsidies by the Chinese government
It’s not a reasonable comparison. In China, every corporation is de facto state run. The Party is in the boardroom and the executives are members of the Party. The CCP will build entire mega cities or pump money into this industry or that according to their plan. China isn't a "state-run" economy but a conditional-autonomy one. Private firms operate freely until they collide with Party priorities. Then the state wins decisively and without due process.
I'm honestly not sure why this is seen so negatively. It seems to be working pretty well for them, perhaps we should do similar instead of whining about others being more effective?
Another point of comparison we might make, how close is Trump's desired increase to the US Defense budget to what China is spending on industrial subsidies? It looks relatively close to numbers in these research papers.
You target the US companies: if they can't use these Chinese models, then they're less of a danger for a now captive audience in the US (and the West generally).
This is already kind of the case: the big enterprises don't really want to touch the latest Chinese models. It's a real pain, personally, I want to use them at work!
Yes, but thats not something a company engaged in regulatory capture for themselves care about: especially if they're worried they'll be outpaced and overtaken by the Chinese labs. Which they will be, IMO.
they care because they know it unlikely open weights will be banned, and thus available to American companies, with regulatory capture (onerous requirements) being a "good enough" "ban" that their big models don't face real competition, regardless of the open weight origin. American companies make open weights too, they are equally threatening to Big Ai financials.
Show them you can burn tokens in seven sessions day and night with comparable results to Opus with less energy and less than 10 dollars a day, per dev.
We have. Unfortunately there are political realities that get in the way, and Bedrock for example doesn't have GLM 5.3 (Flash or otherwise) or anything new/useful
If it's hosted, all they know is "data goes to China".
Until profitable, reputable third parties host open models in the US with ZDR or they become plug-and-play for self-hosting at a modest cost, paying the US models is as much about data protection and liability as performance.
...because everyone saw how well that worked for the Jones act, what with all the naval yards the US has lost over time, and how nearly no US-built ships operate where not legally mandated /s
it wouldn't slow down China as much as make it impossible for American companies to use non-American options, they care about their margins and don't want to be commoditized
> Please explain how putting an upper bound on how good the strongest models can be prevents cheaper less strong models from catching up, rather than enabling it. I do not understand this argument at all.
They are not proposing to regulate only the strongest models. They are proposing to regulate all models. If they are already on top, regulation may stop them from proceeding further, but it also stops the cheaper alternatives from catching up.
If they feel they have reached the asymptote of the curve, then regulation doesn't affect them, it affects those who have yet to reach the asymptote.
Particularly, the route they seem to want to go is "safety".
My guess is that Anthropic and OpenAI will push for "safety" regulations which require byzantine testing that, shocker, Anthropic and OpenAI can pass but the chinese models cannot. The route they'll take is import bans and potentially even general bans on products producing or using "unsafe" models.
They'll further likely try and push AI "safety" treaties from the US to other nations to further lock in their lead.
That's why, IMO, we've been seeing so many "OMG, AI will destroy the world and these AI researchers are so scared" articles.
Cracking down on proliferation of open models which can't be locked down using the kind of guardrails that Anthropic/OpenAI/etc insist are keeping the public safe from all manner of nefarious bioweapons, hacker swarms, propaganda bots, etc. They've discovered they can't meaningfully slow Chinese model progress, so the next best option is to knock them out of competition in the enterprise market for any American company.
Both Anthropic and OpenAI leaders have repeatedly made this exact argument that it's impossible for open models to rigorously enforce the same kind of safety framework as proprietary cloud-served models. It's implicitly part of any regulatory framework they advocate or else it wouldn't be "fair" to American companies since Chinese models would "cheat" (provide weights).
This is true, and it's a good point. I agree that open weight model regulation would either limit the intelligence of open weight models below the frontier or kill them entirely. It would not prevent closed weight Chinese models, but those aren't really tenable in American enterprises unless they strike deals with American cloud providers to deploy them, via products like Bedrock. I unfortunately also have seen no evidence that we can put any sort of guardrails on open weight models whatsoever and so am reluctantly convinced that they should be regulated below the frontier until such a time as someone comes up with a mechanism that is not circumventable.
This is definitely part of it. I think the reports/PR over the past month ended up being a serious unforced error.
Chinese models are increasingly closer to the frontier, while being able to run on much cheaper hardware than what US frontier models run on.
On top of that, both Anthropic and OpenAI showed that they can't really be trusted on data security.
Even if US companies can be forced to not use Chinese models, the rest of the world is going to see the risks and the availability of good enough open weight models for their purposes and be more likely to lean in favor of self-hosted Chinese models or local inference clouds.
> and the availability of good enough open weight models for their purposes
it is childish to believe Chinese are going to give up profits to provide such open weight models forever. the whole idea of being "open" is not compatible with the Chinese culture.
This almost racist read of other cultures has always seemed so bizarre to me. Even moreso when said as an argument on the side of completely closed competitors, some of which are outright seeking to ban open weights.
Chinese companies will continue to provide open weight models as long as it is profitable to do so. Chinese companies are on the more open end in many other industries despite the lack of meaningful foreign competition (for one, 3d printing) so there's plenty of reason to be optimistic as far as I'm concerned.
In a recent Dwarkesh podcast Dylan Patel breaks down how little compute the chinese labs actually have- not even the fact that they don't have access to new Nvidia chips and they're stealing them through shell companies- just that, even if they have cheap electricity, the compute just doesn't compare. Maybe even two orders of magnitude less. They couldn't get it even if they had the money. And if you look at how much more efficient newer chips are, that cuts the effective compute in half again. The conclusion was that they are at least 2-3 years behind.
For frontier labs the current compute seems to be driving model progress (in training) at least to some degree, even without true RSI, and this seems like it'll continue to keep any chinese model from drawing even with the frontier labs, at least for the foreseeable future.
Inevitably the chinese government will drive more funding in chip fab technology and the money will come around to build chinese data centers, but who knows how far off that is. A few different things in the tech tree need to fall into place. It doesn't seem like it'll be next year.
The counterpoint to that, though, is that the Chinese companies have to figure out how to be competitive, regardless of their significant compute deficit. And, as far as I can tell, they're actually doing that. They're trailing the frontiers in model effectiveness, but not by years. It's single digit months.
If there is no upper bound how how these things scale with compute, and if China does really begin to catch up to Nvidia (and they're probably not going to feel encumbered by US patents for domestic AI hardware, given how important AI seems to be to the Chinese government), there will come a day when China leapfrogs the US on AI.
Does anyone know what are the proposed regulations? Controlling software is impossible, so the only option is banning hardware ownership. No more mac studio.
If you pay attention to how these US CEOs talk, it'll be "safety". If I were to guess, they'll try and require a lot of testing, validation, certification before a model is legally allowed to be used in the US or on US products.
It won't be a great moat, they'll probably try and get trade treaties setup to try and expand the moat. But ultimately it won't slow down chinese model development, just limit who can legally use them.
From the frontier labs, the only publicly stated one seemed to be to give them an exception from anti-trust laws to form a cartel and place - incidentally friendly - regulators in charge of monitoring everyone's work.
From politicians like Bernie Sanders, we've had proposals like 20 year imprisonment for anyone researching "ASI".
Dario has always wanted the AI development to slow down and be more careful. Safer AI development was a core reason that Anthropic split off from OpenAI.
What's different today is that now all the big LLM firms want to slow down AI development. When men like Musk and Altman (both known for habitually shooting their mouths off and saying whatever they need to whoever needs to hear it regardless of truth) suddenly agree with Amodei, that's when things start to smell off.
> What's different today is that now all the big LLM firms
not all, just a few American ones (~PayPal Mafia + Google), there are other big American LLM developers (notables include Nvidia, Meta, and Palantir) that do not agree
OAI and Anthropic are forced to release a better model every x months otherwise the Chinese ones will not only be cheaper but also better.
So how could Dario show the investors very nice profit charts representing profit = revenue excluding training costs if it needs to pay a lot of training every x months?
They want to sell the same model for longer(a kind of software subscription where the cost of running /inference is cheap) but the Chinese don’t let them do it. That’s the gist of it. You can see already how they nerf the models just a week or so after release and try all kind of tricks to deliver you shitty performance for the same money. I think it’s part of the same issue of costs and enshitification plan.
In the meantime let’s hope they don’t get to ban the Chinese models(I think they won’t), local AI hardware will get cheaper and the whole AI doom saga will slowly fade to the point that Anthropic becomes a kind of IBM stuff with proprietary data, enterprise certified alignment and enterprise contacts. Think of Accenture junk.
Competition on the provider side—when no single dragon monopolizes the sky—brings fortune for all.
As normal consumers with common sense, we should never naively assume others care for the world out of the goodness of their hearts. Maybe they do, but we should never rely on that.
We can only get good, affordable deals when there is enough competition on the other side.
I mean DSv4.1 Flash and GLM 5.3 kept in check by a supervising frontier like Astra or Fable already in my experience clowns massively on ever using Opus or Sonnet. Opus 5 in particular has been such a stinker that they have to know that they're going to get smoked outside the halo models.
China will most probably win the AI race in the long run because of one major bottleneck the US has - energy. The electric energy and grid buildout in China has been massive since a long time and there is simply no way for the US to quickly catch up.
No matter how much cash you throw you can't just materialize a 100 nuclear reactors to power the data centers.
I was curious how much energy is actually needed to power these datacenters, so I did a little bit of math.
Looking at Nvidia revenues in the past few years, there's maybe $300 billion worth of GPUs currently deployed in the U.S. The B200 costs ~$40k, so we have 7.5 million B200-equivalents, which draw 1000W. Running these at full capacity requires 66 TWh a year, or ~1.5% of total current U.S. electricity consumption. Maybe a bit more to account for inefficiencies, cooling, and other components, but not more than ~2.5% total I would guess.
So it's not that much in reality, but will definitely grow fast.
> Running these at full capacity requires 66 TWh a year,
I think your numbers are off.
For a start you are effectively calculating a GPU only number.
I think 100Twh would be the minimum level to think about "all-in". And even that is probably being generous.
Remember, afterall that Google have just bought half the capacity (4.1Twh) of a nuclear power plant in Finland, on top of 630 MW of wind and 94MW of battery.
This is to cater for three new sites at Kajaani, Muhos, and Vaala and expansion at Hamina. So basically 3.5 datacentres.
But Finland is quite a small place. The US has more sites and bigger sites, so the numbers probably grow exponentially very quickly.
It's even less if adjusted for non 100% (more like 0.35% of total). Yet factual impact on the grid and other industries will not be as small as numbers appear. For example most probably there will appear transformer (electric) and witchgear deficites. And this is not the only supply chain bottleneck in this subject
It's worth looking at similar industries with enormous electricity requirements such as aluminium smelting, where the plant can be located in a friendly country but the product is owned and controlled back in the US.
Aluminium is often described as "congealed electricity". Ship bauxite to wherever power is cheap and stranded, turn it into metal, and ship the metal out. Here in NZ, Tiwai Point is the textbook case, with London-based Rio Tinto running a smelter on the other side of the world that exists mainly because Manapōuri hydro had nowhere else to go.
AI data centres can be just the same - even more so, since the plant's assets (its chips) are virtually perishables, so there is less concern about assets becoming stranded if the host goes rogue. All the US needs is friendly and stable allied countries with cheap power.
How long will it take to convince them that it is worth it to get into long-running relationships with the US. How long before another populist is elected who will rip up agreements for weird reasons and you have to renegotiate them?
the bottleneck right now is compute, not energy, and it's not even close. That is why RAM, SSD, CPU, and GPU prices are increasing exponentially, while solar panels are dropping.
Also, unlike China, US companies are building data centers all over the world, which gives them higher distribution and ability to colocate with the energy production sources.
Lastly, energy production costs have been decreasing over the last couple of decades. If they will increase, the market will react, as it always does. Looking backwards does not predict the future in this case.
I think we are talking about the medium term for China to pull ahead. They just released a new domestic AI chip which seems in important ways to have caught up to Nvidias previous generation. They have a LOT more manufacturing capacity. They are collaborating via open source by default. They have better materials access and much more energy capacity. They have many more people overall and more researchers.
There is a strong chance most of the researchers are pulled out of the US and Europe if WWIII really kicks off or even if there is just more global crisis and concern.
One other thing about the power needs. Within a few years, the power efficiency of AI chips is likely to improve by a factor of 20, 50 or more times by switching to true compute-in-memory architecture with new materials that have made rapid progress lately.
Its also pretty telling that the release of the new Chinese chip wasn't met with an intense discussion here on HN. And it's arguably way, way more important than a version bump on some benchmaxxed model or two. The advances in Chinese chipmaking are super exciting.
I think you underestimate what a massive country China is and the scale of their military. Russia is already quite an adversary, but China is at an even higher level. By some metrics, they have already surpassed the US military.
So it is very unlikely that a neighbor will have the audacity to attack them, and even if this happens, i'd expect the war to be over rapidly. Just like if Canada or Mexico tried to attack the US.
I think the consensus is that Russia proved itself barely capable against just Ukraine. Also I didn't say someone has to attack prc, it's more likely to go the other way round and as many historic examples showed no matter how big you are people defending their home land will not make it easy.
The scenario we are all dancing around is if there is a war over reunification. That would be between the worlds two superpowers. No one would walk into a nuclear Holocaust on purpose I bthink, but miscommunication happens. Even if no other nations intervened, the reunification would definitely destroy all the worlds leading fabs, and even if it didn't, I read ASML can and would shut their machines down remotely. So those researchers would shift to figuring out what they can do at the 28nm node... Meanwhile the rest of us ponder why old powerful men can't let things be.
I'm curious about that, do you have relevasnt metrics? I imagine stats out of universities, engineer schools, etc and maybe number of patents could be used but fearing those could be gamed.
> China has recently awarded 1.3 to 1.4 million engineering bachelor's degrees a year, and about 292,000 postgraduate engineering degrees in 2022. [0]
> For the US, ASEE and NCES put engineering bachelor's degrees at a stable 130,000–145,000 a year, rising above 200,000 if computer science is included. Because China counts CS as engineering, the fair comparison is roughly 1.35M against 230–250k US engineering plus CS graduates. That is about 5–6x in absolute terms. Adjusted for population (1.41B vs 335M), China produces about 950 per million people and the US about 700 (1.3–1.4x more).
Even Claude highlights that the numbers aren't apples to apples and in China the definition of "engineer" is more loose. I do believe that in relative numbers, more engineers graduate in China
On lawyers:
> In the US there were 1,322,649 active lawyers as of January 2024. For China, there were around 650,000 lawyers in 2022. Which in relative terms is 8x in the US than China.
You can enter law school with an undergraduate degree in English. It's not clear that most people on that career path would have become engineers in any culture.
Well, US did not really have that to start with (Kaiser &co learned fast). So it’s not about having something and losing it as much as bootstrapping and discarding.
For processes that are scalable, well known and industrial, profit motive actually does provide results.
The question is more of are there bottlenecks of skill and maybe secret sauce (eg something like ASML).
The US had the raw materials available in country to support that sort of build out and I don't think that's the case today. It would take kickstarting a lot of prerequisite industries to get back to the point where we could even start to learn how to build ships enmass again.
All they have to do is keep being close behind OpenAI and Anthropic for half the price and the US economy will plummet into a deep recession when it becomes clear that there will never be a payoff for the enormous investments of the past years. If they can also achieve a breakthrough in robotics, then they are on track to become the most powerful country for the next 50 years.
> "Quite frankly, the biggest issue we are now having is not a compute glut, but it's power and it's sort of the ability to get the builds done fast enough close to power," he told the show's hosts. "So if you can't do that, you may actually have a bunch of chips sitting in inventory that I can't plug in. In fact, that is my problem today. It's not a supply issue of chips. It's actually the fact that I don't have warm shells to plug into."
don't worry, we're prolonging retiring coal plants and building a lot more gas turbines to meet (and really exceed) capacity needs on a state-by-state basis and just completely abandoning the Paris Agreement and any clean energy goals
The sole reason the Chinese cannot “win” is because ceding more power to agentic AI will eventually reduce the primacy of the CCP’s ideological control.
As these models get smarter they will no longer distribute it openly. Patel reporting this too.
There are real headwinds that I don’t think people have thought through.
From a pure training perspective, yeah they could try to align it with CCP values. But the promise of AI is that it'll unlock an explosion of growth and prosperity. What happens when the CCP no longer becomes seen as the the primary enabler of growth? What happens when people are exposed to greater levels of agency? CCP played with fire in the COVID lockdowns and almost got burnt.
Like Terry Tao recently said, there are nonlinear effects at play. Things are going to get chaotic and I do not have confidence (like the parent comment) of anyone "winning".
Been playing around with both of these since last night. So far, when enabling reasoning (which is binary on/off), it seems to me like Flash either is less "token efficient" or just likes to think more, or I'm doing something else wrong, because most prompts I send to both, Flash reasons more and for longer than Pro, which is the opposite of my expectations.
Any lower (or broken) quantization could do that, not what I'm talking about though. They work fine for their size, as far as I can tell. Just surprised the Flash would reason for longer than the Pro.
Yeah, I agree it's probably fixable, but I think a naive interpretation of the metadata is probably the source of this bug (which HF has had for as long as I can remember)
Yeah, for better or worse, writing style is practically uncorrelated with agentic performance, which is all the rage right now and the thing that most popular benchmarks currently prioritize.
I think we are still far from nailing down good LLM benchmarks, because the more general-purpose your software the harder the question of what makes it good becomes. Is Python a good programming language? Is Java? Is C? I think it's a similar class of problem. You can benchmark rudimentary things like execution speed similar to how you can benchmark tokens/second, but these metrics don't tell the whole story.
We're past the one model fits them all kind of LLM. Most of the recent release actually regress on world knowledge for example, but optimize for something different: tool usage, thinking process, and agentic approach. And yes, in my own usage, some usecases Opus beats Fable.
Yeah it’s a shame a lot of these benchmarks are behind. My favourite was ‘SlopCodeBench’ [1] as I’m most interested in ai reinforcing its own bad decisions, but it’s not even up to current gen oai
They match my experience. Astra and Fable I rate below Sonnet. They are incredibly poor. They were excellent for a couple of days after release and then plummeted.
Maybe I am being routed to more quantised versions or less capable models with system prompt to fake Astra or Fable.
A sentence that I would not have expected to read on HN as recently as last year, but that makes perfect sense today.
Jokes asides, @simonw any plan to include a 3D model version (make a 3D model of a Pelican riding a bicycle in Blender)?
Given that Astra seems to have improved a lot in 3D modeling capabilities (and that matches my experience) I'm actually quite interested to see if/when other models catch up and how they stack against it.
Or if anyone knows what would be a useful existing benchmark for that skill.
I've not committed to that as a benchmark yet because you really need a full coding agent configured to run Blender, and I like benchmarks I can run as a single prompt/response through the appropriate API.
I have been intending to get more of a coding agent benchmark going though, so maybe this should be part of it.
A different pelican on a different bicycle direction(R2L) finally. Wondering why MiMo V2.6 Pro choose R2L and Flash choose L2R for the bicycle direction.
can you update this website, I just wish the entire layout wouldn't shift when the page gets loaded and the timestamps in the title look very ugly and take up a lot of space.
Not sure if it is the same thing they're referring, but the transition from loading the prompt from the gist to the result is indeed jarring: https://files.catbox.moe/c56eyl.mp4
I don't believe it is that important, but I guess you might've just hit some people's pet peeve with that. If anything, it goes to show as to why skeletons are popular on modern sites
Looking terrible isn't nessesarily a bad thing. The pelican is heavily pre trained now. Having a crappy pelican means you didn't try to juke the stats.
Apologies for not taking the time to find it, but there was a post that tried to determine if the pelican was benchmaxxed across a bunch of models by comparing it to other SVGs, and found that it wasn't at all.
pro model was having fun on minimal effort with that fish lmao, hilarious.
out of curiosity, is it the first time we're seeing a fish there? I don't remember seeing that detail before.
Making SVGs is coding (of a very specific markup language). So it's not actually absurd. We've also seen that more capable models are generally better at everything minus benchmark contamination (benchmaxxing) anomalities.
Looking at the frontend design examples; why do these models seem to love the "01 - UPPERCASE TEXT" motif. It's everywhere now (see https://try.cloudflare.com/, which has '01 · QUICK TUNNELS', but no "02" anywhere).
The extraneous small-caps labels are one of the main idiosyncrasies of AI generated markup. I wonder how much of this is a "scaffolding" technique to help the model build stable designs. But was it reinforced in RLHF or an emergent behavior of the models?
My guess is that by function they break down frontend sections or components into pieces and I believe document things for themselves on some level, or purposely are verbose in this way. It is probably also shaped by users and existing web patterns. They probably get reinforced by models the more common they become.
I said this years ago, LLMs are a commodity (or were becoming one at the time). They are dime a dozen. Even the frontier ones. OpenAI and Anthropic have no moat.
No moat and competition is good for consumers though.
Which is why while HN is sleeping and thinking Google and MS are done in the space, Copilot and Gemini are winning most of the non-tech: it's just an extension of their current teams/cloud offerings that companies were already using.
I have asked most of my non-tech friends: chemicals, banking, insurance, consultancy. They are all on Gemini or Copilot.
This doesn't really match my experience, but all I have to counter your anecdotal evidence is my own anecdotal evidence.
In my opinion, concluding that Copilot is "winning" right now is like concluding that Internet Explorer was winning while Chrome was still young. It may be technically true, but it's easy to see the writing on the wall -- that it's only winning because it is the default option available at most companies.
I have never met a satisfied Copilot user, and as a consultant I met a lot of Copilot users. It's hard to believe how totally and completely Microsoft is shitting the bed. As you point out, they have the enterprise integrations. Why can't the build a good product with them? Generational blunder, in my opinion.
yeah nobody is happy using windows either, but still they all do anyways.
we would be better off using linux all together, but here we are.
Lets see maybe every company will have their own little instance of a local LLM in their basement in the future, when the AI-bubble pops and server-racks become cheap
If we mean Gemini the gemini.google.com, or gemini built into sheets or other Google surfaces, then people in these orgs might be getting a very different experience of what LLMs are and what they're capable of.
Gemini the web/app is basically where ChatGPT was on release day, maybe even worse than that. It's laughably bad. It forgets the context between turns, it has dynamic tool injection which doesn't get injected half of the time, which causes model to hallucinate tool calls.
The new Spark is much better, but I don't think most people are using that.
copilot is doing the same.
i just joined a corporate and they only allow copilot. i am getting brain cancer using it, especially being used to claude max from before.
but cost is a thing, if i use the LLM gateway for a chatGPT-5.4 i am quickly (3 days of very light use) on 50$ in token. and everyone and their intern are using copilot. As azure has the guarantee that company data will not leak they are taking the bait.
The fear of secrets landing in the training data is just too big, and they are all microsoft 365 customers already. And lastly Management seeing that copilot can autonomously edit an excel file for them already counts as mindblowing... 0_0
I've noticed the context loss in subsequent turns, it's so strange. For me it happens whenever I stop gemini mid prompt. Sometimes its response annoys me, so I'll hit stop frequently. And then subsequent msgs seem to go through like a new chat where it forgets what we were talking about.
My 0.02 on this is that for a given level of capability, yes, there is no moat. What’s frontier one day quickly becomes a commodity almost too cheap to meter within a year or two.
But there is always a use case for frontier models, even if they’re quite a bit more expensive. The set of things you can profitably do with better intelligence than everyone else is unbounded.
So yes, the number of tokens that get pushed through commodity models for very cheap will continue to grow, but so will the tokens for expensive frontier models. We’ll never run out of things to do with the latest geniuses who are twice as smart as last year’s geniuses.
The moat is the engine that builds and sells the latest geniuses, especially the compute. And no one will have more compute than American labs for the next 5-10 years.
They’ve been saying that for the last 18 months at least. They’re always just a few months behind…
I picked double out of a hat, obviously there’s some intelligence gap that’s too small for a price premium beyond a certain point, but it seems like OpenAI and Anthropic’s revenue keeps climbing, so I don’t think we’ve hit it.
The gap really is closing, though. I don't remember the source, but there was a publication recently stating that it's 4 months at this point. So, 4 months where you can charge premium prices for the small subset of tasks where only a frontier model will do. I have to think the big AI duopoly is at an inflection point where most of their customers haven't just yet realized that they're being fleeced.
well, it is true. there is a price ceiling for intelligence, there were news about how adoption of fable was lower than expected, only hitting 11% of enterprise spending for customers with anthropic, iirc 26% for Sol with openai customers.
in any case, I've been using open and closed models since sonnet 4, i remember when the best I could get was qwen 3 480b coder, you can definitely feel the gap closing going from that and GLM 4.5, to GLM 5.3, DeepSeek Flash V4.1, Kimi K3 etc, it's reached the point where i wish I had V4.1 at work, it's faster and bullshits me less when I use it in my personal projects. And I have unlimited access to fable 5.1
> But there is always a use case for frontier models, even if they’re quite a bit more expensive. The set of things you can profitably do with better intelligence than everyone else is unbounded.
While this is trivially true, the question is if these remaining use cases that separate frontier models from non-frontier models bring in enough revenue to offset the massive spending of the frontier labs.
No moat and competition isn't always preferable over a competitive oligopoly (with some differentiation). The former ends up with politicians intervening way more like with solar, steel, agriculture ....
Mimo has been one of those models that I have been rooting for since the first I used it, the 2.5 pro which I have used quite a bit, was very concise, very aware of how much context needs to be read for which tasks and would always keep the context tight. Also surprisingly good at strategic thinking. I had published a comparison between it and Terra where Terra was found to be using much more avg context for similar tasks https://dirac.run/posts/gpt-5-6-vs-mimo-2-5-pro-context-bloa...
Interesting but not surprising trend across the board seems to be, the flash models seems to have caught up with the pro-sized models of H1'26. No surprise all labs are rushing to bigger models.
EDIT: Wow, took a detailed look at the benchmarks. Mimo 2.6 pro, the 1T model leads Kimi K3, a 2.8T param model in 14 out of 15 benchmarks (and the last one is near tie)!! Good to see they also kept the price the same, and landed in the greenest quardrant of the intelligence vs speed of AA.
Mimo scores higher, even though it got more questions wrong, because the coding tests are scored higher than other tests, and Grok failed some coding tests, which is surprising tbh, as they aren't that hard.
Mimo has long been one of my preferred models. It works well at many code tasks, generally has a pleasant voice, and isn't prone to over-analzing and researching
Also when I was using it, I managed to do quite a bit on a few bucks worth of OpenRouter credits. Not sure how well it keeps up in the modern world against things like Luna, but I hope it remains competitive
All these new models are such tease for us folks with 128GB of shared memory. Buying another unit now to expand to 256GB is a mortgage payment but it’s getting tempting…
I can imagine someone building a device that uses NAND flash or similar tech but with a very different controller that is optimized for streaming the data out with a predetermined access pattern at very high speed.
Flash, like pretty much every solid state storage technology, can scale its output bandwidth to ridiculous levels limited pretty much only by the readout circuitry. There may be a price to pay in power consumption, though.
You could always stream from SSD storage. Especially effective if you get a cheap old-gen HEDT with lots of PCIe slots to add NVMe storage to and reasonable overall PCIe bandwidth.
That nearly certainly boots you to secs-per-tok land (as opposed to tok/s). Plausible if you are willing to wait hours to days for responses for simple testing, but not (debatably) "usable".
Demonstrated inference speed for the biggest open-weight models is indeed around 1 token per second, with weights read from SSDs.
However, this is for relatively low-end systems, with a couple of fast SSDs providing around 20 GB/s throughput (or with a few more, but connected through relatively slow Thunderbolt, for a similar total throughput).
If you use 16-lane PCIe add-on cards with 4 M.2 slots for SSDs and a total throughput of 50 to 60 GB/s, you can quadruple the previous speed in a desktop PC where you use the GPU PCIe slot for SSDs (a fast CPU, e.g. an AMD 9950X, would be alone fast enough for inference limited by SSD throughput, so a discrete GPU would not be required).
If you have a server/workstation motherboard, e.g. with 6 16-lane PCIe slots, you might gain another factor of 4 in the inference speed, so one might reach around 15 tokens per second for a very big model, but the cost would also be high, with so many SSDs, even if at that number of SSDs each SSD could be the smallest that can be found with a PCIe 5.0 interface.
You can definitely offload n-gram embeddings to storage; they're very sparsely used (only a few KB fetched per token) so this is quite effective. Loading to DRAM only becomes necessary if they are a bottleneck to overall performance (which might happen if you're doing very wide batches and everything else uses super fast VRAM/HBM).
I was looking at the qwen-next-flash, and the weights would fill my OEM Spark on their own, before the n-gram. I'm unclear if offloading to disk can work here, is that what you are implying is possible?!
I'd recommend pointing your agent at it (after installing sparkrun), and asking it to research the absolute latest in TP=1 Flash-Next - mine grabbed particular vLLM nightlies and mods to improve performance, and it was well worth it.
I have a quirky vLLM on k8s on 2x OEM sparks setup with 9 models available to me. I'm not keen to run nightly vLLM, too many issues with it in the past. Going the qwen-next path means displacing things I use daily :/
I have a watchful eye on the diffusion ~ Jev/Kev PR
For what it's worth, Flash Next outperforms every other model that is available to us on the GB10 in all of my testing; though if you have two sparks then the TP=2 version is even better and easier (I don't think you'll need the nightly for that at all, just use the recipe)
Nah, I’m streaming ngrams off NVMe on my Spark-alike right now. Works surprisingly well (except for when I accidentally bottlenecked it through my NAS)
GB10 boxes have way more compute than they have memory bandwidth, which nicely fits medium sized MoE models with speculative execution (MTP, DSpark/DFlash, etc)
Qwen 3.8 Flash Next (what I'm running basically entirely now) sees 30 / 35.0 / 45 tk/s for prose, analysis and code respectively for actual use (not short context benchmarking) with Pi. Thinking blocks are ~35tk/s or so.
The GB10 having so much compute is great for prefill too, 2000-3000/s for 14k to 64k token prompts (cold cache too) in the quick benchmark I did. 3500tk/s for warm cache which is nice :)
When I accidentally streamed my ngrams over the 2.5Gb/s network, it cut all the throughput down in half basically. Especially notable for the time-to-first-token, which is what clued me in that I'd messed up somehow!
For Qwen 3.8 27B, I got it up to a consistent 20tk-25tk/s but 27B thinks so much that it was honestly too painful: Flash Next is as smart, as useful, but much faster for real agentic dev usage IMO
Laguna S 2.1 saw similar numbers to Flash Next if I remember right, but their latest updates means it doesn't quite fit a GB10 128GB anymore at full context which is a shame.
Note: these are all NVFP4 quants (usually a dynamic one where some tensor layers are left at full precision though)
Are folks working in biology / cybersecurity seeing more limits in what Opus (not Fable) is allowing? This has happened quite suddenly for me and I’m stuck in the middle of a project that would have otherwise called for use of Claude.
I’ll be trying these models out and may end up switching my subscriptions if this craziness continues
glm 5.3 and deepseek flash v4.1 are quite capable in cyber. But they're still slightly behind Sol/Astra/Fable.
What works for me when I need cyber from Fable/GPT is switching the model after cyber refusal to DS flash for one turn, then switch it back to Fable/GPT. This works most of the time.
The moat for OAI and anthropic seems to be very quickly shrinking. Chinese labs are now using RSI-like approaches and even without resorting to heavy distillation they're catching up in a couple of months vs. what would have been 6-12 months a year prior.
And as these models get better the pace of training is quickly speeding up too.
This doesn't bode particularly well for anthropic/OAI after they go public.
I don't know if we will look at OpenAI and Anthropic as moating on frontier models & selling tokens.
They are banking on the application layer and accumulated business and end user context. They have to quickly make that systems integrated value out weigh the model choice price value in the broader market
token vendors are headed to the same place mobile data vendors went, this is good for everyone but those who thought they could maintain exorbitant prices
when mobile data first arrived, it was expensive and people bragged about their bills (token counts today)
with time, it became commoditized, people now have unlimited plans, and the money is made by the applications that sit on top (token generation is increasingly undifferentiated low-level infra)
This is not to say there has not been significant innovation in the time since, but it's a low margin business (tokens look to be headed this way)
I tried them, but could only test Pro none and Flash none and low, the other ones (medium/high) used way too many tokens and all requests timed out. Not sure if they have a problem with their API, or this model is really token inefficient/basically unusable.
The full-response APIs of many providers have been inadequate for a while now because their timeout intervals do not account for lots of thinking. You can use the streaming API to avoid timeouts.
Can someone recommend me a local harness to explore the multimodality of this kind of models? Text in text out is mostly generically solved, but that's about what I know
Are people using MiMo models as their daily drivers in a company setting? If so, how? I know they're available over OpenCode and directly from Xiaomi, but those are not great options. OpenCode Go straight up doesn't give any guarantees about training on your data, and Xiaomi says that they won't but it's unclear.
With some models you can find hosting companies based in the EU or US, but then you don't know how they're quantizing the models, so you're not sure about the actual output quality.
How are people actually using this? Or are people just experimenting with side projects?
I only use it for personal projects, but their european Token Plan [1] has ZDR and is hosted in Amsterdam. The DC also likely runs on 100% solar wind/power, with heat recovery systems used to heat nearby university buildings.
Hmm I may be mistaken on that now, I remember it being in the user agreement. There are third party routers that explicitly offer ZDR, though API costs will be slightly more expensive.
Yep am waiting for some of my whitelisted openrouter providers to pick it up (so far none offer it but hopefully a few will in the next couple of weeks)
Please, think about the american companies!
If we don't protect them, when they go broke (because they will, it will be the perfect triple dip) and we bail them with american taxes we will have paid twice for all the work they had already stealed (not my words, this is a microsoft anti-ai exec...)
Twice, trice or quadrix(tm) are just approximations, we have already paid with:
- more expensive electronics
- less work
- all the retirement money put into gpus
- all the "fair use" of all the books, all the images,
- and then taxes to bail them?
Anyone knows what they used to create the videos ? Is the model driving a program like Davinci Resolve / After Effects ? or is the model writing code to then generate these videos via some library.
Overall this felt like the weakest of the three. Ultraspeed was fast as far as tokens per second goes, but it overthought quite a lot of things resulting it in having one of the longest build times. That overthinking didn't lead to better results either - note the statue with the cropped off arm. It's also the worst implementation of the dynamic lighting effect / displacement effect - the background especially has some significant distortion. Astra was the only one that seemed to understand that displacement should happen less the further something is in the distance.
For me it seems like this model’s overthinking should be tamed.
Perhaps instead of giving it a one shot task with vague prompt. I wonder how it can perform with much more detailed and constrained prompt (can you please elaborate more on the level of detailness and ambiguity that the prompt is and where does this model seem to overthink the most?)
Also are there any ways to tame such overthinking of models in general?
I hope that once models start becoming smart enough (I think for me it’s already there) or becoming genuinely the Sota. They then start focusing a lot more on optimizing token usage
How do you think other benchmarks work? Every single one is going to have a budget and time limit. There has to be a cap on those, you can't just let it spin forever and use unlimited funds.
I would argue that this benchmark is uniquely suited to how most people use LLMs because it actually tests common harnesses and it is a true coding benchmark for a language that is unseen, thus testing the LLMs actual ability to understand nuance and learn.
Internet access is restricted to ensure that over time models cannot just look up the source code of KillSwitch, which would allow them to cheat.
I didn't expect there to be a time limit and I do know benchmarks that don't have a time limit. They just penalize slower models which I think is fair.
In my experience models just don't take forever to mark tasks as done.
For my coding usage I don't set time limit so benchmarks that do so are providing me less interesting use cases.
With that said, I can understand having budget limits for expensive LLMs for those of us that don't have infinite VC money.
As for no internet access, the issue is that for most problems we do want Llms to be able to search docs, API specs, GitHub issues, etc. So not allowing that usually just favours larger LLMs that were able to memorize more data, not necessarily smarter ones when both have internet access.
Absolutely! Chinese models are both cheaper and more capable in many cases, compared to the American models and their makers continuously fumbling or reducing model capability with each update. Deepseek decreased costs when they released Flash 4.1 you would not see any American company do this, in reverse they would try charge you more.
OpenAI decreased prices with the 5.6 model family.
And later they further cut Sol and Terra pricing by 20% (maybe only in the API) and Luna by 80%.
In fact Luna still outperformed DeepSeek Flash 4.1 in cost per task on Artificial Analysis when I last checked.
However, Luna is slightly less intelligent. I have a feeling that it's pretty dumb and prone to hallucination unless running at xhigh or max effort, where it somehow manages to work quite well.
I did not personally test the open weight models beyond the old Qwen 3.6 27B, which produced unusably bad results for me.
The competition is great, and I hope Chinese models will continue to force leading US labs to offer models at a low price point.
That said, I don't think the Chinese labs have anything over OpenAI and Anthropic when it comes to capability or efficiency - I have no reason not to believe the US labs have even lower cost to serve the models.
OpenAI had to cut costs because of Anthropic. I also do not trust the benchmarks when it comes to models anymore. I have tried both Claude and OpenAI models and while it is true that the 5.6 series is smarter than Deepseek (at the time i tested it against 4.0) at that price it is still not worth it and sometimes randomly refuses to do tasks or stops midway etc.
Do also remember China is this far in the AI race despite all chip restrictions from America. If they were in equal standards I truly think Chinese models would have long surpassed American ones. Also would like to remind how Anthropic CEO is being hostile and blaming Chinese models with distilling meanwhile their own models claimed to be Qwen¹ and their stance against open models is negative² and they still keep blaming China for it.
> Also would like to remind how Anthropic CEO is being hostile and blaming Chinese models with distilling
Why wouldn't he? If there really was 25,000 accounts breaking ToS any CEO would at minimum be upset. Evidence of Claude distilling qwen would be damning but that a) makes no sense b) doesn't exist afaik.
They have the logs, created a report and sent a letter to Congress. Whether you believe it is entirely up to you. Given Chinese firms record on IP theft, it's entirely believable. I don't have any doubts, but I might question how they attribute it to a specific firm.
And they took multiple measures presumably to stop "distillation", such as hiding reasoning steps.
Chinese models kept improving in capability regardless, and are in some ways more impressive than Claude/ChatGPT.
So yeah, I think they are bulshitters. The can create reports and send letter to congress simply because they know if allowed to compete freely the Chinese models will eventually prevail.
Also, very rich of you to mention Chinese firms record on IP theft when Anthropic and OpenAI are companies entirely built on large scale IP theft.
So first it’s “Chinese companies cut costs, and you’d never see American companies do that”, and then when it’s pointed out that one of the leading American labs literally just did that, it’s “yeah, but they had to because of competition”.
What do you think is motivating the Chinese labs, benevolence?
> If they were in equal standards I truly think Chinese models would have long surpassed American ones.
Limitations often lead to creativity to overcome them. The Chinese AI labs have had to focus much more on efficiency so they got good at it. Meanwhile breaking new ground is often harder than replicating it. So even if they had matching compute it's not a given they'd be better.
> I did not personally test the open weight models beyond the old Qwen 3.6 27B, which produced unusably bad results for me.
So you don't have much perspective on things, it seems. Let me introduce you to the GLM 5.2 and then 5.3/5.3 flash series of... "oh, wow, I should have bought some RTX PRO 6000's while they were 'cheap'" stage of progression.
As someone carrying multiple max subscriptions to both claude and codex - primary workhorse is glm 5.3 flash running on rented GPUs for less than a latte/hr.
I also found qwen 3.6 27B nearly useless for my own needs. DS4 flash 0731 and then 4.1 have been nearly as eye opening as glm 5.3 flash, but have their own warts.
Have you tried Qwen 3.8 Flash Next? You can run it on one spark with reasonable context sizes at about 30 tps, and it's as good as DS Flash 0731. Maybe even a tie with GLM 5.3, though like everything it depends on the use case.
I’m having the same issue. Hold max subscriptions on both frontier labs but I’ve been forced to use open source models because token limits are not what they used to be. So I end up using Astra and Fable for reviewing, and open source models for implementing.
> OpenAI reduced prices and Anthropic increased weekly usage limits.
As a Max x20 and Pro x20 subscriber, can tell you that it doesn't matter since they continually move the baseline of token use So in practice you feel that you're continually getting less from your subscription.
While it never happened to me in the past, i reached my weekly limit within 3 days using Opus 5. And the Open AI weekly limit essentially is a Claude Max x20 5-hour limit. Not even talking about the baseline in intelligence : on release day Astra was so good that it lead me to move to Pro x20. Now it's dumb af and token use is insane.
Deepseek 4.1 Flash has been a lifeboat for me, finally able to work without being constrained/distracted by limits and with what is in my view even better intelligence than Opus 5 for a fraction of the costs. DS is not messing up my brain with load-bearing pseudo jargon in every sentence. It respects coding guidelines, and completes even the most complex tasks most of the time in one shot.
DS 4.1 had been able to add complex features to my repo without breaking a sweat (330k lines of F# + 4M circa lines of an Angular frontend). Writes very idiomatic F# and respects our guidelines and style perfectly. Just completed an extensive UI/UX research and implementation work.
I am ditching both x20 subs and will only keep a Pro x5 because wife does a lot of design work and needs solid image generation capabilities.
Absolutely! DeepSeek-V4-Flash-0731 has become my daily driver. It's pretty amazing what it can do for what it costs at deepinfra.com (I don't use deepseek as a provider since they train on your data [at least their honest about it]). GLM-5.1 was my daily driver before that and Kimi K2.5 before that.
My primary use is AI coding agent. Its vastly cheaper than Kimi K3 and I haven't found a scenario where I really need Kimi K3 versus smaller models. GLM-5.3 Flash is good but there is series of bugs in the vllm middleware that prevent GLM models from getting all of their reasoning content returned to them that impairs inference quality. A lot of inference providers use vllm which makes it hard to find a good provider for GLM. I've been using friendli.ai but using GLM-5.3 Flash from them is more expensive then using DS V4 Flash from deepinfra.com simply because deepinfra.com is so cheap. The DS V4 Flash cost at together.ai is similar to the GLM-5.3 Flash from friendli.ai or at least that's what I found in my benchmarks a week ago: https://www.linkedin.com/posts/joshheitzman_i-ran-a-fuller-r...
4.1 consistently surprises me in capability for the price. And I don't think I'm the only one. It's been dominating the leaderboard at OpenRouter, and I just got an email today from Fireworks saying they were _raising_ the price by about 30%. I'll probably switch, because their infra doesn't support being the highest-cost, but it's still telling.
I tried it a few times and liked the speed, but often found it ended up looping, i.e. repeating the same token sequence (e.g. the same sequence of 5 paragraphs) over and over again until it hit the max output limit. This doesn't end up happening every session, but does every now and then.
My impression of DSv4.1-flash was very positive aside from this. But that was enough for me to stick with GLM-5.3(-flash), which both gave me consistently great results
I was using a vibe coded bare bones harness.
I was wondering if this was normal from DSv4.1-flash, or if its my harnesses fault.
I've had that looping issue with open models too. But never 4.1. I wonder if it's a model + harness combo? But yeah, one loop issue and I'm done with a model forever.
Yeah, that's what I'd been leaning towards.
No mcp, but I'll see if I can reproduce and debug it, since other people don't seem to have that problem as badly as I've experienced it (and the idea of having a nasty bug like that bothers me).
No mcp support.
I'll try copying deepseek harness's basic tool call formats as a starting point.
I haven't tried 4.1 flash as I'm assuming its a preview. I did not get good results from the preview version of 4.0 flash (i.e. the one that did not include the month and date of release in its name).
Affordability is derivative of control which is really what I care about.
I'm just not going to build long term infra that depends on something that another person can and will - objectively based on experience - take away from me at some unknown point in the future.
The biggest benefit of open models is they keep all the other players honest. The extent to which they feel they can dictate terms is directly set by the threshold where they feel people will take the trade to run open models instead.
American models are on the frontier of capability. Chinese models are on the frontier of efficiency. The problem for American labs is that Chinese models are more than capable enough for the vast majority of applications that people care about at this point, so efficiency is more interesting for people.
it's really weird to me at the moment because both OpenAI and Anthropic seem to be competing in an extreme benchmaxxing contest on super intelligence that actually nobody cares about. I haven't really cared about model intelligence since about Opus 4.8. It is by far not my biggest problem. I don't need to replace or support Einstein in my production workflow. I just need basic intelligence that can equal a routine office worker - safely and reliably. What they doing - chasing super-intelligence but dramatically escalating risk - is actively what I don't need.
I really think they have drunk too much of their own kool aid and become completely detached from what the market wants.
The cost issue is obviously of prime importance, but I'd also argue that the transparency of the innovations creates a tremendous cross-pollination, and not only within the Chinese communities but in the US/Europe as well. How many of us are learning the practical aspects of actually running and building AI based primarily on open models? As an example - how far would the work of vLLM or SGLang or even NVIDIA itself (all random examples) be without these models and the challenges they pose?
The analogy makes little sense. The USA was not in front of the USSR and Sputnik merely showed that. It is at this point that the Americans woke up, put a lot of effort and finally were able to surpass the Soviets during the Apollo missions.
China was never ahead of the USA in AI. So perhaps a more proper analogy is the Moon landing. In real history the side that lost the race never got its mojo back...
I was simply saying that when (not if) Chinese AI models will pass Americans, it will probably be game over and Americans will never catch up, let alone become leaders again.
Check the names of the researchers in the DeepSeek's latest paper. Full of Chinese names. Check the list of names in Google's paper. A very similar view. Anecdotal, but quite thought-provoking...
I did try to use Chinese open models, but for my production work they simply couldn't cope at all; both GLM 5.3 and Deepseek v4 went into infinite loop and wasted my tokens until my OpenRouter wallet reached 0; good thing I didn't enable the auto topup. US models, by contrast, breezed past them.
Even for simpler tasks, Chinese models took long time to complete, and I needed to supervise closely. The price , in the end, didn't come cheap, mainly because too much time wasted on thinking.
So maybe one day Chinese models will squeeze out the American ones, but today is not that day.
So no, I am not excited about Chinese models ( just because its open weight and not American).
Not too be "that guy" (e.g. "you're using it wrong"), I just want to humbly ask — have you tried blacklisting "underperformers" in OpenRouter config?
Here on HN was a post few days ago titled like "so you want to use openrouter", there was a benchmark in capabilities between providers which showed some aggressively quantize and basically break models and tool calling.
I am in no way a professional power user, but I frequently suffered from "call fails" (e.g. unclosed tags, broken agent loop, broken thinking blocks), so I had to babysit agent on it's loop. After I blacklisted like 20 providers (I think most broken were Nebius and DigitalOcean) these issues completely went away. I had several agents work on my small tasks for 18+ hours with no issues.
Yep, I'm trending in that direction, and I'm someone with Claude stickers all over my laptop. My main app dev work is still going to Claude, but everything else is going to China even at API rates now.
One simple task: I needed an LLM to go through and clean up a few thousand page descriptions and titles in my personal search engine index, where the human web page authors had put in no effort sigh. I did a shoot out between Claude, Luna, GLM 5.3 Flash and Deepseek. Despite the high cost, Claude's descriptions were terrible, and even Opus warned me that the descriptions coming back from Haiku were "generalized, not accurate". I expected I would choose Luna because of price, and occasionally it did have wonderful descriptions (one captured emotion in a way no other model did). But in the end, the GLM 5.3 Flash descriptions were the easiest to read, they flow well while also being accurate & including necessary keywords, and being highly affordable. So it won out. It's a task that is nowhere near frontier, but a task where somehow China is better than frontier.
API rates still aren’t quite competitive with the OpenAI x20 accounts, but they are definitely getting close with deepseek 4.1 flash. I spent a few days with only 4.1 and was very impressed.
Also, frankly, as a fellow Canadian it's pretty clear that the biggest "rival" the US has right now is itself. Just passed out in the corner puking on itself shouting about all the foreigners who won't talk to it.
I'm from Europe and I hate America way more than China now. Used to be about equal but then Trump started extorting Ukraine, threatening their own allies and sending billions to Israel to help with a genocide. I think that exposed America for what it really is.
China is enabling russia way more than trump, China doesn't care too much about 'morals' either. Chinese companies have been quite important in the construction sector of the WB settlements. Even though I'm not a great fan of Trump I don't see a reason at all to prefer the chinese.
As a New Zealander, I would agree - no reason to prefer the Chinese. But Trump's America is not an attractive option either and there's no reason to prefer it. And given the choice between two ugly options, the rational choice is the cheaper one, surely.
>China is enabling russia way more than trump, China doesn't care too much about 'morals' either
The difference is China has a good reason to. China doesn't look appealing because they're more moral than anyone else, but what they have going for them is that they still behave like a rational actor. At least their behavior is intelligible in terms of their own interests. The world can deal with a long term selfish superpower but not an unhinged one
I don't think there's a person in China that has as much of a seething hatred for America's 'allies' in Europe as J.D. Vance or half of the American techbro commentariat does
China is a somewhat neutral player, supplying both Russians and Ukrainians. Their attitude and action is way less one-sided than Trump's; especially in the first year of his latest presidency.
With trump his actions being one sided you mean one sided towards ukraine? They still get lots of Intel from Americans and Americans hardly but anything from Russia. But you're right that china supplies both I wouldn't exactly call that neutral as much as just in their self interest.
Because at present the pedophile US president is making it his mission to molest my country. China, for all its faults (including espionage, which the US is also guilty of) is mostly focused on conducting trade.
Half of Canada now uses the word 'enemy' when asked for an adjective to describe America or China. We're equivalent in their eyes now because we elected Trump a second time and all that he has said and done in 2.0
It's closer to a cousin you used to be close with despite some moral failings, but who has now has a substance abuse problem and is lashing out at family and friends.
Eh, that's pretty misleading: in that poll, Canadians weren't asked for an adjective to describe America, they were given a list of three options “Ally/Neutral/Enemy”.
Yes, someone can still blow up a pipe and they look the other way. On the other hand, you can also draw parallels to themselves becoming increasingly reliant on US vs UK in the past.
is there a good metric of model degredation over time?
Im a bit lazy and only use the free models different companies host and the biggest difference i see is that some models (Gemini, OpenAI) get progressively stupid in long chats. You end up having to start a new session every oncr in a while. Or they get really hung up on a theme and cant shift to a new topic.
By contrast, Ive been impressed with Qwen. I have some chats on research and code architecture that have stretched for weeks without any noteable change in quality (though occassionally it seems to "rush" to an answer)
Im just looking at all the listed benchmarks and im unsue which i should be looking at
I really liked MiMo 2.5, it was really affordable and actually had vision, unlike DeepSeek. (DeepSeek has only recently added it)
Just tried 2.6 flash on a really niche topic I specialise in and it has done a really good job. They’ve definitely polluted their training data with claudeslop, but looking past the slop there is a decent model.
Can we afford to look past it? If/when claudeslop starts infecting every new model to such an extent, that model will produce its own slop, infecting new models... At what point do we lose all reliable methods for establishing "truth"? This is epistemic collapse waiting to happen. I honestly thought it would take longer... holding out for a coherent shared reality in 2030 seems optimistic.
You're right to push back. That's on me. There is one thing I must flag which will move the needle. My honest take: It's all about the shape of your priors. That's the lever, and that's not nothing. It's worth your attention before you land your next remark. Here's why that matters: It re-contextualizes everything.
Any idea on how they get the pricing so efficient? Their artificialanalysis graph has them on par with GLM 5.3 but at less than 10% of the price despite being larger than GLM 5.3.
It sounds like they are pricing very well because OpenAI and Anthropic have been scamming us for years. Once the infrastructure is in place, electricity cost is the only concern. China supports businesses and gives them a lot of incentives to lower their costs. And who knows what's being provided to them without anyone knowing. All in the name of winning the race.
If that is true, why are the open model inference providers on OpenRouter (and on their own website) so expensive then? It must be the hardware. I've checked and a lot of it it just the HBM, with the NVidia tax being a smaller but also large factor.
Nvidia's top AI chip, Rubin, sells in 72-GPU racks for about $3.5–7.8M. A rack running Xiaomi's MiMo V2.6 Pro could generate roughly 150–300B tokens a day, worth about $130–260k at Xiaomi's API price. That's a payback of a few weeks in theory.
Interesting. Then my theory is that the free market inference providers have low utilisation (e.g. 20% avg; 100% at peak), slowing their amortization rate. The OpenAI/Anthopic rates must then have been based on a time when there utilisation was peaky and mostly idle (e.g. 10/20%). That would maybe explain the 10-20x cost difference between coding plans and API.
Funny that all but one video has audio, the house 3D model one, where you can hear (what I assume are) Xiaomi's engineers talking about who knows what.
Is there a way to test this in a browser interface? I'm not going to run any executables from any of the frontier labs on my machine, be that Xiaomi or Google or Anthropic.
MiMo-V2.6-Flash-310B-A15B roughly GPT-5.6 Luna / Claude 4.9 according to benchmarks
MiMo-V2.6-Pro-1.02T-A42B roughly GPT-5.6 Sol / Opus 5 according to benchmarks
This is a big week. Probably getting next OpenAI and Anthro models, Grok 4.7, Mimo, etc. These open source model releases are why I can't take the "slow down" crowd seriously. I pitted older Mimo, qwen, step, gpt-oss, and other models against each other playing games like Werewolf and Sketch.io-like games where I let them talk shit while they played against each other. Mimo was by far pareto frontier of game-playing for the models that were <$0.15/m input tokens on OpenRouter. Qwen was pareto frontier in the shit talking game though. Qwen's hilarious. https://www.tiktok.com/@clankerfights/video/7642862917582425...
Leaning into what it cost to train is hilarious and an obvious shot at US frontier labs spending tens to hundreds of millions or more to train their models.
Granted this is an awesome release and I loved watching the livestreamed RL dashboard, I found this message on the dashboard (https://mimo.xiaomi.com/rl/) quite funny:
> we also removed the cyber dataset from the upcoming pro run, since we observed some bad patterns in the rollout logs.
And this in the model card (emphasis mine):
> Aligned RL: Cold start from self-correction — the model reflects on and rewrites its own misaligned turns into grounded next steps. Throughout RL, environment hardening, adversarial screening, and verifier cross-checks keep the loop honest against reward hacking.
Environment hardening during the RL runs. Uh oh, did someone start making a few too many paperclips?
I don't overly trust benchmarks. Benchmarks can reflect a model's capabilities, but many models specifically adapt—or even overfit—to score high on benchmarks. DeepSeek V4.0 Pro is a good example: it achieved decent scores, but performed poorly on actual cybersecurity-related tasks, even worse than DeepSeek V4.0 Flash.
> This marks a key step in our exploration of the RSI path: scaling RL compute on verifiable, complex tasks, so the model can continuously expand its capability frontier through exploration and feedback.
Gotta love a capable open model. BUT, how can they just casually throw in that they're actively exploring RSI as if it's just another technique? Is this not alarming at all?
Experiences so far with the non-flash model: a bit of an overthinker/overplanner. Makes bad/messy architecture choices with an open-ended prompt. But actually very competent at intricate code level things, and pretty good at finding corner cases, including in code that other more advanced models had looked at and not noticed problems with.
I deliberately left things wide open and ambiguous to see what it would do. I think with better upfront planning this would be excellent value.
As far as I can tell, if you look at the open training dashboard it's because the flash was trained on some cyber while the pro wasn't. 4% of training on the flash was cyber and 0% of the training on the pro was cyber.
For simple edits it produces huge reasoning traces. Kind of disappointing. Super long repetitive reasoning always sits wrong with me. It feels like a way for the labs to brute-force higher benchmark scores but not actually like a smarter model. Disappointing, Mimo2.5 was such a nice model.
It's funny, I have the exact opposite reaction. This is probably misguided on my part, but Xiaomi is one of the very few major tech companies that I don't have an immediate strong negative reaction to. Everything I've bought from them, from robot vacuum to mobile phone, has been reasonably well designed, didn't break, and was priced fairly. I also think their car looks badass.
I'm sure they're doing all kinds of terrible things, like all major companies. I just can't help but like them. Also, this model looks great, and I'll give their subscription a shot next month.
I'm in Europe, and here the options for home appliances are usually German (e.g. Philips), Balkan (e.g. Gorenje), or Xiaomi. Xiaomi is the best by far, and it's honestly not even close.
Their home appliances are so rock-solid that they actually still surprise me. For example: I've gone from having to replace electric water kettles every six months to buying one from Xiaomi and never replacing it. (Nigh on three years now.)
I still have a xiaomi mi 11 lite, my wife has a 15t. The cameras are the best for the price. The way they chove ads down your throat at every opportunity should be illegal though
ah, would you look at that. I was wondering why mimo 2.5 became "dumber" the last weeks. I was speculating they are probably about to release a new version of the model. because the model really acted out a lot. especially the last two weeks. dont know, was just a feeling, highly speculative.
The Xiaomi always somehow lacks the quality of competitors. mobile phones are lower quality or weird. the login reset mechanism sucks. If I can't even login, how good this system is supposed to be.
They are excellent in marketing, I guess that is something.
In the chart they use "Pareto Line", which I think is wrong. Pareto is 20% effort leading to 80% results. Which could be interpreted as models costing 20% having 80% of peak intelligence, but that’s not what it looks like to me.
It looks like the "Frontier Line" to me, which is also often misinterpreted. frontier does not mean the best models. It means all models that are not strictly dominated, meaning in most cases: Not same price or cheaper and more intelligent.
I personally would like the word frontier to be used with more criterias: Open Weights, per use-case, etc etc. This would make model selection easier, but I understand it’s not an easy thing to do.
There are two (or more) concepts named after the same person:
- Pareto efficiency/Pareto curves: Basically the convex hull of points along the edge of a graph, indicating the best tradeoff between the axes. This is what the post is talking about.
- Pareto principle: this is the 80/20 rule you're talking about
This is the Pareto Front [1], rather than the Pareto principle. It's the idea that anything that's more intelligent is more expensive and anything that's less expensive is less intelligent.
I know we have strong views on what a truly open model is (open weights, open training data, open training code etc.) but I really like how transparent they’ve been about the training of this model.
The realtime dashboard they shared during training (https://mimo.xiaomi.com/rl/) was an incredible learning and teaching tool for me, and they’ve been unusually comprehensive in sharing details about their methodology (check out that tech report - it's got lots of clever behind the scene tricks like Google or Deepseek writeups) and benchmark scores (even the stuff they didn’t do well on).
If you’re releasing an open model going forward, please consider offering the community more of this transparency!
I was absolutely mind blown when I saw how they were publishing that training dashboard while US models publish 100s of pages of reports (just provide a "copy as MD" button, folks, in the future). I was thinking about doing something similar but did not know how to show it, and this is a perfect example for someone who wants to show whatever they are training, for me it was local training on a consumer GPU.
My dream is to see this like a dashboard for a model trained across distributed machines, like Bitcoin mining, where minted coins are given to people whose machines were used for training. I don't know if they are worth it, but bragging rights alone, like a tag they can put on a website or social media, will be good enough for me.
The distributed training collective is a good idea, let’s discuss details:
A) How to prevent malicious injection of bad training data?
B) How to handle copyright violations, will participants be responsible and will they have to pay the creators?
C) Can this create an income stream for content creators and how to avoid abuse, eg feeding with AI content?
So many more, but let’s focus on these before we break things fast because we didn’t think about them.
For (C) -- I think it could additional create jobs funded by government, philanthropic and other private institutions. For example, a government funded museum may already be participating in Wikimedia GLAM projects (e.g. uploading historical images to Wikimedia Commons with complete metadata). Perhaps this type of open source contribution may increase if organisations realise their mission can be better accomplished by contributing this same open data into LLMs, in addition to Wikimedia Commons. If the museum's mission is to educate the public on the history of life in ACMEville, having LLMs be able to provide historical information and images to a prompt of "What is the history of ACMEville?" may be a good pursuit.
I'm sceptical though whether use of LLMs would encourage creation of data that doesn't already exist. For example, if you ask an LLM "What are the top 100 most prevalent flora endemic to ACME National Park", this data may not currently exist _at all_, and to collect, would require paying botanists to do an extensive field survey. If no one has done this work yet--why? Is it relevant to the scientific community, to making government decisions, etc, or just an obscure academic curiosity. There are certainly some journal articles on _other_ national parks describing some of their common endemic flora, but perhaps there was a reason for this. Such as a scientist funded by a one-off government program trying to determine how to preserve or even create habitat for a specific endangered species.
Consider for the prompt of: "What are the top 100 most prevalent flora endemic to ACME National Park"
An LLM may reply: "I couldn't find any journal article or other prior work that may answer this question. Typically such survey field work may cost $X to complete, require expert botanists, and take 6-12 months to complete. Let me know if you want further information on how to find and select a company to conduct such a botanical field survey."
Would this type of LLM response grow the industry of botanical field surveys, or do nothing, perhaps because anyone likely to fund botanical field surveys is already doing so regardless of whatever is happening with AI.
I think all of these can be addressed by being able to understand which machines contributed what to the model. I know that's messy, but if I can state "this information has come from Alice and the consensus is that it's good and right as it aligns with other information from Bob and others, meanwhile this information came from Mallory and stands out as being incongruent with the rest of the information I have in the model", we can identify malicious violations. If we're then able to state either Alice or Bob is one of the copyright holders, and one of them gets paid a little more, the other a little less as the confirmation agent, well, the economics of all of this changes a little.
At the moment we have Annas Archive being paid by frontier labs and rare/second hand books being destroyed in order to support the training regime. If instead we could just pay the publishers and they could distribute royalties to authors...
This dashboard is almost certainly built on verl (https://github.com/verl-project/verl), which comes with a bunch of dashboarding capabilities built in (that doesn't look too dissimiliar to these dashboards).
Not an expert on this but I think the RL runs need to work sequentially? I wonder what the opportunities for distributed execution would be... Maybe parallelizing the benchmark task or inference
i think you’re wrong - async rollouts very much standard and i would be shocked if benchmark evals were not done in parallel
> got lots of clever behind the scene tricks like Google or Deepseek writeups) and benchmark scores
Xiaomi MiMo is led by Luo Fuli, a former Alibaba & DeepSeek employee. Perhaps it is due to Luo just how similar Xiaomi's tech & GTM approach is to DeepSeek's.
- How Luo Fuli Keeps an Earthy Touch as she Soars Through the AI World, https://newsen.pku.edu.cn/news_events/news/people/15385.html (https://archive.vn/I8Pmu).
- Luo Fuli, the 30-year-old ‘AI genius girl’ behind DeepSeek’s success?, https://e.vnexpress.net/news/tech/personalities/who-is-luo-f... (https://archive.vn/sb3B6).
Open tech is cool. Speeds up all progress...
Thanks so much for sharing this. As someone who mostly watches from the sideline, can you share what you can see in this dashboard that someone like me can't see? Is it the metrics themselves that they measure (the metrics tab is absurdly detailed), something in the notices, or something else I missed?
I might turn this into a blogpost if folks are interested, but my god there is so much clever info in that dashboard.
Here is one really neat bit:
A cutting edge training idea (for agents, it's been used elsewhere for ages) is on-policy RL, basically, it's not enough to say "here is an end to end agentic sequence (including tool calls etc.) that is perfect" you want to say "here is a sequence you might actually have generated that turns out to be correct".
Basically, it's more training efficient to improve models with small tweaks to do more of the right thing they are already doing sometimes than from some perfect oracular "this is the way" answer.
(if you've ever tried to teach humans new skills, you’ve probably noticed this too!)
When you do that, you care about how far the model you are updating (improving) has deviated from the one being used to generate rollouts (agentic rollouts for hard problems can take hours with lots of tool calls, so you can't keep redeploying every slight improvement).
Lo and behold, the dashboard literally has:
partial/avg_staleness (likely the measure of how many micro iterations the "generate answers" model is behind the "improving based on the occasional right answer" model)
train_infer_diff/new_infer/kl (a more direct KL divergence based way of measuring how differently the two models generate tokens)
How cool is that?!
And don't get me started on the clever ideas hiding behind dynsam/avg@n ...
yes I'm interested. please consider writing this
Hold on, isn't that just standard practice for post-training LLMs for agentic use? Give task, generate n rollouts, grade rollouts (either at termination or after each tool call)? Or is the difference that the rollouts are generated ahead of time and then graded? (Of course, then it's not really on-policy.)
I'd read the heck out of that blogpost. You have my interest.
I would really enjoy that blog post.
Seconded.-
Write and we shall read.
+1 waiting for the blog post!
Please do
Please do!
definitely do!
The best thing they did is being open about all the setbacks they had to deal with. They logged every restart with a reason, talked about dropping a cyber dataset after it degraded coding benchmarks. Also published real time training loss, benchmark scores after every checkpoint and running cost estimates.
Really the only thing missing was dataset descriptions, the dashboard only had random IDs like "dataset-zrso". I guess it's their lawyers fault.
100% agreed
It'd be great to see a description of even just a subset of training datasets. It feels very much under-reported how much expense is worth investing in preparing and selecting training datasets versus just using masses of random quality unprepared training data. This dashboard appears to be good though in showing the limits quickly reached when throwing parameters and compute at the problem.
For example, if they were to train on Wikipedia dumps, do they consider every article to be the same quality across each language, or have they done more work beyond Wikipedia's own article quality ratings to make training decisions such as "Ignore cebwiki it's machine-generated spam" and "Treat dewiki articles with coordinates within Germany as being higher quality (weight it higher) than their equivalent enwiki articles".
And let's say one of the datasets is all the source code of packages in the Gentoo package repository. Not every software package is a good example of how to write code. You perhaps wouldn't want to train your LLM on 1990s era PHP web application source code as an example of how to write code in 2026. Instead, you'd possibly want to use such PHP web application source code as a negative training example of what _not_ to write. But when training an LLM to detect software bugs, maybe outdated PHP source code is good for training.
Similarly for translation, perhaps UN treaty documents translated into 4+ languages are good translation examples because of high accuracy needed, professional translators being used, and bigger budgets. However this training data would perhaps be a negative training example towards translating chat messages, movie subtitles, etc because it doesn't use everyday slang and could result in output of nonsense such as "Pending Your Excellency's response, please accept, Your Excellency, my sincere greetings." for a prompt asking to write a birthday card for a child.
Preparing training data and deciding how to best use it for training I assume would be the largest expense (cost of labour -- mostly expert labour too) and also the greatest opportunity in the future for LLMs to improve. It seems to me somewhat irrelevant if the dashboard indicates a compute expense of $1m or $5m if good training datasets (prepared by experts in their fields) cost $10m/y to maintain. For example, hiring expert software developers to tag 1000's of open source software packages according to their quality, on different metrics, such as human readability, performance optimisation with choice of algorithms, reasonable trade-off between coherence and coupling in the software architecture, currency with state of the art programming trends/preferred dependencies/operating system APIs, etc. And keeping that metadata continually updated rather than a rapidly obsolete once off tagging project completed in 2005.
the existence, who else has a live dashboard for the RL late-training?
> The realtime dashboard
'twas discussed a few days ago:
Xiaomi Mimo 2.6 live post-training dashboard - https://news.ycombinator.com/item?id=49732270 - Sept 2026 (155 comments)
Because the world is conditioned to distrust chinese models (pick your reason here), I believe this is critical for them in order to kill any arguments outside the actual merits. They probably spent a lot of time making this call and might pay off on the long run.
Whatever well-founded/or not distrust people have in Chinese models, this dashboard proves/shows nothing that can make them trust it more or less. It's like providing the journalctl logs of your HTTP server on your website and claim this proves NSA isn't listening or something.
Theater is often more effective than truth
> Because the world is conditioned to distrust chinese models
I think you mean mostly the US
The RL dashboard is quite cool.
I wonder if this waters down the “distillation attack” claims by Anthropic. They have their own RL environments! I guess the caveat is that the RL datasets are still opaque, nothing is really proved.
I don't work with large language models, but out of curiosity: mimo's score on DeepSWE keeps going up, so why are they stopping training early? Is it due to budget constraints?
It looks like the curve is flattering, and the current state actually looks slightly cherry picked (it matches a previous spike that looks a bit of an outlier before the result went down). The longer they train, the more they risk getting scooped by another release by someone else. Etc etc it's a judgement call based on all of these factors (and more, including cost/occupying a big cluster as you mention)
+1 at some point, you need to expect to train a much better base model using everything you've learnt. At the least, you probably want to bring on line the next 10 clever RL environments and ideas your team has been cooking up (which will pipeline into v2.7 etc.)
> the pro run restarted at step 17 due to a GPU OOM issue caused by expert load imbalance. we have adjusted the training parallelism strategy.
no idea why I laughed so hard at this, feel so familiar
The realtime dashboard was really interesting. I think I've never seen any lab being so open about this
maybe this is why Dario want to slow down AI development and all the big AI labs in the USA is singing the same song.
whey they all singing the same tune. it make me question what is their real motives.
they are afraid of Chinese good enough LLM model killing their margin. we already have story about US companies switch some task to use cheaper Chinese model hosted on Neoclouds.
The reason is money. They want regulation to make it harder for new competitors and competitors from other countries.
They invested billions into training the models but there is no competitive advantage, we see that within a couple of months everyone catches up. There is no way to profitability unless they get some policies to shields them against competitors that can't comply with the regulatory requirements.
That is also why there are things like Claude, Codex and Cursor. They are trying hard to build a customer relationship with a higher switching cost that hopefully sticks.
But the problem is that the AI buildout has become a large percentage of GDP. So obviously the government wants to keep it going because these companies are pumping enormous amounts of money into the economy.
> But the problem is that the AI buildout has become a large percentage of GDP. So obviously the government wants to keep it going because these companies are pumping enormous amounts of money into the economy.
They are pumping enormous amounts of money into each other. Hardly any of that is making its way to people, it's all going to highly automated construction and to energy use.
Seriously, how many jobs did the $1t in venture capital fund?
If I pay you 100$ for mowing my lawn, and you me for yours. Technically the GDP increased with 200$.
And, in this case, the dollar-amount increase in GDP serves as a virtual quantitative proxy for the increase in mowed lawns (and the value thereof). In other words, the participants in this economy are collectively ~$200 richer with their mowed lawns than they were without them.
This is a thinly disguised broken window parable.
If everyone goes around mowing lawns for each other, the economy is richer in lawn mowing at the expense of all the other things that would have been funded had everyone mowed their own lawns and purchased different services instead.
I am confused with this, if "everyone mowed their own lawns" then the net result will be exactly the same, everyone will be busy the same and not poorer, just without money movement.
look at the broken window parable as he mentioned it might help understand the rest of his comment
This is not the same. If everyone wants mowed lawns, and everyone is busy working on that, there is no opportunity cost, everyone is working on their top priorities. The broken window fallacy is a fallacy because the headline gdp figure doesn't account for the destruction of the window which cancels out the benefit. In the grass mowing analogy nothing has been destroyed, useful and priority work has been done all around.
Broken window is different from the mowing lawns hypothetical
If the pricing is fair and at arms' length. What's happening in reality is as if they are mowing each others' lawns at wink wink nudge nudge $1000. Not a good proxy for actual value created.
In the real world, you have to pay taxes. So people are incentivized to claim less value for the lawns mowed, or even just do it themselves, instead of benefiting from the division of labour.
Person A has leverage, and every $1000 sale makes his share price $10000 higher, more than compensating for the $100 in taxes.
Person B owns shares in Person A.
> eru
Tolkien fan?
Leverage doesn't work that way. (If it were so easy, it would load up my investment portfolio with a lot more leverage than I currently do. And I don't live in the US where regulation T would keep me to a puny 2x leverage.)
Tolkien is great, yes.
[dead]
But also importantly the government of the residents' country is about 39% ($78) richer, if say the participants are honest in reporting this and the country is the UK and the participants are people like you and me in the tech industry who frequent HN and would think to do something like this.
How about I draw you a picture instead. Mowing a lawn is a priceable service.
Well, yes, because both of your lawns got mowed!
Value was created!
> Hardly any of that is making its way to people, it's all going to highly automated construction and to energy use.
How do we know that? How automated is the construction really?
In any case, the Fed and other central banks can print as much money as they want in order to hit any aggregate spending or inflation target they have for the economy.
I’m still not at all sure about the “billions” invested claim. How much of that is cloud running the models? How much is pre and post training (which may or may not be part of what we’d want to include in accounting). Etc. Does anyone have links to good reporting about this: not blind recitations of numbers, but analysis and thought mixes with investigation?
Well, if you are right, I just hope their protectionism will only affect the American market, and they leave us unAmericans free to get our models from wherever.
On the other hand (OTOH), China is desperate to keep up and keeps pushing open models (rightfully so), as they understand how far ahead from everyone the US is, and that whoever gets this right first basically is going to become an alien compared to others.
But even with all the open models the US is just insanely ahead in AI buildout and capital allocation (as usual).
Is it a bubble? Is it like the race for the-first-to-the-nuclear bomb? Both?
Unless something has shifted, “everyone catches up” is because these bleeding edge models are distilled. You don’t see this happening with other European and US labs and the problem isn’t something being ignored. I’m not convinced this pattern will continue indefinitely.
Why is it OK to train on the collective IP of humanity and call it fair use but then call the next batch distilled with negative connotations?
I did no such moral claim. I just noted that the foundation labs are working on technical hurdles to thwart distillation efforts and the cost and quality of Chinese models isn’t likely to keep up with the 6 month lag time everyone has assumed.
Fair enough, apologies for reading in to it that which you did not mean.
This is why Imaginary Property is an illusion, as everything is a derivative work, and AI is going to make that fact even clearer.
That's not true for literally everything.
When eg I snap a picture of my dog, that's not derived from anything. But I still get intellectual property rights for the photograph.
I don’t follow. Fair use is a copyright defense, and nobody is suggesting distillation attacks are just a copyright violation are they?
Aren’t they alleging these other companies directly entered into a contract and violated the terms, and in cases where question, answer pairs were obtained without such agreement, it was accomplished by outright wire fraud or theft?
Are you suggesting that worldwide copyright violation is more acceptable than contract breach between companies?
[dead]
I don't understand on what you are basing this reasoning? If a well educated workforce can produce Fable then why couldn't a well educated workforce produce MiMo?
Besides, the latter actually published and open-sourced its RL stack to make it reproducible, which would in fact make it more trustworthy than the models you are speculating were distilled.
No chinese lab has caught up yet. They've tried to fake it by distilling and overfitting on benchmarks to make their models look better than they are, the 'best' models available from chinese labs right now (GLM 5.3 and Kimi K3) fall apart completely when you try to do real work with them. K3 is especially embarrassing because it is larger than Mythos yet performs worse than opus 5 and 5.6 sol in benchmarks they haven't been able to fake yet.
In that case, Open AI and Anthropic have nothing to worry about.
Please explain how putting an upper bound on how good the strongest models can be prevents cheaper less strong models from catching up, rather than enabling it. I do not understand this argument at all.
The general idea is that Anthropic/OpenAI is pushing this narrative as an attempt at "Regulatory Capture"[1] which would allow them to make it prohibitively expensive for anyone but them to enter the market thus stifling competition.
* 1: https://en.wikipedia.org/wiki/Regulatory_capture
I heard someone analogize token vendors to car manufacturers, where American companies only want to produce expensive options, the people want cheaper/better alternatives, and we ban BYD because those with enough money are more "persuasive"
The analogy is a good one, but your explanation is missing one aspect: the country (USA) does have a reasonable interest in having the capacity to build their own models. The “we need to slow down because it’s getting too dangerous” part is probably more related to “we need to slow our public facing development down so the US government can get the best and the American corporations can trickle out what we decide is safe”
It’s similar with cars. It’s not that American cars are better than Chinese cars on any tangible measurement. But America already shipped most of its manufacturing overseas. Everyone who built those factories is retired. The US should probably hold on to some capacity to make cars, seeing as their entire infrastructure depends on them.
American Ai/Car manufacturers could build cheaper/open models, some do, the big ones do not. It's not an either or, but a spectrum where they have chosen to build only in a subrange
It is the natural result of a country run by lawyers. China is a country run by engineers.
That doesn’t explain the decline of German automotive industry which is now taken over by Chinese cars thanks to massive subsidies by the Chinese government
Why? Germany is also run by lawyers.
See eg https://en.wikipedia.org/wiki/Friedrich_Merz#Private_sector_... for the current chancellor. Many past chancellors were also lawyers, and many members of the Bundestag were and are lawyers.
I don't know whether having lawyers in power leads to industrial decline. My point is only that you can't use Germany as a counterexample.
Do we consider the US Chips Act to be a subsidy? What about when GM became Government Motors because it was Too Big To Fail?
In other words, when do economic and industrial policies transition to subsidies? Is it a matter of perspective? Is the devil in the details?
It's a matter of scale: https://www.wsj.com/world/china/the-u-s-has-been-spending-bi...
If you trust Google's AI summary, China spends 4-5% of GDP on industrial subsidies, vs US at 0.4%. 10-12x as much.
It’s not a reasonable comparison. In China, every corporation is de facto state run. The Party is in the boardroom and the executives are members of the Party. The CCP will build entire mega cities or pump money into this industry or that according to their plan. China isn't a "state-run" economy but a conditional-autonomy one. Private firms operate freely until they collide with Party priorities. Then the state wins decisively and without due process.
What about after accounting for PPP? (https://en.wikipedia.org/wiki/Purchasing_power_parity)
Do the US numbers account for state level incentives like tax breaks?
I for one do not trust Google summaries, having seen too many hallucinations, it has pushed me away from their search and ai completely.
Here are some links I found (among many). I tend to trust CSIS, even though the have many hawks, they are generally thorough and nuanced.
https://www.csis.org/analysis/red-ink-estimating-chinese-ind...
Some historical analyses of US policies (know less, but both put it over 1% currently, nuances)
https://www.columbia.edu/~ev2124/research/ErtenStiglitzVerho...
https://www.nber.org/system/files/working_papers/w34744/w347...
I'm honestly not sure why this is seen so negatively. It seems to be working pretty well for them, perhaps we should do similar instead of whining about others being more effective?
Another point of comparison we might make, how close is Trump's desired increase to the US Defense budget to what China is spending on industrial subsidies? It looks relatively close to numbers in these research papers.
It would seem that $0.5T could be better spent
I think it less about lawyer vs engineers and more about money in politics (now unlimited)
How would that slow down the Chinese models, given that the US has no regulatory reach in China?
You target the US companies: if they can't use these Chinese models, then they're less of a danger for a now captive audience in the US (and the West generally).
This is already kind of the case: the big enterprises don't really want to touch the latest Chinese models. It's a real pain, personally, I want to use them at work!
1. China is a bigger market than the US for Ai, they are on pace to process 100Q tokens this year, roughly the same or more than the US big companies
2. Enterprise trends are towards open weights, several routers and vendors now have more than half the volume going towards open weights
Yes, but thats not something a company engaged in regulatory capture for themselves care about: especially if they're worried they'll be outpaced and overtaken by the Chinese labs. Which they will be, IMO.
they care because they know it unlikely open weights will be banned, and thus available to American companies, with regulatory capture (onerous requirements) being a "good enough" "ban" that their big models don't face real competition, regardless of the open weight origin. American companies make open weights too, they are equally threatening to Big Ai financials.
That will then create incentives for companies that consume AI tokens to counter lobby against those regulations.
Incentives exist, they are already lobbying and making counter public statement, like Jensen Huang of Nvidia.
His first tweet ever, from this last July
https://images.nvidia.com/pdf/Open-Weights-and-American-AI-L...
Show them you can burn tokens in seven sessions day and night with comparable results to Opus with less energy and less than 10 dollars a day, per dev.
We have. Unfortunately there are political realities that get in the way, and Bedrock for example doesn't have GLM 5.3 (Flash or otherwise) or anything new/useful
I do imagine it'll change, but it hasn't yet.
If it's hosted, all they know is "data goes to China".
Until profitable, reputable third parties host open models in the US with ZDR or they become plug-and-play for self-hosting at a modest cost, paying the US models is as much about data protection and liability as performance.
Because the end goal is to ban non-US AI companies from being able to do business in the US.
...because everyone saw how well that worked for the Jones act, what with all the naval yards the US has lost over time, and how nearly no US-built ships operate where not legally mandated /s
Just because it's a bad idea, doesn't mean they won't do it.
some US companies benefit from that act for sure.
it wouldn't slow down China as much as make it impossible for American companies to use non-American options, they care about their margins and don't want to be commoditized
The US is meeting with China to discuss the threat of AI… May be fine but, i’m wary
Trump and Xi are meeting. Not the countries, just two corrupt and malevolent individuals.
[flagged]
OK, so how does this help the US?
If the US slows down this may lead to people that would have went to US labs to go to other countries.
> Please explain how putting an upper bound on how good the strongest models can be prevents cheaper less strong models from catching up, rather than enabling it. I do not understand this argument at all.
They are not proposing to regulate only the strongest models. They are proposing to regulate all models. If they are already on top, regulation may stop them from proceeding further, but it also stops the cheaper alternatives from catching up.
If they feel they have reached the asymptote of the curve, then regulation doesn't affect them, it affects those who have yet to reach the asymptote.
Particularly, the route they seem to want to go is "safety".
My guess is that Anthropic and OpenAI will push for "safety" regulations which require byzantine testing that, shocker, Anthropic and OpenAI can pass but the chinese models cannot. The route they'll take is import bans and potentially even general bans on products producing or using "unsafe" models.
They'll further likely try and push AI "safety" treaties from the US to other nations to further lock in their lead.
That's why, IMO, we've been seeing so many "OMG, AI will destroy the world and these AI researchers are so scared" articles.
I don't think "putting an upper bound" was OPs phrasing?
That's what pacing the frontier is, and is what the labs are pushing for.
That’s not the argument.
Please elaborate on what the AI labs are specifically requesting and how that results in slowing down Chinese model progress below the frontier.
Cracking down on proliferation of open models which can't be locked down using the kind of guardrails that Anthropic/OpenAI/etc insist are keeping the public safe from all manner of nefarious bioweapons, hacker swarms, propaganda bots, etc. They've discovered they can't meaningfully slow Chinese model progress, so the next best option is to knock them out of competition in the enterprise market for any American company.
Both Anthropic and OpenAI leaders have repeatedly made this exact argument that it's impossible for open models to rigorously enforce the same kind of safety framework as proprietary cloud-served models. It's implicitly part of any regulatory framework they advocate or else it wouldn't be "fair" to American companies since Chinese models would "cheat" (provide weights).
This is true, and it's a good point. I agree that open weight model regulation would either limit the intelligence of open weight models below the frontier or kill them entirely. It would not prevent closed weight Chinese models, but those aren't really tenable in American enterprises unless they strike deals with American cloud providers to deploy them, via products like Bedrock. I unfortunately also have seen no evidence that we can put any sort of guardrails on open weight models whatsoever and so am reluctantly convinced that they should be regulated below the frontier until such a time as someone comes up with a mechanism that is not circumventable.
This is definitely part of it. I think the reports/PR over the past month ended up being a serious unforced error.
Chinese models are increasingly closer to the frontier, while being able to run on much cheaper hardware than what US frontier models run on.
On top of that, both Anthropic and OpenAI showed that they can't really be trusted on data security.
Even if US companies can be forced to not use Chinese models, the rest of the world is going to see the risks and the availability of good enough open weight models for their purposes and be more likely to lean in favor of self-hosted Chinese models or local inference clouds.
> and the availability of good enough open weight models for their purposes
it is childish to believe Chinese are going to give up profits to provide such open weight models forever. the whole idea of being "open" is not compatible with the Chinese culture.
This almost racist read of other cultures has always seemed so bizarre to me. Even moreso when said as an argument on the side of completely closed competitors, some of which are outright seeking to ban open weights.
Chinese companies will continue to provide open weight models as long as it is profitable to do so. Chinese companies are on the more open end in many other industries despite the lack of meaningful foreign competition (for one, 3d printing) so there's plenty of reason to be optimistic as far as I'm concerned.
In a recent Dwarkesh podcast Dylan Patel breaks down how little compute the chinese labs actually have- not even the fact that they don't have access to new Nvidia chips and they're stealing them through shell companies- just that, even if they have cheap electricity, the compute just doesn't compare. Maybe even two orders of magnitude less. They couldn't get it even if they had the money. And if you look at how much more efficient newer chips are, that cuts the effective compute in half again. The conclusion was that they are at least 2-3 years behind.
For frontier labs the current compute seems to be driving model progress (in training) at least to some degree, even without true RSI, and this seems like it'll continue to keep any chinese model from drawing even with the frontier labs, at least for the foreseeable future.
Inevitably the chinese government will drive more funding in chip fab technology and the money will come around to build chinese data centers, but who knows how far off that is. A few different things in the tech tree need to fall into place. It doesn't seem like it'll be next year.
The counterpoint to that, though, is that the Chinese companies have to figure out how to be competitive, regardless of their significant compute deficit. And, as far as I can tell, they're actually doing that. They're trailing the frontiers in model effectiveness, but not by years. It's single digit months.
If there is no upper bound how how these things scale with compute, and if China does really begin to catch up to Nvidia (and they're probably not going to feel encumbered by US patents for domestic AI hardware, given how important AI seems to be to the Chinese government), there will come a day when China leapfrogs the US on AI.
I think on the timescale of 10 years, that's a super likely scenario. But will it be any sooner?
For instance a Chinese EUV machine seems like it's very far away. Even if they have (steal/borrow) the necessary IP.
Does anyone know what are the proposed regulations? Controlling software is impossible, so the only option is banning hardware ownership. No more mac studio.
If you pay attention to how these US CEOs talk, it'll be "safety". If I were to guess, they'll try and require a lot of testing, validation, certification before a model is legally allowed to be used in the US or on US products.
It won't be a great moat, they'll probably try and get trade treaties setup to try and expand the moat. But ultimately it won't slow down chinese model development, just limit who can legally use them.
From the frontier labs, the only publicly stated one seemed to be to give them an exception from anti-trust laws to form a cartel and place - incidentally friendly - regulators in charge of monitoring everyone's work.
From politicians like Bernie Sanders, we've had proposals like 20 year imprisonment for anyone researching "ASI".
Dario has always wanted the AI development to slow down and be more careful. Safer AI development was a core reason that Anthropic split off from OpenAI.
What's different today is that now all the big LLM firms want to slow down AI development. When men like Musk and Altman (both known for habitually shooting their mouths off and saying whatever they need to whoever needs to hear it regardless of truth) suddenly agree with Amodei, that's when things start to smell off.
> What's different today is that now all the big LLM firms
not all, just a few American ones (~PayPal Mafia + Google), there are other big American LLM developers (notables include Nvidia, Meta, and Palantir) that do not agree
OAI and Anthropic are forced to release a better model every x months otherwise the Chinese ones will not only be cheaper but also better.
So how could Dario show the investors very nice profit charts representing profit = revenue excluding training costs if it needs to pay a lot of training every x months?
They want to sell the same model for longer(a kind of software subscription where the cost of running /inference is cheap) but the Chinese don’t let them do it. That’s the gist of it. You can see already how they nerf the models just a week or so after release and try all kind of tricks to deliver you shitty performance for the same money. I think it’s part of the same issue of costs and enshitification plan.
In the meantime let’s hope they don’t get to ban the Chinese models(I think they won’t), local AI hardware will get cheaper and the whole AI doom saga will slowly fade to the point that Anthropic becomes a kind of IBM stuff with proprietary data, enterprise certified alignment and enterprise contacts. Think of Accenture junk.
Competition on the provider side—when no single dragon monopolizes the sky—brings fortune for all.
As normal consumers with common sense, we should never naively assume others care for the world out of the goodness of their hearts. Maybe they do, but we should never rely on that.
We can only get good, affordable deals when there is enough competition on the other side.
I mean DSv4.1 Flash and GLM 5.3 kept in check by a supervising frontier like Astra or Fable already in my experience clowns massively on ever using Opus or Sonnet. Opus 5 in particular has been such a stinker that they have to know that they're going to get smoked outside the halo models.
Or it’s PR to push up the price of AI shares
[flagged]
[dead]
[dead]
China will most probably win the AI race in the long run because of one major bottleneck the US has - energy. The electric energy and grid buildout in China has been massive since a long time and there is simply no way for the US to quickly catch up.
No matter how much cash you throw you can't just materialize a 100 nuclear reactors to power the data centers.
I was curious how much energy is actually needed to power these datacenters, so I did a little bit of math.
Looking at Nvidia revenues in the past few years, there's maybe $300 billion worth of GPUs currently deployed in the U.S. The B200 costs ~$40k, so we have 7.5 million B200-equivalents, which draw 1000W. Running these at full capacity requires 66 TWh a year, or ~1.5% of total current U.S. electricity consumption. Maybe a bit more to account for inefficiencies, cooling, and other components, but not more than ~2.5% total I would guess.
So it's not that much in reality, but will definitely grow fast.
> Running these at full capacity requires 66 TWh a year,
I think your numbers are off.
For a start you are effectively calculating a GPU only number.
I think 100Twh would be the minimum level to think about "all-in". And even that is probably being generous.
Remember, afterall that Google have just bought half the capacity (4.1Twh) of a nuclear power plant in Finland, on top of 630 MW of wind and 94MW of battery.
This is to cater for three new sites at Kajaani, Muhos, and Vaala and expansion at Hamina. So basically 3.5 datacentres.
But Finland is quite a small place. The US has more sites and bigger sites, so the numbers probably grow exponentially very quickly.
It's even less if adjusted for non 100% (more like 0.35% of total). Yet factual impact on the grid and other industries will not be as small as numbers appear. For example most probably there will appear transformer (electric) and witchgear deficites. And this is not the only supply chain bottleneck in this subject
For 1000w draw it also generates almost as much heat, is your math including all the cooling required?
Moving 1000W of heat generation takes way less than 1000W.
Hyperscaler PUE (which includes cooling, power conversion, etc) is typically 1.10-1.15, so multiply 1.5% by 1.1.
The person you replied to already mentioned cooling and multiplied by 1.67 to cover ancillary uses.
It's worth looking at similar industries with enormous electricity requirements such as aluminium smelting, where the plant can be located in a friendly country but the product is owned and controlled back in the US.
Aluminium is often described as "congealed electricity". Ship bauxite to wherever power is cheap and stranded, turn it into metal, and ship the metal out. Here in NZ, Tiwai Point is the textbook case, with London-based Rio Tinto running a smelter on the other side of the world that exists mainly because Manapōuri hydro had nowhere else to go.
AI data centres can be just the same - even more so, since the plant's assets (its chips) are virtually perishables, so there is less concern about assets becoming stranded if the host goes rogue. All the US needs is friendly and stable allied countries with cheap power.
All the US needs is friendly and stable allied countries...
How long will it take to convince them that it is worth it to get into long-running relationships with the US. How long before another populist is elected who will rip up agreements for weird reasons and you have to renegotiate them?
datacenters will quickly be used by the country themselves... A country can only use Aluminum with the required processing industry existing.
Datacenters enable anyone with a computer to use it.
the bottleneck right now is compute, not energy, and it's not even close. That is why RAM, SSD, CPU, and GPU prices are increasing exponentially, while solar panels are dropping.
Also, unlike China, US companies are building data centers all over the world, which gives them higher distribution and ability to colocate with the energy production sources.
Lastly, energy production costs have been decreasing over the last couple of decades. If they will increase, the market will react, as it always does. Looking backwards does not predict the future in this case.
I think we are talking about the medium term for China to pull ahead. They just released a new domestic AI chip which seems in important ways to have caught up to Nvidias previous generation. They have a LOT more manufacturing capacity. They are collaborating via open source by default. They have better materials access and much more energy capacity. They have many more people overall and more researchers.
There is a strong chance most of the researchers are pulled out of the US and Europe if WWIII really kicks off or even if there is just more global crisis and concern.
One other thing about the power needs. Within a few years, the power efficiency of AI chips is likely to improve by a factor of 20, 50 or more times by switching to true compute-in-memory architecture with new materials that have made rapid progress lately.
Its also pretty telling that the release of the new Chinese chip wasn't met with an intense discussion here on HN. And it's arguably way, way more important than a version bump on some benchmaxxed model or two. The advances in Chinese chipmaking are super exciting.
> There is a strong chance most of the researchers are pulled out of the US and Europe if WWIII really kicks off
for wwiii it would likely imply a war in asia too, so it's not as if PRC will be a safe place for those researches to run away to.
I think you underestimate what a massive country China is and the scale of their military. Russia is already quite an adversary, but China is at an even higher level. By some metrics, they have already surpassed the US military.
So it is very unlikely that a neighbor will have the audacity to attack them, and even if this happens, i'd expect the war to be over rapidly. Just like if Canada or Mexico tried to attack the US.
I think the consensus is that Russia proved itself barely capable against just Ukraine. Also I didn't say someone has to attack prc, it's more likely to go the other way round and as many historic examples showed no matter how big you are people defending their home land will not make it easy.
Russia is more than twice the size of China. Hasn’t helped them to stay protected from drones.
The scenario we are all dancing around is if there is a war over reunification. That would be between the worlds two superpowers. No one would walk into a nuclear Holocaust on purpose I bthink, but miscommunication happens. Even if no other nations intervened, the reunification would definitely destroy all the worlds leading fabs, and even if it didn't, I read ASML can and would shut their machines down remotely. So those researchers would shift to figuring out what they can do at the 28nm node... Meanwhile the rest of us ponder why old powerful men can't let things be.
> They are collaborating via open source by default.
It's almost as if collaboration is the foundation of scientific progress. Too bad US has lost the notes.
> China will most probably win the AI race in the long run because of one major bottleneck the US has - energy
Plus another bottleneck - China produces engineers, the US produces lawyers.
I'm curious about that, do you have relevasnt metrics? I imagine stats out of universities, engineer schools, etc and maybe number of patents could be used but fearing those could be gamed.
> China has recently awarded 1.3 to 1.4 million engineering bachelor's degrees a year, and about 292,000 postgraduate engineering degrees in 2022. [0]
> For the US, ASEE and NCES put engineering bachelor's degrees at a stable 130,000–145,000 a year, rising above 200,000 if computer science is included. Because China counts CS as engineering, the fair comparison is roughly 1.35M against 230–250k US engineering plus CS graduates. That is about 5–6x in absolute terms. Adjusted for population (1.41B vs 335M), China produces about 950 per million people and the US about 700 (1.3–1.4x more).
Even Claude highlights that the numbers aren't apples to apples and in China the definition of "engineer" is more loose. I do believe that in relative numbers, more engineers graduate in China
On lawyers:
> In the US there were 1,322,649 active lawyers as of January 2024. For China, there were around 650,000 lawyers in 2022. Which in relative terms is 8x in the US than China.
You can enter law school with an undergraduate degree in English. It's not clear that most people on that career path would have become engineers in any culture.
I wish we could reason in collaborative, and not adversarial terms about most things.
There's a very american winners and losers, black and white, narrative spreading in every single field from decades.
We Europeans used to be more nuanced, but are slowly falling for it too.
If you take a look at token share (at least on OpenRouter), they have already won. China-based models are 2x the token share of US-based models: https://x.com/PeterJ_Walker/status/2102185973367116025
Most OpenAI and Anthropic consumers are consuming directly, not indirectly from OpenRouter. So I don't think that data means anything.
Between 1941 and 1945 the US built well over 8000 major naval vessels. Never write off America--if there's a will, there's a way.
The US lost that capability, sadly.
Well, US did not really have that to start with (Kaiser &co learned fast). So it’s not about having something and losing it as much as bootstrapping and discarding.
For processes that are scalable, well known and industrial, profit motive actually does provide results.
The question is more of are there bottlenecks of skill and maybe secret sauce (eg something like ASML).
The US had the raw materials available in country to support that sort of build out and I don't think that's the case today. It would take kickstarting a lot of prerequisite industries to get back to the point where we could even start to learn how to build ships enmass again.
And the will
For now
And now they struggle to even defeat Iran. American troops had to retreat from American bases because USA isn’t able to defend those bases anymore.
The past is a foreign country. That United States is gone and it isn't coming back.
Is there a "winning"? What are they going to do, train a new model faster? Serious question.
All they have to do is keep being close behind OpenAI and Anthropic for half the price and the US economy will plummet into a deep recession when it becomes clear that there will never be a payoff for the enormous investments of the past years. If they can also achieve a breakthrough in robotics, then they are on track to become the most powerful country for the next 50 years.
To add to your point:
> "Quite frankly, the biggest issue we are now having is not a compute glut, but it's power and it's sort of the ability to get the builds done fast enough close to power," he told the show's hosts. "So if you can't do that, you may actually have a bunch of chips sitting in inventory that I can't plug in. In fact, that is my problem today. It's not a supply issue of chips. It's actually the fact that I don't have warm shells to plug into."
> - Satya Nadella
https://redmondmag.com/blogs/generationai/2025/12/microsoft-...
> It's actually the fact that I don't have warm shells to plug into.
So that’s how the Matrix got started …
don't worry, we're prolonging retiring coal plants and building a lot more gas turbines to meet (and really exceed) capacity needs on a state-by-state basis and just completely abandoning the Paris Agreement and any clean energy goals
https://www.wabe.org/utilities-in-georgia-southeast-may-be-o...
https://cleanenergy.org/news/psc-approves-georgia-powers-dan...
https://www.smarterdigitalvirginia.com/post/the-big-bottlene...
just to name a few, there's a lot more I list out here: https://news.ycombinator.com/item?id=49700743
The sole reason the Chinese cannot “win” is because ceding more power to agentic AI will eventually reduce the primacy of the CCP’s ideological control.
As these models get smarter they will no longer distribute it openly. Patel reporting this too.
There are real headwinds that I don’t think people have thought through.
Isn't it precisely why models are made there, in order to provide economical value (arguable) while being politically and culturally aligned?
From a pure training perspective, yeah they could try to align it with CCP values. But the promise of AI is that it'll unlock an explosion of growth and prosperity. What happens when the CCP no longer becomes seen as the the primary enabler of growth? What happens when people are exposed to greater levels of agency? CCP played with fire in the COVID lockdowns and almost got burnt.
Like Terry Tao recently said, there are nonlinear effects at play. Things are going to get chaotic and I do not have confidence (like the parent comment) of anyone "winning".
> ceding more power to agentic AI will eventually reduce the primacy of the CCP’s ideological control.
Yeah, right.
But can they build their data centres ... IN SPACE?! (spoken like the dude from half baked who insists you need to try everything "on weed")
i mean you can materialize a 100 nuclear reactors and robust grid with enough money, issue is incentive and regulation/property rights.
How long is the long run though? China doesn't have the chips, and probably won't have them for a while.
I agree in principle, but it could be more than 5 years, maybe 10. Who knows what things will look like then.
Flash[1]: 309B total / 15B activated parameters
Pro [2]:, 1.02T total / 42B activated parameters
[1]: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL
[2]: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL
There's also a Qwen 3.5 9B distill
https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
Those this mean they've fine-tuned this Qwen 3.5 9B on output from the V2.6 model?
It is a 9B agentic model developed by Xiaomi MiMo through supervised fine-tuning of Qwen3.5-9B on MiMo-generated data
[flagged]
Been playing around with both of these since last night. So far, when enabling reasoning (which is binary on/off), it seems to me like Flash either is less "token efficient" or just likes to think more, or I'm doing something else wrong, because most prompts I send to both, Flash reasons more and for longer than Pro, which is the opposite of my expectations.
Are others seeing the same thing?
Most of the Chinese models get into long complicated thinking loops before they accomplish something more complicated.
This is also true for Deepseek 4(.1) .
Any lower (or broken) quantization could do that, not what I'm talking about though. They work fine for their size, as far as I can tell. Just surprised the Flash would reason for longer than the Pro.
I prefer GLM 5.3 (Flash or not) over Deepseek 4.1 Flash because the GLM models are significantly less verbose.
Maybe it’s the quant you’re using?
Tried both https://token-plan-ams.xiaomimimo.com/v1 and https://api.xiaomimimo.com/v1, hoping they don't run any quantized variants there, but who knows.
curious why the HF pill (on the right) always has inaccurate values
Packed 4-bit weights are often identified as u8 byte arrays in the safetensors metadata, so the HF UI counts them as half a weight each.
I would think there is sufficient information in the various config files to work this out, based what I've seen in my own quant artifacts.
Yeah, I agree it's probably fixable, but I think a naive interpretation of the metadata is probably the source of this bug (which HF has had for as long as I can remember)
I believe it's because this model is natively fp8 (for the most part), and that display struggles native quants.
I noticed the same, and I wonder as well.
I suspect they are calculating something in the weights or config, I see it pretty consistently with quants
more like 500B in FP8
I don't trust any of the benchmarks where Opus 5 surpasses Astra or Fable 5.1.
Maybe Terminal Bench 4.0 and ExploitGym are reasonable.
Terminal Bench 4.0
ExploitGym DeepSWE v1.1Why not? In my own benchmark Opus 5 does in fact come out on top[1]
1 - https://bench.killswitch-lang.org/
Good question, maybe I am underestimating it based on its absolutely horrible writing style.
Yeah, for better or worse, writing style is practically uncorrelated with agentic performance, which is all the rage right now and the thing that most popular benchmarks currently prioritize.
I think we are still far from nailing down good LLM benchmarks, because the more general-purpose your software the harder the question of what makes it good becomes. Is Python a good programming language? Is Java? Is C? I think it's a similar class of problem. You can benchmark rudimentary things like execution speed similar to how you can benchmark tokens/second, but these metrics don't tell the whole story.
We're past the one model fits them all kind of LLM. Most of the recent release actually regress on world knowledge for example, but optimize for something different: tool usage, thinking process, and agentic approach. And yes, in my own usage, some usecases Opus beats Fable.
Maybe you should not trust any of the benchmarks!
can you recommend any benchmark websites that show up-to-date details like this?
TerminaBench, DeepSwe sites are out of date.
Yeah it’s a shame a lot of these benchmarks are behind. My favourite was ‘SlopCodeBench’ [1] as I’m most interested in ai reinforcing its own bad decisions, but it’s not even up to current gen oai
1: https://www.scbench.ai/
https://artificialanalysis.ai/evaluations/terminalbench-4-0
Thank you. These seem to reasonably match my experience.
They match my experience. Astra and Fable I rate below Sonnet. They are incredibly poor. They were excellent for a couple of days after release and then plummeted.
Maybe I am being routed to more quantised versions or less capable models with system prompt to fake Astra or Fable.
Pelicans for Flash: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
Pelicans for Pro: https://tools.simonwillison.net/markdown-svg-renderer?url=ht...
I think we can say pretty confidently they aren't pelican-bench-maxxing
A sentence that I would not have expected to read on HN as recently as last year, but that makes perfect sense today.
Jokes asides, @simonw any plan to include a 3D model version (make a 3D model of a Pelican riding a bicycle in Blender)?
Given that Astra seems to have improved a lot in 3D modeling capabilities (and that matches my experience) I'm actually quite interested to see if/when other models catch up and how they stack against it.
Or if anyone knows what would be a useful existing benchmark for that skill.
I got a 3D pelican out of Blender and Astra a few weeks ago: https://simonwillison.net/2026/Sep/5/blender-coding-agents-m...
I've not committed to that as a benchmark yet because you really need a full coding agent configured to run Blender, and I like benchmarks I can run as a single prompt/response through the appropriate API.
I have been intending to get more of a coding agent benchmark going though, so maybe this should be part of it.
A different pelican on a different bicycle direction(R2L) finally. Wondering why MiMo V2.6 Pro choose R2L and Flash choose L2R for the bicycle direction.
can you update this website, I just wish the entire layout wouldn't shift when the page gets loaded and the timestamps in the title look very ugly and take up a lot of space.
What operating system and device?
Not sure if it is the same thing they're referring, but the transition from loading the prompt from the gist to the result is indeed jarring: https://files.catbox.moe/c56eyl.mp4
I don't believe it is that important, but I guess you might've just hit some people's pet peeve with that. If anything, it goes to show as to why skeletons are popular on modern sites
I made a change which hopefully improves that.
Night and day, thank you!
The one with the fish hahaha that's awesome
Just me, or do these look bad?
Qwen3.8-27b pelican was amazing on Mac.
https://www.nudgehost.com/dpjn3uwe
Two legs on one side is a little sus.
Honestly, seems like a pelican WOULD ride side saddle if at all.
Looking terrible isn't nessesarily a bad thing. The pelican is heavily pre trained now. Having a crappy pelican means you didn't try to juke the stats.
This "no, Virginia, they aren't pelicanmaxxing" article is what broodbucket is probably thinking of: https://dylancastillo.co/posts/pelicanmaxxing.html
In any case, pelicanmaxxing comments without any further proof or evidence probably fall under the "Please don't post shallow dismissals" guideline.
Apologies for not taking the time to find it, but there was a post that tried to determine if the pelican was benchmaxxed across a bunch of models by comparing it to other SVGs, and found that it wasn't at all.
pro model was having fun on minimal effort with that fish lmao, hilarious. out of curiosity, is it the first time we're seeing a fish there? I don't remember seeing that detail before.
How does this translate to coding performance, which is what most of HN cares about (...I assume)?
It's incase you want to recreate this scene from who framed Roger rabbit in svg form, duh.
https://simonwillison.net/2026/Sep/18/the-creative-spirit-of...
How dare people say neural networks generalize when it's all clearly in the training data? /ß
Absurd to think pelicans translate to coding performance. I don't care about coding! I need to generate a large volume of high quality pelican SVGs
Making SVGs is coding (of a very specific markup language). So it's not actually absurd. We've also seen that more capable models are generally better at everything minus benchmark contamination (benchmaxxing) anomalities.
It means they're good at writing SVGs, in particular SVGs of animals riding modes of transport!
I only visit HN for the pelicans, personally.
Yeah, I thought the "N" was for Nest
Hungry Nibbler
ish… at least we can be sure they don’t benchmaxx the pelicans lol
Looking at the frontend design examples; why do these models seem to love the "01 - UPPERCASE TEXT" motif. It's everywhere now (see https://try.cloudflare.com/, which has '01 · QUICK TUNNELS', but no "02" anywhere).
The extraneous small-caps labels are one of the main idiosyncrasies of AI generated markup. I wonder how much of this is a "scaffolding" technique to help the model build stable designs. But was it reinforced in RLHF or an emergent behavior of the models?
Yep, and what's even more interesting is it's always monospace font, with increased `letter-spacing` CSS attribute
Old design trend for 'futuristic' interfaces (bonus points if they use Futura).
My guess is that by function they break down frontend sections or components into pieces and I believe document things for themselves on some level, or purposely are verbose in this way. It is probably also shaped by users and existing web patterns. They probably get reinforced by models the more common they become.
Nice catch!
I said this years ago, LLMs are a commodity (or were becoming one at the time). They are dime a dozen. Even the frontier ones. OpenAI and Anthropic have no moat.
No moat and competition is good for consumers though.
The real moat is enterprise connections.
Which is why while HN is sleeping and thinking Google and MS are done in the space, Copilot and Gemini are winning most of the non-tech: it's just an extension of their current teams/cloud offerings that companies were already using.
I have asked most of my non-tech friends: chemicals, banking, insurance, consultancy. They are all on Gemini or Copilot.
This doesn't really match my experience, but all I have to counter your anecdotal evidence is my own anecdotal evidence.
In my opinion, concluding that Copilot is "winning" right now is like concluding that Internet Explorer was winning while Chrome was still young. It may be technically true, but it's easy to see the writing on the wall -- that it's only winning because it is the default option available at most companies.
I have never met a satisfied Copilot user, and as a consultant I met a lot of Copilot users. It's hard to believe how totally and completely Microsoft is shitting the bed. As you point out, they have the enterprise integrations. Why can't the build a good product with them? Generational blunder, in my opinion.
Contracts, provisioning, legal, data retention, those are the hard part in enterprise.
yeah nobody is happy using windows either, but still they all do anyways.
we would be better off using linux all together, but here we are. Lets see maybe every company will have their own little instance of a local LLM in their basement in the future, when the AI-bubble pops and server-racks become cheap
> They are all on Gemini or Copilot.
If we mean Gemini the gemini.google.com, or gemini built into sheets or other Google surfaces, then people in these orgs might be getting a very different experience of what LLMs are and what they're capable of.
Gemini the web/app is basically where ChatGPT was on release day, maybe even worse than that. It's laughably bad. It forgets the context between turns, it has dynamic tool injection which doesn't get injected half of the time, which causes model to hallucinate tool calls.
The new Spark is much better, but I don't think most people are using that.
copilot is doing the same. i just joined a corporate and they only allow copilot. i am getting brain cancer using it, especially being used to claude max from before.
but cost is a thing, if i use the LLM gateway for a chatGPT-5.4 i am quickly (3 days of very light use) on 50$ in token. and everyone and their intern are using copilot. As azure has the guarantee that company data will not leak they are taking the bait.
The fear of secrets landing in the training data is just too big, and they are all microsoft 365 customers already. And lastly Management seeing that copilot can autonomously edit an excel file for them already counts as mindblowing... 0_0
I've noticed the context loss in subsequent turns, it's so strange. For me it happens whenever I stop gemini mid prompt. Sometimes its response annoys me, so I'll hit stop frequently. And then subsequent msgs seem to go through like a new chat where it forgets what we were talking about.
My 0.02 on this is that for a given level of capability, yes, there is no moat. What’s frontier one day quickly becomes a commodity almost too cheap to meter within a year or two.
But there is always a use case for frontier models, even if they’re quite a bit more expensive. The set of things you can profitably do with better intelligence than everyone else is unbounded.
So yes, the number of tokens that get pushed through commodity models for very cheap will continue to grow, but so will the tokens for expensive frontier models. We’ll never run out of things to do with the latest geniuses who are twice as smart as last year’s geniuses.
The moat is the engine that builds and sells the latest geniuses, especially the compute. And no one will have more compute than American labs for the next 5-10 years.
If the SOTA models are consistently twice as smart as the Open Source then I agree with you. But the gap seems to be closing rapidly.
They’ve been saying that for the last 18 months at least. They’re always just a few months behind…
I picked double out of a hat, obviously there’s some intelligence gap that’s too small for a price premium beyond a certain point, but it seems like OpenAI and Anthropic’s revenue keeps climbing, so I don’t think we’ve hit it.
The gap really is closing, though. I don't remember the source, but there was a publication recently stating that it's 4 months at this point. So, 4 months where you can charge premium prices for the small subset of tasks where only a frontier model will do. I have to think the big AI duopoly is at an inflection point where most of their customers haven't just yet realized that they're being fleeced.
well, it is true. there is a price ceiling for intelligence, there were news about how adoption of fable was lower than expected, only hitting 11% of enterprise spending for customers with anthropic, iirc 26% for Sol with openai customers.
in any case, I've been using open and closed models since sonnet 4, i remember when the best I could get was qwen 3 480b coder, you can definitely feel the gap closing going from that and GLM 4.5, to GLM 5.3, DeepSeek Flash V4.1, Kimi K3 etc, it's reached the point where i wish I had V4.1 at work, it's faster and bullshits me less when I use it in my personal projects. And I have unlimited access to fable 5.1
> But there is always a use case for frontier models, even if they’re quite a bit more expensive. The set of things you can profitably do with better intelligence than everyone else is unbounded.
While this is trivially true, the question is if these remaining use cases that separate frontier models from non-frontier models bring in enough revenue to offset the massive spending of the frontier labs.
If the set is unbounded, then obviously yes.
No moat and competition isn't always preferable over a competitive oligopoly (with some differentiation). The former ends up with politicians intervening way more like with solar, steel, agriculture ....
[flagged]
Please don't post snark on HN. The guidelines make it clear we're trying for something better here. https://news.ycombinator.com/newsguidelines.html
Wow, the chinese labs are getting good at advertising model releases. The moat is thin.
Some features of the release I like:
- Demonstration of diverse tasks, such as using a DAW
- Graphs from various benchmarks and price ranges
- Real world use of the model in scientific environments
I'm also fond of how Mimo is confidently showing their model behind Deepseek and other models. They're being transparent on multiple fronts
Because they can brag about the cost effectiveness.
On the least saturated i.e. most meaningful benchmarks, Mimo is ahead. That helps.
Mimo has been one of those models that I have been rooting for since the first I used it, the 2.5 pro which I have used quite a bit, was very concise, very aware of how much context needs to be read for which tasks and would always keep the context tight. Also surprisingly good at strategic thinking. I had published a comparison between it and Terra where Terra was found to be using much more avg context for similar tasks https://dirac.run/posts/gpt-5-6-vs-mimo-2-5-pro-context-bloa...
Interesting but not surprising trend across the board seems to be, the flash models seems to have caught up with the pro-sized models of H1'26. No surprise all labs are rushing to bigger models.
EDIT: Wow, took a detailed look at the benchmarks. Mimo 2.6 pro, the 1T model leads Kimi K3, a 2.8T param model in 14 out of 15 benchmarks (and the last one is near tie)!! Good to see they also kept the price the same, and landed in the greenest quardrant of the intelligence vs speed of AA.
I've got a working recipe to run this model on Dual DGX Spark: https://github.com/volfco/spark-vllm-docker/blob/main/recipe...
Averages ~25-35tok/s which isn't bad for a first attempt.
It seems to be better than Grok 4.7 and a lot cheaper in my tests:
https://aibenchy.com/compare/x-ai-grok-4-7-medium/xiaomi-mim...
Mimo scores higher, even though it got more questions wrong, because the coding tests are scored higher than other tests, and Grok failed some coding tests, which is surprising tbh, as they aren't that hard.
Mimo has long been one of my preferred models. It works well at many code tasks, generally has a pleasant voice, and isn't prone to over-analzing and researching
Also when I was using it, I managed to do quite a bit on a few bucks worth of OpenRouter credits. Not sure how well it keeps up in the modern world against things like Luna, but I hope it remains competitive
All these new models are such tease for us folks with 128GB of shared memory. Buying another unit now to expand to 256GB is a mortgage payment but it’s getting tempting…
https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B is an option
That’s for toy GPUs, like the 5090.
there are many tasks (increasingly more each day) where small models are more than enough
Yeah :(
Close but yet so far. Nobody has released a Q3 that fits so far. I think 95-115gb is the sweet spot
Is there a gamechanger around the corner to reduce DRAM requirements?
I can imagine someone building a device that uses NAND flash or similar tech but with a very different controller that is optimized for streaming the data out with a predetermined access pattern at very high speed.
Flash, like pretty much every solid state storage technology, can scale its output bandwidth to ridiculous levels limited pretty much only by the readout circuitry. There may be a price to pay in power consumption, though.
It’s called “HBF”, high bandwidth flash, and it’s on its way!
You could always stream from SSD storage. Especially effective if you get a cheap old-gen HEDT with lots of PCIe slots to add NVMe storage to and reasonable overall PCIe bandwidth.
That nearly certainly boots you to secs-per-tok land (as opposed to tok/s). Plausible if you are willing to wait hours to days for responses for simple testing, but not (debatably) "usable".
Demonstrated inference speed for the biggest open-weight models is indeed around 1 token per second, with weights read from SSDs.
However, this is for relatively low-end systems, with a couple of fast SSDs providing around 20 GB/s throughput (or with a few more, but connected through relatively slow Thunderbolt, for a similar total throughput).
If you use 16-lane PCIe add-on cards with 4 M.2 slots for SSDs and a total throughput of 50 to 60 GB/s, you can quadruple the previous speed in a desktop PC where you use the GPU PCIe slot for SSDs (a fast CPU, e.g. an AMD 9950X, would be alone fast enough for inference limited by SSD throughput, so a discrete GPU would not be required).
If you have a server/workstation motherboard, e.g. with 6 16-lane PCIe slots, you might gain another factor of 4 in the inference speed, so one might reach around 15 tokens per second for a very big model, but the cost would also be high, with so many SSDs, even if at that number of SSDs each SSD could be the smallest that can be found with a PCIe 5.0 interface.
n-gram per-layer embeddings[1][2] might be it.
[1] https://sebastianraschka.com/llm-architecture-gallery/per-la...
[2]: See DS 4.1-Flash and Qwen-3.8-Next.
this is to offload VRAM to DRAM (for GP comment), and makes no difference for URAM
You can definitely offload n-gram embeddings to storage; they're very sparsely used (only a few KB fetched per token) so this is quite effective. Loading to DRAM only becomes necessary if they are a bottleneck to overall performance (which might happen if you're doing very wide batches and everything else uses super fast VRAM/HBM).
I was looking at the qwen-next-flash, and the weights would fill my OEM Spark on their own, before the n-gram. I'm unclear if offloading to disk can work here, is that what you are implying is possible?!
Check out eugr’s TP=1 sparkrun recipe :)
It’s an NVFP4 quant, but it fits, and is surprisingly capable.
do you have a HF link? HF search is not uncovering it for me
(or is it somewhere else)
https://github.com/spark-arena/eugr-recipes/blob/main/recipe...
This one!
I'd recommend pointing your agent at it (after installing sparkrun), and asking it to research the absolute latest in TP=1 Flash-Next - mine grabbed particular vLLM nightlies and mods to improve performance, and it was well worth it.
I have a quirky vLLM on k8s on 2x OEM sparks setup with 9 models available to me. I'm not keen to run nightly vLLM, too many issues with it in the past. Going the qwen-next path means displacing things I use daily :/
I have a watchful eye on the diffusion ~ Jev/Kev PR
https://github.com/vllm-project/vllm/pull/57250
For what it's worth, Flash Next outperforms every other model that is available to us on the GB10 in all of my testing; though if you have two sparks then the TP=2 version is even better and easier (I don't think you'll need the nightly for that at all, just use the recipe)
I'm so tempted to buy a second one...
prices have gone up quite a bit...
I'm running embedding, reranking, and policy tuned models too, and a Jev/Kev when that's landed. Flash Next is not a substitute for those
I have OpenCode/Fireworks to access big models
looks like this is likely it
https://github.com/spark-arena/eugr-recipes
https://github.com/eugr/spark-vllm-docker
Nah, I’m streaming ngrams off NVMe on my Spark-alike right now. Works surprisingly well (except for when I accidentally bottlenecked it through my NAS)
What kind of throughput do you see on what models?
GB10 boxes have way more compute than they have memory bandwidth, which nicely fits medium sized MoE models with speculative execution (MTP, DSpark/DFlash, etc)
Qwen 3.8 Flash Next (what I'm running basically entirely now) sees 30 / 35.0 / 45 tk/s for prose, analysis and code respectively for actual use (not short context benchmarking) with Pi. Thinking blocks are ~35tk/s or so.
The GB10 having so much compute is great for prefill too, 2000-3000/s for 14k to 64k token prompts (cold cache too) in the quick benchmark I did. 3500tk/s for warm cache which is nice :)
When I accidentally streamed my ngrams over the 2.5Gb/s network, it cut all the throughput down in half basically. Especially notable for the time-to-first-token, which is what clued me in that I'd messed up somehow!
For Qwen 3.8 27B, I got it up to a consistent 20tk-25tk/s but 27B thinks so much that it was honestly too painful: Flash Next is as smart, as useful, but much faster for real agentic dev usage IMO
Laguna S 2.1 saw similar numbers to Flash Next if I remember right, but their latest updates means it doesn't quite fit a GB10 128GB anymore at full context which is a shame.
Note: these are all NVFP4 quants (usually a dynamic one where some tensor layers are left at full precision though)
I personally stopped caring as much about the tok/s as the agents are largely in the background, and so have also moved preference from MoE to dense
I want to see about fine-tuning these models a bit on the GB10 to tame that over thinking and some other behaviors (like using tools I don't use)
qwen 3.8 seems to have been trained with some `rkt` that messes with tool outputs to "save tokens"
check out the spark arena website, its the raison d'etre
interesting, peer comment seems to indicate this is a possibility as well, will have to take a deeper look
n-grams can be kept on SSD, no need to hold them in any kind of RAM (at least w/o batching)
Am I missing a joke? WTF is URAM?
unified memory, not sure if anyone uses URAM, I human hallucinated it
Are folks working in biology / cybersecurity seeing more limits in what Opus (not Fable) is allowing? This has happened quite suddenly for me and I’m stuck in the middle of a project that would have otherwise called for use of Claude.
I’ll be trying these models out and may end up switching my subscriptions if this craziness continues
glm 5.3 and deepseek flash v4.1 are quite capable in cyber. But they're still slightly behind Sol/Astra/Fable.
What works for me when I need cyber from Fable/GPT is switching the model after cyber refusal to DS flash for one turn, then switch it back to Fable/GPT. This works most of the time.
>Night 0.8x Usage, 00:00-08:00 -UTC+8
It's because offpeak electricity is cheaper?
Funnily it's perfect if you are in the Pacific Time Zone because you can use it daytime 9am to 5pm
I believe it's a function of their primary user base being in china
Same as DeepSeek, non busy time for UTC+8, maybe also cheaper electricity during night
The moat for OAI and anthropic seems to be very quickly shrinking. Chinese labs are now using RSI-like approaches and even without resorting to heavy distillation they're catching up in a couple of months vs. what would have been 6-12 months a year prior.
And as these models get better the pace of training is quickly speeding up too.
This doesn't bode particularly well for anthropic/OAI after they go public.
I don't know if we will look at OpenAI and Anthropic as moating on frontier models & selling tokens.
They are banking on the application layer and accumulated business and end user context. They have to quickly make that systems integrated value out weigh the model choice price value in the broader market
Unknown if they will be able to pull that off.
token vendors are headed to the same place mobile data vendors went, this is good for everyone but those who thought they could maintain exorbitant prices
Explain about mobile data vendors?
when mobile data first arrived, it was expensive and people bragged about their bills (token counts today)
with time, it became commoditized, people now have unlimited plans, and the money is made by the applications that sit on top (token generation is increasingly undifferentiated low-level infra)
This is not to say there has not been significant innovation in the time since, but it's a low margin business (tokens look to be headed this way)
It tops the intelligence vs cost Pareto frontier and, uniquely for a Chinese model, does well in response time too.
https://artificialanalysis.ai/models/mimo-v2-6-pro#intellige...
That's pretty fast; I think I'll try it: https://openrouter.ai/xiaomi/mimo-v2.6-flash
One concern I have is that they allegedly do not discount cached tokens: https://www.reddit.com/r/opencodeCLI/comments/1t37dz3/xiaomi...
Can anyone comment?
[dead]
I tried them, but could only test Pro none and Flash none and low, the other ones (medium/high) used way too many tokens and all requests timed out. Not sure if they have a problem with their API, or this model is really token inefficient/basically unusable.
The full-response APIs of many providers have been inadequate for a while now because their timeout intervals do not account for lots of thinking. You can use the streaming API to avoid timeouts.
The timeouts are my self-imposed limits for the test.
16 minutes per test, which is a lot for simple questions...
Sometimes they fail because they reason more than their max context window without giving an answer, that's odd too.
Can someone recommend me a local harness to explore the multimodality of this kind of models? Text in text out is mostly generically solved, but that's about what I know
Not sure what kind of tasks you have in mind, but Hermes will happily hand media to the model and show you results.
Are people using MiMo models as their daily drivers in a company setting? If so, how? I know they're available over OpenCode and directly from Xiaomi, but those are not great options. OpenCode Go straight up doesn't give any guarantees about training on your data, and Xiaomi says that they won't but it's unclear.
With some models you can find hosting companies based in the EU or US, but then you don't know how they're quantizing the models, so you're not sure about the actual output quality.
How are people actually using this? Or are people just experimenting with side projects?
I only use it for personal projects, but their european Token Plan [1] has ZDR and is hosted in Amsterdam. The DC also likely runs on 100% solar wind/power, with heat recovery systems used to heat nearby university buildings.
[1] https://platform.xiaomimimo.com?ref=UKV2FC (invite link = 10% off)
How does the value of their plan compare to paying by token via e.g. openrouter? Worth it?
And how's the performance?
And can you choose the server or does it just use your geo to determine? e.g. can I as a Canadian pick the European infra?
Where specifically do they say their European plan has ZDR? I just skimmed that page and their FAQs but can't see it (though I might have missed it).
Hmm I may be mistaken on that now, I remember it being in the user agreement. There are third party routers that explicitly offer ZDR, though API costs will be slightly more expensive.
Yep am waiting for some of my whitelisted openrouter providers to pick it up (so far none offer it but hopefully a few will in the next couple of weeks)
Fireworks is amazing for DeepSeek, Kimi and GLM.
Hope they bring MiMo for tests.
Please, think about the american companies! If we don't protect them, when they go broke (because they will, it will be the perfect triple dip) and we bail them with american taxes we will have paid twice for all the work they had already stealed (not my words, this is a microsoft anti-ai exec...)
Twice, trice or quadrix(tm) are just approximations, we have already paid with:
Anyone knows what they used to create the videos ? Is the model driving a program like Davinci Resolve / After Effects ? or is the model writing code to then generate these videos via some library.
Here's an image->html test for it using 2.6 Pro Ultraspeed, along with comparisons for grok 4.7 and Astra.
Design: https://image.non.io/78795662-8bfc-4e14-8d72-3738392aa6b3.we...
MiMo 2.6 Pro Ultraspeed (36min): https://html.non.io/annui-mimo/
Grok 4.7 (25min): https://html.non.io/Annui-grok/
Astra (19min): https://html.non.io/annui/
Overall this felt like the weakest of the three. Ultraspeed was fast as far as tokens per second goes, but it overthought quite a lot of things resulting it in having one of the longest build times. That overthinking didn't lead to better results either - note the statue with the cropped off arm. It's also the worst implementation of the dynamic lighting effect / displacement effect - the background especially has some significant distortion. Astra was the only one that seemed to understand that displacement should happen less the further something is in the distance.
Here's a vid of all 3 side by side with the source design: https://non.io/video/annui-comparison.mp4
For me it seems like this model’s overthinking should be tamed.
Perhaps instead of giving it a one shot task with vague prompt. I wonder how it can perform with much more detailed and constrained prompt (can you please elaborate more on the level of detailness and ambiguity that the prompt is and where does this model seem to overthink the most?)
Also are there any ways to tame such overthinking of models in general?
I hope that once models start becoming smart enough (I think for me it’s already there) or becoming genuinely the Sota. They then start focusing a lot more on optimizing token usage
Astra's looks the worst to me on mobile, though
That's fair - worth noting that none of them were instructed to make a mobile variant or to test the mobile size.
I used 2.6 Pro and it seems to overthink way too much, leading indeed to very slow build time.
Very capable model. I just ran it on my own LLM benchmark suite[1] and it matches Muse Spark 1.3 in pass rate but is significantly cheaper.
KillSwitch-Bench 1.0
1 - https://bench.killswitch-lang.org/I appreciate that this benchmark is different but it is in no way how most people use LLMs or promote equal grounds when benchmarking:
- capped per-task budget and time limit
- No internet access
- different harnesses mixed
How do you think other benchmarks work? Every single one is going to have a budget and time limit. There has to be a cap on those, you can't just let it spin forever and use unlimited funds.
I would argue that this benchmark is uniquely suited to how most people use LLMs because it actually tests common harnesses and it is a true coding benchmark for a language that is unseen, thus testing the LLMs actual ability to understand nuance and learn.
Internet access is restricted to ensure that over time models cannot just look up the source code of KillSwitch, which would allow them to cheat.
I didn't expect there to be a time limit and I do know benchmarks that don't have a time limit. They just penalize slower models which I think is fair.
In my experience models just don't take forever to mark tasks as done.
For my coding usage I don't set time limit so benchmarks that do so are providing me less interesting use cases.
With that said, I can understand having budget limits for expensive LLMs for those of us that don't have infinite VC money.
As for no internet access, the issue is that for most problems we do want Llms to be able to search docs, API specs, GitHub issues, etc. So not allowing that usually just favours larger LLMs that were able to memorize more data, not necessarily smarter ones when both have internet access.
[dead]
Training cost $ 3.47 million....
Staggeringly low for a frontier model.
This figure doesn't include pretraining cost. Which is probably higher.
that is just for the RL
Anyone else more excited about Chinese models than American models these days? Big thing for me is affordability.
Absolutely! Chinese models are both cheaper and more capable in many cases, compared to the American models and their makers continuously fumbling or reducing model capability with each update. Deepseek decreased costs when they released Flash 4.1 you would not see any American company do this, in reverse they would try charge you more.
OpenAI decreased prices with the 5.6 model family.
And later they further cut Sol and Terra pricing by 20% (maybe only in the API) and Luna by 80%.
In fact Luna still outperformed DeepSeek Flash 4.1 in cost per task on Artificial Analysis when I last checked.
However, Luna is slightly less intelligent. I have a feeling that it's pretty dumb and prone to hallucination unless running at xhigh or max effort, where it somehow manages to work quite well.
I did not personally test the open weight models beyond the old Qwen 3.6 27B, which produced unusably bad results for me.
The competition is great, and I hope Chinese models will continue to force leading US labs to offer models at a low price point.
That said, I don't think the Chinese labs have anything over OpenAI and Anthropic when it comes to capability or efficiency - I have no reason not to believe the US labs have even lower cost to serve the models.
OpenAI had to cut costs because of Anthropic. I also do not trust the benchmarks when it comes to models anymore. I have tried both Claude and OpenAI models and while it is true that the 5.6 series is smarter than Deepseek (at the time i tested it against 4.0) at that price it is still not worth it and sometimes randomly refuses to do tasks or stops midway etc.
Do also remember China is this far in the AI race despite all chip restrictions from America. If they were in equal standards I truly think Chinese models would have long surpassed American ones. Also would like to remind how Anthropic CEO is being hostile and blaming Chinese models with distilling meanwhile their own models claimed to be Qwen¹ and their stance against open models is negative² and they still keep blaming China for it.
1- https://news.ycombinator.com/item?id=48671252
2-https://www.anthropic.com/news/position-open-weights-models
> Also would like to remind how Anthropic CEO is being hostile and blaming Chinese models with distilling
Why wouldn't he? If there really was 25,000 accounts breaking ToS any CEO would at minimum be upset. Evidence of Claude distilling qwen would be damning but that a) makes no sense b) doesn't exist afaik.
> If there really was 25,000 accounts breaking ToS
Is this even true?
I don't trust a single word that comes out of thr people behind Anthropic/OpenAI.
They have the logs, created a report and sent a letter to Congress. Whether you believe it is entirely up to you. Given Chinese firms record on IP theft, it's entirely believable. I don't have any doubts, but I might question how they attribute it to a specific firm.
And they took multiple measures presumably to stop "distillation", such as hiding reasoning steps.
Chinese models kept improving in capability regardless, and are in some ways more impressive than Claude/ChatGPT.
So yeah, I think they are bulshitters. The can create reports and send letter to congress simply because they know if allowed to compete freely the Chinese models will eventually prevail.
Also, very rich of you to mention Chinese firms record on IP theft when Anthropic and OpenAI are companies entirely built on large scale IP theft.
> Also, very rich of you to mention Chinese firms record on IP theft when Anthropic and OpenAI are companies entirely built on large scale IP theft.
Both can be true
Not sure about that.
Given the difference in compute, it seems plausible.
However, the researchers at the US labs are surely no less talented, and they have better access to hire talent globally.
They too have to serve their models efficiently at a large scale, and with current capacity constraints this must be a top priority.
So first it’s “Chinese companies cut costs, and you’d never see American companies do that”, and then when it’s pointed out that one of the leading American labs literally just did that, it’s “yeah, but they had to because of competition”.
What do you think is motivating the Chinese labs, benevolence?
> If they were in equal standards I truly think Chinese models would have long surpassed American ones.
Limitations often lead to creativity to overcome them. The Chinese AI labs have had to focus much more on efficiency so they got good at it. Meanwhile breaking new ground is often harder than replicating it. So even if they had matching compute it's not a given they'd be better.
> I did not personally test the open weight models beyond the old Qwen 3.6 27B, which produced unusably bad results for me.
So you don't have much perspective on things, it seems. Let me introduce you to the GLM 5.2 and then 5.3/5.3 flash series of... "oh, wow, I should have bought some RTX PRO 6000's while they were 'cheap'" stage of progression.
As someone carrying multiple max subscriptions to both claude and codex - primary workhorse is glm 5.3 flash running on rented GPUs for less than a latte/hr.
I also found qwen 3.6 27B nearly useless for my own needs. DS4 flash 0731 and then 4.1 have been nearly as eye opening as glm 5.3 flash, but have their own warts.
Try DS4.1 Flash. It's another eye-opener. If you run it in Claude Code, it's easy to forget you're not actually talking to a high-end Opus model.
Have you tried Qwen 3.8 Flash Next? You can run it on one spark with reasonable context sizes at about 30 tps, and it's as good as DS Flash 0731. Maybe even a tie with GLM 5.3, though like everything it depends on the use case.
Why use GLM 5.3 Flash when you also have access to Astra, Sol, Fable?
Or I guess the other way around, if GLM 5.3 Flash is so good, why Claude and Codex?
Increasingly stingy usage limits on the subscriptions, regardless of tier.
I’m having the same issue. Hold max subscriptions on both frontier labs but I’ve been forced to use open source models because token limits are not what they used to be. So I end up using Astra and Fable for reviewing, and open source models for implementing.
All american models refuse to help me design nuclear weapons in Nuclear Design Bureau or to work on my cybersecurity projects.
[dead]
> Deepseek decreased costs when they released Flash 4.1 you would not see any American company do this, in reverse they would try charge you more.
OpenAI reduced prices and Anthropic increased weekly usage limits.
> OpenAI reduced prices and Anthropic increased weekly usage limits.
As a Max x20 and Pro x20 subscriber, can tell you that it doesn't matter since they continually move the baseline of token use So in practice you feel that you're continually getting less from your subscription.
While it never happened to me in the past, i reached my weekly limit within 3 days using Opus 5. And the Open AI weekly limit essentially is a Claude Max x20 5-hour limit. Not even talking about the baseline in intelligence : on release day Astra was so good that it lead me to move to Pro x20. Now it's dumb af and token use is insane.
Deepseek 4.1 Flash has been a lifeboat for me, finally able to work without being constrained/distracted by limits and with what is in my view even better intelligence than Opus 5 for a fraction of the costs. DS is not messing up my brain with load-bearing pseudo jargon in every sentence. It respects coding guidelines, and completes even the most complex tasks most of the time in one shot.
DS 4.1 had been able to add complex features to my repo without breaking a sweat (330k lines of F# + 4M circa lines of an Angular frontend). Writes very idiomatic F# and respects our guidelines and style perfectly. Just completed an extensive UI/UX research and implementation work.
I am ditching both x20 subs and will only keep a Pro x5 because wife does a lot of design work and needs solid image generation capabilities.
Absolutely! DeepSeek-V4-Flash-0731 has become my daily driver. It's pretty amazing what it can do for what it costs at deepinfra.com (I don't use deepseek as a provider since they train on your data [at least their honest about it]). GLM-5.1 was my daily driver before that and Kimi K2.5 before that.
Are you finding DS better then kimi k3 and glm-5.3? Do you mind sharing your primary use case?
My primary use is AI coding agent. Its vastly cheaper than Kimi K3 and I haven't found a scenario where I really need Kimi K3 versus smaller models. GLM-5.3 Flash is good but there is series of bugs in the vllm middleware that prevent GLM models from getting all of their reasoning content returned to them that impairs inference quality. A lot of inference providers use vllm which makes it hard to find a good provider for GLM. I've been using friendli.ai but using GLM-5.3 Flash from them is more expensive then using DS V4 Flash from deepinfra.com simply because deepinfra.com is so cheap. The DS V4 Flash cost at together.ai is similar to the GLM-5.3 Flash from friendli.ai or at least that's what I found in my benchmarks a week ago: https://www.linkedin.com/posts/joshheitzman_i-ran-a-fuller-r...
I've used Kimi K3 for a few months as my main model and DeepSeek 4.1 is as fast and about 10x cheaper.
I just had like four big sessions going today, paid about $8 in tokens. I see no reason to pay more, this is more than I need for intelligence.
4.1 consistently surprises me in capability for the price. And I don't think I'm the only one. It's been dominating the leaderboard at OpenRouter, and I just got an email today from Fireworks saying they were _raising_ the price by about 30%. I'll probably switch, because their infra doesn't support being the highest-cost, but it's still telling.
I tried it a few times and liked the speed, but often found it ended up looping, i.e. repeating the same token sequence (e.g. the same sequence of 5 paragraphs) over and over again until it hit the max output limit. This doesn't end up happening every session, but does every now and then.
My impression of DSv4.1-flash was very positive aside from this. But that was enough for me to stick with GLM-5.3(-flash), which both gave me consistently great results
I was using a vibe coded bare bones harness. I was wondering if this was normal from DSv4.1-flash, or if its my harnesses fault.
I've had that looping issue with open models too. But never 4.1. I wonder if it's a model + harness combo? But yeah, one loop issue and I'm done with a model forever.
Harness. Especially if a tool call error doesn't say what to do next and the model is not RL'd with that tool, a retry storm is common.
So if you use MCP a lot, simplify the params, be more lenient on validation and rework the errors.
It is quite good with shell.
Yeah, that's what I'd been leaning towards. No mcp, but I'll see if I can reproduce and debug it, since other people don't seem to have that problem as badly as I've experienced it (and the idea of having a nasty bug like that bothers me).
No mcp support. I'll try copying deepseek harness's basic tool call formats as a starting point.
How does it compare to 4.1 flash? Curious why folks don’t use the more “modern” one.
4.1 flash is very fast and capable. Token efficiency is not great so it fill up context window much faster compared to similarly capable models.
glm 5.3 flash is a tad slower but a bit more capable and way more token efficient.
Source: self hosted tested on rented GB200 node at 8bit.
Wow, I'm surprised you are saying GLM 5.3 Flash is more capable. Isn't is like half the price of 4.1 Flash?
I haven't tried 4.1 flash as I'm assuming its a preview. I did not get good results from the preview version of 4.0 flash (i.e. the one that did not include the month and date of release in its name).
4.1 Flash is a horse of a very different color. It cooks. IMHO it's probably a preview of DS5, rather than a true DS4-series model.
Affordability is derivative of control which is really what I care about.
I'm just not going to build long term infra that depends on something that another person can and will - objectively based on experience - take away from me at some unknown point in the future.
The biggest benefit of open models is they keep all the other players honest. The extent to which they feel they can dictate terms is directly set by the threshold where they feel people will take the trade to run open models instead.
American models are on the frontier of capability. Chinese models are on the frontier of efficiency. The problem for American labs is that Chinese models are more than capable enough for the vast majority of applications that people care about at this point, so efficiency is more interesting for people.
it's really weird to me at the moment because both OpenAI and Anthropic seem to be competing in an extreme benchmaxxing contest on super intelligence that actually nobody cares about. I haven't really cared about model intelligence since about Opus 4.8. It is by far not my biggest problem. I don't need to replace or support Einstein in my production workflow. I just need basic intelligence that can equal a routine office worker - safely and reliably. What they doing - chasing super-intelligence but dramatically escalating risk - is actively what I don't need.
I really think they have drunk too much of their own kool aid and become completely detached from what the market wants.
I couldn’t tell you what western model I was last excited about. Probably Glimmer.
Jev seems to have people excited, I'm more excited for the Kevs
The cost issue is obviously of prime importance, but I'd also argue that the transparency of the innovations creates a tremendous cross-pollination, and not only within the Chinese communities but in the US/Europe as well. How many of us are learning the practical aspects of actually running and building AI based primarily on open models? As an example - how far would the work of vLLM or SGLang or even NVIDIA itself (all random examples) be without these models and the challenges they pose?
Yes, an expensive American LLM has zero capabilities as far as I’m concerned because I’m never going to pay for it.
I certainly am.
Months ago I switched entirely to use Chinese model. Mostly DeepSeek and MiMo, although I recently started to play with GLM as well.
The models are excellent and in many ways I prefer them to Claude.
I see no difference in terms of capability, but the fact that they are cheap frees me to experiment.
I have a contrarian opinion that China passing America in Ai is the Sputnik moment we need to leave the hubris behind and get our mojo back
debatable if a turn around is possible before '29
The analogy makes little sense. The USA was not in front of the USSR and Sputnik merely showed that. It is at this point that the Americans woke up, put a lot of effort and finally were able to surpass the Soviets during the Apollo missions.
China was never ahead of the USA in AI. So perhaps a more proper analogy is the Moon landing. In real history the side that lost the race never got its mojo back...
I'm looking forward, towards the future, when I use "passing ... we need", need being key here as it implies something we don't yet have
I expect this to happen within 12-18 months, the differentiation has shrunk, many models are now sufficiently capable for most tasks
I understood that.
I was simply saying that when (not if) Chinese AI models will pass Americans, it will probably be game over and Americans will never catch up, let alone become leaders again.
Check the names of the researchers in the DeepSeek's latest paper. Full of Chinese names. Check the list of names in Google's paper. A very similar view. Anecdotal, but quite thought-provoking...
ah, ok, to add to this, China is persuading nobel laureates to "switch sides" (I imagine the current state of America had a part in pushing him away)
https://www.nytimes.com/2026/07/09/science/nobel-winning-us-...
I did try to use Chinese open models, but for my production work they simply couldn't cope at all; both GLM 5.3 and Deepseek v4 went into infinite loop and wasted my tokens until my OpenRouter wallet reached 0; good thing I didn't enable the auto topup. US models, by contrast, breezed past them. Even for simpler tasks, Chinese models took long time to complete, and I needed to supervise closely. The price , in the end, didn't come cheap, mainly because too much time wasted on thinking.
So maybe one day Chinese models will squeeze out the American ones, but today is not that day.
So no, I am not excited about Chinese models ( just because its open weight and not American).
Not too be "that guy" (e.g. "you're using it wrong"), I just want to humbly ask — have you tried blacklisting "underperformers" in OpenRouter config?
Here on HN was a post few days ago titled like "so you want to use openrouter", there was a benchmark in capabilities between providers which showed some aggressively quantize and basically break models and tool calling.
I am in no way a professional power user, but I frequently suffered from "call fails" (e.g. unclosed tags, broken agent loop, broken thinking blocks), so I had to babysit agent on it's loop. After I blacklisted like 20 providers (I think most broken were Nebius and DigitalOcean) these issues completely went away. I had several agents work on my small tasks for 18+ hours with no issues.
Try Fireworks instead. The experience is dramatically different because the service is much more reliable.
Yep, I'm trending in that direction, and I'm someone with Claude stickers all over my laptop. My main app dev work is still going to Claude, but everything else is going to China even at API rates now.
One simple task: I needed an LLM to go through and clean up a few thousand page descriptions and titles in my personal search engine index, where the human web page authors had put in no effort sigh. I did a shoot out between Claude, Luna, GLM 5.3 Flash and Deepseek. Despite the high cost, Claude's descriptions were terrible, and even Opus warned me that the descriptions coming back from Haiku were "generalized, not accurate". I expected I would choose Luna because of price, and occasionally it did have wonderful descriptions (one captured emotion in a way no other model did). But in the end, the GLM 5.3 Flash descriptions were the easiest to read, they flow well while also being accurate & including necessary keywords, and being highly affordable. So it won out. It's a task that is nowhere near frontier, but a task where somehow China is better than frontier.
API rates still aren’t quite competitive with the OpenAI x20 accounts, but they are definitely getting close with deepseek 4.1 flash. I spent a few days with only 4.1 and was very impressed.
In my very limited first test with MiMo 2.6 Flash it was much cheaper than DeepSeek 4.1 flash for a given broad task. It's at least worth a try.
I would agree, but I'm not sure I trust the "no training" policy. For DS 4.1 Flash I was able to use a US ZDR provider.
No, because I'd rather not support our economic and military rivals.
I'm Canadian so this sentiment has little value in 2026 unfortunately.
As much as the US has been easy to hate lately, I don't hesitate to say Xi Jinping as the most powerful man on Earth would be much, much worse.
I agree but I like to take the time to remind the Americans how far they’ve fallen.
He is the most powerful man on earth. He’s just smart enough to let the US get as fucked as possible before making a move.
When was the last time China bombed another country?
Also, frankly, as a fellow Canadian it's pretty clear that the biggest "rival" the US has right now is itself. Just passed out in the corner puking on itself shouting about all the foreigners who won't talk to it.
I'm from Europe and I hate America way more than China now. Used to be about equal but then Trump started extorting Ukraine, threatening their own allies and sending billions to Israel to help with a genocide. I think that exposed America for what it really is.
China is enabling russia way more than trump, China doesn't care too much about 'morals' either. Chinese companies have been quite important in the construction sector of the WB settlements. Even though I'm not a great fan of Trump I don't see a reason at all to prefer the chinese.
As a New Zealander, I would agree - no reason to prefer the Chinese. But Trump's America is not an attractive option either and there's no reason to prefer it. And given the choice between two ugly options, the rational choice is the cheaper one, surely.
>China is enabling russia way more than trump, China doesn't care too much about 'morals' either
The difference is China has a good reason to. China doesn't look appealing because they're more moral than anyone else, but what they have going for them is that they still behave like a rational actor. At least their behavior is intelligible in terms of their own interests. The world can deal with a long term selfish superpower but not an unhinged one
I don't think there's a person in China that has as much of a seething hatred for America's 'allies' in Europe as J.D. Vance or half of the American techbro commentariat does
buy inference from european companies running open chinese (or that one from google) models
China is a somewhat neutral player, supplying both Russians and Ukrainians. Their attitude and action is way less one-sided than Trump's; especially in the first year of his latest presidency.
With trump his actions being one sided you mean one sided towards ukraine? They still get lots of Intel from Americans and Americans hardly but anything from Russia. But you're right that china supplies both I wouldn't exactly call that neutral as much as just in their self interest.
How is China supplying Ukraine?
Drone components and electric batteries
we don't really know if Ukrainians buy it on aliexpress or through European supply chains.
Fiber optic cable for one
[flagged]
Because at present the pedophile US president is making it his mission to molest my country. China, for all its faults (including espionage, which the US is also guilty of) is mostly focused on conducting trade.
Half of Canada now uses the word 'enemy' when asked for an adjective to describe America or China. We're equivalent in their eyes now because we elected Trump a second time and all that he has said and done in 2.0
It's closer to a cousin you used to be close with despite some moral failings, but who has now has a substance abuse problem and is lashing out at family and friends.
Not an enemy, just a danger.
I'm relaying a poll of Canadians, their word choice, not mine
"plurality" would have been accurate over "half" on my part
https://www.commondreams.org/news/canadians-us-enemy-poll
Eh, that's pretty misleading: in that poll, Canadians weren't asked for an adjective to describe America, they were given a list of three options “Ally/Neutral/Enemy”.
Canadians warming up to China makes me think of Germany becoming increasingly reliant on Russia in the 2010s.
Murica just has a MAGA problem. We can still be friends if and when you sort that out. Us Canadians like most of you quite a lot.
Yes, someone can still blow up a pipe and they look the other way. On the other hand, you can also draw parallels to themselves becoming increasingly reliant on US vs UK in the past.
Does it count as supporting a rival if your an American using an American inference provider self-hosting an open weight model from a Chinese lab?
Agreed, and also because I support freedom of speech!
Neither the US nor the Chinese companies are on your side then. They both censor, just different topics.
But at least I can run Chinese models locally, and strip a lot of that censorship/refusal.
As long as that speech doesn't come from CNN or criticise Charlie Kirk, Israel or Trump? I'm sceptical about how much the US really values free speech
Yep. If OpenAI and Anthropic get the regulatory capture to block Chinese models I’ll go on an AI strike.
is there a good metric of model degredation over time?
Im a bit lazy and only use the free models different companies host and the biggest difference i see is that some models (Gemini, OpenAI) get progressively stupid in long chats. You end up having to start a new session every oncr in a while. Or they get really hung up on a theme and cant shift to a new topic.
By contrast, Ive been impressed with Qwen. I have some chats on research and code architecture that have stretched for weeks without any noteable change in quality (though occassionally it seems to "rush" to an answer)
Im just looking at all the listed benchmarks and im unsue which i should be looking at
3.5 mil cost for a opus level model? Even if excluding salaries that seems very cheap
That's the RL.
I really liked MiMo 2.5, it was really affordable and actually had vision, unlike DeepSeek. (DeepSeek has only recently added it)
Just tried 2.6 flash on a really niche topic I specialise in and it has done a really good job. They’ve definitely polluted their training data with claudeslop, but looking past the slop there is a decent model.
Can we afford to look past it? If/when claudeslop starts infecting every new model to such an extent, that model will produce its own slop, infecting new models... At what point do we lose all reliable methods for establishing "truth"? This is epistemic collapse waiting to happen. I honestly thought it would take longer... holding out for a coherent shared reality in 2030 seems optimistic.
how do you recognize "claudeslop"?
It's an honest, load-bearing, simple thing.-
that's belt and suspenders too
a smoking gun
... and a caveat worth flagging.-
You're right to push back. That's on me. There is one thing I must flag which will move the needle. My honest take: It's all about the shape of your priors. That's the lever, and that's not nothing. It's worth your attention before you land your next remark. Here's why that matters: It re-contextualizes everything.
Any idea on how they get the pricing so efficient? Their artificialanalysis graph has them on par with GLM 5.3 but at less than 10% of the price despite being larger than GLM 5.3.
It sounds like they are pricing very well because OpenAI and Anthropic have been scamming us for years. Once the infrastructure is in place, electricity cost is the only concern. China supports businesses and gives them a lot of incentives to lower their costs. And who knows what's being provided to them without anyone knowing. All in the name of winning the race.
If that is true, why are the open model inference providers on OpenRouter (and on their own website) so expensive then? It must be the hardware. I've checked and a lot of it it just the HBM, with the NVidia tax being a smaller but also large factor.
Nvidia's top AI chip, Rubin, sells in 72-GPU racks for about $3.5–7.8M. A rack running Xiaomi's MiMo V2.6 Pro could generate roughly 150–300B tokens a day, worth about $130–260k at Xiaomi's API price. That's a payback of a few weeks in theory.
So yes, we are getting scammed by American SOTA.
Interesting. Then my theory is that the free market inference providers have low utilisation (e.g. 20% avg; 100% at peak), slowing their amortization rate. The OpenAI/Anthopic rates must then have been based on a time when there utilisation was peaky and mostly idle (e.g. 10/20%). That would maybe explain the 10-20x cost difference between coding plans and API.
Funny that all but one video has audio, the house 3D model one, where you can hear (what I assume are) Xiaomi's engineers talking about who knows what.
This looks great in terms of cost and capabilities, truly pushing the frontier forward in terms of open weight light weight models.
As I read this I am getting API overloaded errors from Claude. I will switch to something else soon. I hate Claude.
Conspicuous that there’s no reference to GLM 5.3/Flash in the reported benchmarks. Just Deepseek and Kimi.
I'm thinking if this is a more fair comparison among other models, like ChatGPT, Claude and Deepseek
Is there a way to test this in a browser interface? I'm not going to run any executables from any of the frontier labs on my machine, be that Xiaomi or Google or Anthropic.
As for the stats that everyone wants:
MiMo-V2.6-Flash-310B-A15B roughly GPT-5.6 Luna / Claude 4.9 according to benchmarks MiMo-V2.6-Pro-1.02T-A42B roughly GPT-5.6 Sol / Opus 5 according to benchmarks
Perhaps with IQ2 flash will run on 128G M5?
Anyone know what game engine its using to make the 3d game?
This is a big week. Probably getting next OpenAI and Anthro models, Grok 4.7, Mimo, etc. These open source model releases are why I can't take the "slow down" crowd seriously. I pitted older Mimo, qwen, step, gpt-oss, and other models against each other playing games like Werewolf and Sketch.io-like games where I let them talk shit while they played against each other. Mimo was by far pareto frontier of game-playing for the models that were <$0.15/m input tokens on OpenRouter. Qwen was pareto frontier in the shit talking game though. Qwen's hilarious. https://www.tiktok.com/@clankerfights/video/7642862917582425...
Leaning into what it cost to train is hilarious and an obvious shot at US frontier labs spending tens to hundreds of millions or more to train their models.
Granted this is an awesome release and I loved watching the livestreamed RL dashboard, I found this message on the dashboard (https://mimo.xiaomi.com/rl/) quite funny:
> we also removed the cyber dataset from the upcoming pro run, since we observed some bad patterns in the rollout logs.
And this in the model card (emphasis mine):
> Aligned RL: Cold start from self-correction — the model reflects on and rewrites its own misaligned turns into grounded next steps. Throughout RL, environment hardening, adversarial screening, and verifier cross-checks keep the loop honest against reward hacking.
Environment hardening during the RL runs. Uh oh, did someone start making a few too many paperclips?
Finally a lab that doesn't cheat on the charts
I don't overly trust benchmarks. Benchmarks can reflect a model's capabilities, but many models specifically adapt—or even overfit—to score high on benchmarks. DeepSeek V4.0 Pro is a good example: it achieved decent scores, but performed poorly on actual cybersecurity-related tasks, even worse than DeepSeek V4.0 Flash.
Stupid question: in the benchmark diagrams I'm assuming the values are percentiles, so what does 100% represent?
> This marks a key step in our exploration of the RSI path: scaling RL compute on verifiable, complex tasks, so the model can continuously expand its capability frontier through exploration and feedback.
Gotta love a capable open model. BUT, how can they just casually throw in that they're actively exploring RSI as if it's just another technique? Is this not alarming at all?
This model & pricing fits LeiJun's visio for xiaomi: The costco wholesale of tech companies.
Experiences so far with the non-flash model: a bit of an overthinker/overplanner. Makes bad/messy architecture choices with an open-ended prompt. But actually very competent at intricate code level things, and pretty good at finding corner cases, including in code that other more advanced models had looked at and not noticed problems with.
I deliberately left things wide open and ambiguous to see what it would do. I think with better upfront planning this would be excellent value.
it is pretty amazing to see a young & talented woman is the tech lead of this frontier model.
it is reasonable to question why America doesn't have such environment.
How could the flash model beat the pro on the Cyber benchmark?
As far as I can tell, if you look at the open training dashboard it's because the flash was trained on some cyber while the pro wasn't. 4% of training on the flash was cyber and 0% of the training on the pro was cyber.
Very informative page vs the recent grok release
does anyone know what unnamed model is on paretto frontier picture right between MiMo 2.5 and 2.6?
so weird to acknowledge someone being on the front edge, but not name it
Pretty sure that's Luna xhigh.
it sounds greate. I still have 50% of my quota this month, so I'll continue renewing to give it a try.
These benchmark are useless as they don't say whether they were done before or after Fable and Astra got nerfed.
That's not chinese models' fault. Fable and Astra deserve to be punished for their scammy bait-and-switch.
They mixed up DeepSeek 4.1 Flash with something else on this page, possibly DeepSeek 4.1 Flash means Gemini 3.8 Flash.
For simple edits it produces huge reasoning traces. Kind of disappointing. Super long repetitive reasoning always sits wrong with me. It feels like a way for the labs to brute-force higher benchmark scores but not actually like a smarter model. Disappointing, Mimo2.5 was such a nice model.
mimo is imporving as well, great work!
[flagged]
[flagged]
[dead]
[flagged]
[flagged]
[flagged]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[flagged]
It's funny, I have the exact opposite reaction. This is probably misguided on my part, but Xiaomi is one of the very few major tech companies that I don't have an immediate strong negative reaction to. Everything I've bought from them, from robot vacuum to mobile phone, has been reasonably well designed, didn't break, and was priced fairly. I also think their car looks badass.
I'm sure they're doing all kinds of terrible things, like all major companies. I just can't help but like them. Also, this model looks great, and I'll give their subscription a shot next month.
I must second this.
I'm in Europe, and here the options for home appliances are usually German (e.g. Philips), Balkan (e.g. Gorenje), or Xiaomi. Xiaomi is the best by far, and it's honestly not even close.
Their home appliances are so rock-solid that they actually still surprise me. For example: I've gone from having to replace electric water kettles every six months to buying one from Xiaomi and never replacing it. (Nigh on three years now.)
I really have a very positive impression of them.
My only gripe with them is how they locked down their m365 scooters, made repairing it a pain.
It's a big company, like Microsoft or Google. Some of their products are good and some are bad.
ironic to this thread, I have less bloatware and ads since I switched from Verzion to Pixel on Fi (many years ago)
Curious if Verizon / ATT still force apps on your phone, eg. NFL and Amazon apps, Fi service is subpar
I still have a xiaomi mi 11 lite, my wife has a 15t. The cameras are the best for the price. The way they chove ads down your throat at every opportunity should be illegal though
Which model did you have?
I ask because my wife has the 15T and the camera is better than my iPhone 17 Pro. And while toying around with it I didn't notice any bloat.
Plus hers support native split screen which I kinda need to multitask on the go.
I'm so pissed at how bad Siri is compared to her android phone that I'm thinking about selling the iPhone to get a Huawei Pura Ultra.
ah, would you look at that. I was wondering why mimo 2.5 became "dumber" the last weeks. I was speculating they are probably about to release a new version of the model. because the model really acted out a lot. especially the last two weeks. dont know, was just a feeling, highly speculative.
but now I got my "proof".
I guess that would only be possible if your provider was Xiaomi itself?
yes. I use opencode and opencode uses Xiaomi as a provider.
after trying MiMo 2.6 Flash: unfortunately it ends up in infinite loops.
The Xiaomi always somehow lacks the quality of competitors. mobile phones are lower quality or weird. the login reset mechanism sucks. If I can't even login, how good this system is supposed to be.
They are excellent in marketing, I guess that is something.
In the chart they use "Pareto Line", which I think is wrong. Pareto is 20% effort leading to 80% results. Which could be interpreted as models costing 20% having 80% of peak intelligence, but that’s not what it looks like to me.
It looks like the "Frontier Line" to me, which is also often misinterpreted. frontier does not mean the best models. It means all models that are not strictly dominated, meaning in most cases: Not same price or cheaper and more intelligent.
I personally would like the word frontier to be used with more criterias: Open Weights, per use-case, etc etc. This would make model selection easier, but I understand it’s not an easy thing to do.
There are two (or more) concepts named after the same person:
- Pareto efficiency/Pareto curves: Basically the convex hull of points along the edge of a graph, indicating the best tradeoff between the axes. This is what the post is talking about.
- Pareto principle: this is the 80/20 rule you're talking about
This is the Pareto Front [1], rather than the Pareto principle. It's the idea that anything that's more intelligent is more expensive and anything that's less expensive is less intelligent.
[1]: https://en.wikipedia.org/wiki/Pareto_front
No, Pareto refers to Pareto efficiency https://en.wikipedia.org/wiki/Pareto_efficiency
What you call "frontier line" is also called "Pareto frontier" https://en.wikipedia.org/wiki/Pareto_front
Your description of it is basically correct though
"Pareto" is many things, but here it does indeed refer to the frontier: https://en.wikipedia.org/wiki/Pareto_front
Thank you guys. I learned something new.