Like the weight space stuff, I think there's a small misdirection in how this is presented: Phrases like "write directly in machine code" seem to imply that ordinary LLMs intuitively understand machine code as just another language, like English or JavaScript, that they read from context and write directly as tokens, and that this intuitive understanding could mean increased efficiency or capabilities if you ask the LLM to just use the bytes rather than asking it to use the tools that human reverse engineers use.
If that's what you assumed, the DeepSeek reasoning trace shatters that illusion. It's just disassembling by hand like what a human reverse engineer would do if you (sadistically?) denied them a disassembler. Yeah, the human reverse engineer would need an excellent memory of the x86_64 instruction reference, but we already know LLMs can recall a breadth of information. What we don't know, that this article also provides no evidence for, is if there will be any benefit in using the LLM for this over objdump.
The article seems to intentionally conflate disassembly (computationally cheap) with decompilation (can be computationally expensive) to imply that there's some benefit to asking the LLM to work with machine code (seems impressive to the RE layman) rather than working in assembly (something you probably already assumed LLMs can do,) but the DeepSeek trace has a disassembly in it, and it's hard to imagine that inference will ever be cheaper or more reliable per-token than objdump is at producing the same output.
I’m seeing the conflation between disassemble and decompile in other posts here too. I don’t know why you’d waste time and money using an LLM to disassemble an executable when it’s been trivial to do that with existing (free) tools for years!
I've found the decompiled code from LLM's to be much more similar to human-written code (and therefore easier for me to understand) than standard Ghidra/other decompiler output. Plus LLM's for this kind of work are pretty cheap.
The parent comment was making the point that disassembling and decompiling are not the same. There is no room for interpretation on disassembling, it's purely mechanical.
There's a little room for interpretation in disassembling. You can choose label names, and in instruction sets like amd64, you have to guess where the instructions begin. Also ideally you won't disassemble ARM literal pools as instructions, even though it's possible to branch, call, or return into them by computing their addresses at runtime. In binary formats that don't separate code from .rodata the problem can get arbitrarily hairy.
> in instruction sets like amd64, you have to guess where the instructions begin
Can't you just trace through all reachable instructions starting from a complete binary's entry point(s), as long as the code doesn't do unusual things like jumping into the middle of instructions, i.e. intentionally exploiting variable-length instruction ambiguity?
Not necessarily, because there is often code that is only accessible through function pointers; for example, C++ virtual methods. (In theory you can compute function pointers dynamically at runtime, too, which is less common.)
In https://canonical.org/~kragen/sw/dev3/qfitzah.s a lot of the function calls are conditionally compiled by the macro `do` (search for `.macro do`) to index into a table of function pointers pointed to by EBP, because that makes the call instructions 3 bytes instead of 5 bytes. Here I've disassembled one of the functions from the parser code from an unstripped version that has debugging symbols:
The actual source code to that subroutine looks like this:
proc read_term
do read_factor
jnz 1f # if it failed, skip ahead
push %eax # save returned term
do read_term # recursive call for tail of term
push %eax
do skip_whitespace
pop %ecx
pop %eax
do cons # XXX tail call
ret
1: setnil %eax # return nil if no factor found
ret
I didn't do this to make it harder to disassemble, but it does have that drawback.
For what it's worth, Claude Opus 5.5 had no trouble disassembling it and making a list of its bugs, including six I already knew about and one I didn't. It commented:
> Reverse-engineering a program this small is satisfying in a way bigger programs rarely are, because you can hold the whole thing in your head. Working out the scheme of keeping type tags in the low pointer bits from nothing but `test al,3` and `test cl,2` scattered through the code was the best part. Watching `(D x) (x x)` print 4 MB correctly, and then segfault on the next query, was a close second.
In 32-bit ARM code you have the additional problem that you have to know whether the T bit is intended to be set when the CPU is decoding a particular subroutine. The low bit of the branch destination tells you — which is fine if it's statically reachable, but you may have to guess if it's only dynamically reachable.
Only if you do "straight line" disassembly. On the other hand, code/data separation is extremely intensive, especially when indirect branches are involved, and approaches a decompiler in computational effort.
IIRC even disassembly is undecidable. The hardness indeed comes from jumps, you don’t know where instructions start (they don’t all have the same number of bytes).
Edit: I think the problem I have in mind which is undecidable is whether a byte is (the start of) an instruction.
Ooh I'm not sure that I would buy into that! I've been experimenting with giving small binaries without strings or other direct clues, and asking them "So tell me what you think this does" in a bare container without disassembly tools.
So I'm actually fairly confident that the ability does come straight from the ability to read the bytes into context (usually as hex bash/perl/python3 is almost always available), and Then to directly interpret what the binary does from the code. And yes, when I do use open models, I can see from their reasoning traces that they do "hand-translate" the hex into assembly, but I kind of suspect that the translation stage into assembly may just be a narration, and the "understanding" of what is going on is actually a pure latent skill. That's just my guess, though.
The reason that I think this is because I've been using minimal implementations of inference engines. And frontier closed-weight models and open-weight models are able to observe a binary with no strings and the weights not attached, and they can infer the exact architecture and model (eg qwen3 0.6b, qwen3.5-0.8b, whatever) from the fixed shape of the inference kernels.
So in effect, over the course of 30 seconds, they go from looking at 500 instructions of x86 code and reply, “I know what this is. It's an inference engine for Qwen 3 0.6B!”
And I'm fairly sure that 30 seconds or not, it's a skill that no single human has.
Assembly code is almost just machine code, which itself is nothing magical. Human cannot work with machine code because of the logical complexity, not because it's hex.
Matching decompilation is very tricky, but the decompiled output that ghidra spits out is good enough for many cases and is not significantly more computationally than getting a disassembly.
I've suspected for years that AI companies have been using commercial software to train their models including games, audio plugings, DAWs, etc. It would not surprise me one bit if we find out they used torrented pirated apps like they did with books and music.
There's certainly eg open source audio dev material out there but not nearly as much as web dev. How else do you explain all the avalanche of vibe coded audio plugins and DAWs appearing this year?
And (from a training material point of view) it makes perfect sense.
I would assume that to a LLM, a binary executable is not much worse than actual source code. And it's much denser, just missing the semantics of human-written identifiers. It's the code's pure logic, condensed into machine language.
And there's tons of it, many vendors supply it "freely" (you don't need license keys to train on a binary) It's often battle-tested code that you wouldn't get hold of otherwise, and compared to ingested source code its unlikely that Codex regurgitates a recognizable part of a Windows 2000 DLL into your C++ source code.
That's not quite right - source expresses the idea relative to a language syntax and grammar, whereas machine code (or assembler) implements the idea against a specific target machine, for a specific set of circumstances.
So as an example, there are a bunch of different ways a for or while loop can be implemented in compiled code, but there are also situations where two quite different source constructs can result in the same machine code sequence.
Machine code certainly won't teach an LLM much about how to write good source code.
Do you think a programmer that knows the low level assembly that will be generated for any source, and necessarily the reverse possibilities, would be a better or worse programmer?
Compressed doesn't sound quite right. Source code contains abstractions which are often then expanded out into many variants in an executable (most obviously by function inlining, but at a smaller scope also by things like loop unrolling etc)
No, it goes the other way. Source code is a lossy compression of the billions of machine operations it can be compiled into. Abstraction is a way of compressing, both in the mechanical LoC sense, as well as in comprehension.
Is plate of spaghetti a compressed version of a recipe?
Executables are often larger than their source code.
A toolchain that creates repeatable binary builds is difficult to create. And a nonrepeatable build is potentially nondeterministic, in the sense that uncontrolled or hidden inputs impact the output.
This is very analogous to vector vs raster graphics. In a way, the vector format is strictly more powerful than the raster format as it can be rendered smoothly at any size. Thus the raster graphic, despite being larger in size, is a lossy distillation of the vector graphic.
Also it _is_ possible to gzip a file and get a larger file, there’s an inflection point based on the distribution of symbols in the source where the same compression algorithm produces a larger file than the original
A raster is not always larger than a vector; if you try to precisely vectorize an image of the real world (modeling each pixel), the vector version will be larger
Let me make a more rigorous claim: For any image, there is always a vector encoding that is at least as small as the smallest possible raster encoding. The converse is not true.
>Executables are often larger than their source code.
Not because they contain more information. Just stuff like inlining or different expression, plus image and other similar assets that we can ignore for this.
The term I would use is “projection” rather than compression.
Code belongs to the domain of source code. Compilation is a projection of that code into the domain of executables, and that involves enough types of transformation and mapping that it is a one-way operation.
From an information standpoint, some information is added (e.g. optimizations that encode knowledge about efficient bytecode) and some information is lost (e.g. the use of types that adapt to the native word size).
My hunch is that one of the handicaps Google is dealing with (vs. OpenAI and Anthropics) is that Google is not training on pirated media. They are using YouTube, Google book project scans, and purchased copies of media.
YouTube has a massive system in place to find and block pirated material. I can't remember the last time I saw a simple video rip on YouTube.
Nowadays you're getting gray area stuff like new AI generated stories based in the Harry Potter universe, which is probably copyright violation but it's not actually pirated video, so Content ID doesn't spot it.
Nowadays you're getting gray area stuff like new AI generated stories based in the Harry Potter universe
Fanfiction has always existed, albeit in written form, but copyright takedowns are almost unheard of. The gray area is definitely getting bigger, and AI is effectively showing everyone that everything is a derivative work to some extent. I suspect we'll see a massive shift in the general sentiment around IP in the near future.
Its the doctor google problem, some people take any random page with health advise as authentic and authorative. My guess Google and its training also has a similar problem, how can they tell which sources are correct and which are the opposite. Of course there are some easy wins, such as checking the domain but considering that the majority of the internet is filled with random pages of stuff its difficult to judge those.
I suspect that Anthropic, being public, and under scrutiny, will get sued into compliance from all over the world. It's not a foregone conclusion that the data they trained on remains fair game over the long term.
At this point they'll get nothing more than a slap on the wrist. No judge wants to be the one to wipe hundreds of billions of dollars off the stock market.
I agree 100% with sentiment. It is inconceivable to me that models were not trained, either directly, or indirectly through other aggregates, on code and binaries and, of course, media, fetched from torrent or other filesharing sites.
At least for Google, they have almost every game worth playing documented in precise frame by frame detail on YouTube. They don't need access to the source code, or compiled binaries to reverse engineer game mechanics. They built vision models perfectly capable of ingesting those game play videos and derive game play understanding from them.
Plus, there's so much open source, and game engine SDKs, freely available, that with a few game engineers at Deep Mind, they can write training systems that connect the dots.
It's definitely a gray area because these models are doing what an indie game dev would do, but at a scale and precision that's basically pirating the game, and getting around the rules of copyright to do it legally.
Meta dropped out of the LLM race for a while shortly after the court rulings came out. (and their LLM sucked relatively to other offerings) Then Zuckerberg installed a new AI guy who specialized in things like data labelling. Then Meta came back with a better LLM.
Can't say for sure, but the coincidences align pretty well.
Not saying you're wrong, but LLMs can work in areas they've never been exposed to in languages they've never been exposed to. Training is not needed for something to be possible
Up to a point I imagine. There's very domain specific stuff that I seriously doubt an LLM can create from scratch. Eg: how a DAW balances plugins/audio channels/etc with CPU cores.
He’s just saying that it can generalize. In your hypothetical, if it was given the rules of your syntax it could write programs in it despite not ever being trained on any actual programs in your syntax
There is absolutely no way to prove that this claim is true because no model advanced enough to do so at a useful level has any fine grained report of its training data that could be used to verify this.
This is trivially true. Why can they work with a tool or API I’ve written or something internal to a company? They definitely have not trained on my local machine from 2020.
The only reason at this point that there is no way to prove the claim is true is that we cannot inspect the weights, nor query for copyright violations.
I imagine that someone will eventually figure out how to query black-box LLMs for copyright violations without triggering their guardrails. This is an existential threat for anyone running a black-box LLM with stolen data.
We already know how to and have proved it, before the guardrails got "better" (mostly at hiding it). Early versions of Claude and ChatGPT were fed with a paragraph or two of popular fiction and spat out a half dozen paragraphs more, sometimes verbatim. They're significantly lossy compression algorithms. Not judging just based on that, but it's clear that their creators engaged in far more than the already outrageous amounts of copyright infringement that they've admitted and been so gently slapped on the wrist for.
I think the vibe coded plugins/DAWs are multiplying because electronic musicians with good ideas but lacking programming skills now have the tools to make things. SynthEdit, SynthMaster and Kontakt scripting engine were tools of choice for this previously.
I’ve noticed this at KVRaudio.com, people vibecode mostly for the love of the tools and curiosity about making machines that make noise. The ship for making real money from electronic music/tools pretty much sailed long ago unless you’re a brilliant dev who also understands the business, QC, and marketing side of things and had the foresight/integrity not to hit The Devil’s Johnson of VC/PE $$$ - say u-he, Ableton, Valhalla DSP, Cytomic, Madrona Labs, FabFilter and a few others.
It should be theoretically possible to use C/etc. source code, assembly output, and compiled output in the training set with equivalence mapping between them. That would make it possible for the LLM to learn disassembly and decompiling to some degree.
You don't need software for that aside from the compiler/linker tools (which are free for C/C++ compilers, rust, etc.). You can then permutate/generate the source code using tools/techniques like fuzzing.
There's also things like Intel's instruction set documentation.
Also the 1 Million token context window helped make it more viable.
I've made a bunch of unreleased vibe-coded VST plugins to replace the hundreds that I've bought over the years - and many of the barriers to development weren't the model. Just having belief that the model was capable of the work, a bit like the 4-minute mile. Also knowing to persevere and iterate with the model, not giving up because the first UI is inevitably so amateurish.
There's also a flywheel happening. If you make permanent notes on the process, your rules for UI design, a library of re-usable components and algorithms, tools so the model can judge its own work... the whole process for the next VST just gets much faster & much easier. Same goes for any software you make with AI too.
Yes, plus there is the fact that there is a HUGE amount of open source audio code out there, many, many DAW’s and so on.
I chalk it up to synth and audio guys being among the more innovative users, personally - its a market of folks who have always wanted to design their own DSP/audio-graphs, and were primed for the opportunity that AI/ML presented to do so.
The same goes for the vibe-coded game communities, and as well all the retro guys.
Folks tend to not realize just how much open source is out there in fields/subjects they’re not really familiar with.
It hasn't been. Since Opus 4.5 "letting Dario take the wheel" became useful for the vast majority of programmers.
As of this fall, you are being professionally irresponsible and SHOULD NOT be working in the field if AI is not integral to your SDLC. At a minimum it should be checking your code for bugs and vulnerabilities.
There are legal issues, ethical issues, moral issues, aesthetic issues, economic issues, financial issues, availability issues, longevity issues, consistency issues, reproducibility issues, sovereignty issues, regulatory issues, licensing issues, terms of service issues, and data retention issues that need to be addressed before AI can even be permitted in my software development life cycle - much less made mandatory on pain of unemployment.
Did it come across as dismissive of security issues? I apologise. The security issues are exactly what’s being weighed against those other issues. Both the security issues and the other issues are valid issues and it’s valid to weigh them against each other.
And I sort of understand what you’re trying to say in that second line, something along the lines of “the only thing that stops a bad guy with a gun, is a good guy with a gun”, right? Unfortunately it’s poorly phrased.
If AI isn’t hardening my code, then my code is very vulnerable to AI hacking from an adversary, I’ll grant that. But an absence of my AI does not cause the presence of an adversary’s AI in the way that the binary construction claims. (Compare “if you’re not with us, you’re against us”, which is a threat that says the absence of your fealty does cause the presence of my hostility.)
Likewise, I’ll grant that AI hardening my code does mitigate the damage from AI hacking, but again its presence does not cause the absence of an adversary’s AI hacking in the way your phrasing says it does.
Audio plugins are something that lots of people wanted to try making, but few (until now) could. Now they all can, and they do.
They're about to learn the hard way that the market wasn't profitable even without 10x the number of new entrants.
And that a third of the paying customer base (which was not that large to begin with) gave up on plug-ins anyway to generate finished music tracks from prompts.
In that case it would be movie-plot worthy when the LLM was trained on all those embedded viruses and malware. "Believing" it is intentional and then start embedding it into all software it creates for us.
I don't think it'll make sense for an LLM to program directly on the cpu opcode level due to the overhead.
But I hope you all realize that the programming language of the future is going to be for LLMs. It'll tightly optimized for LLMs and we'll get nice tools to understand them. They'll probably be great, with loads of fancy tooling.
The programming language of the future will be a purely functional language, where the LLM acts as the compiler using correctness preserving transforms.
Any other way of coding will be considered ridiculous (except maybe if you do it for a hobby).
In its purest form there would be no programming at all, you would just be interacting directly with the AI model. The interface would be like an amorphous digital goo that can instantly shape-shift into any state.
A learned comment in this forum stuck with me: suppose diff has not been written. You can use an AI to write diff. Or you can use AI to do a diff. The latter might be easier, but now you have to pay every time you diff files.
But as AI proliferates and gets cheaper, why not pay to diff files. It will be sufficiently cheap.
But you pay an electricity fee for diffing. and/or you pay a battery life fee. You pay a timeslice of your finite human life. You pay an amount of depreciation cost of your computer hardware - if you bought a $1000 computer and all you did was diff, that would be $1000 to diff.
My daily computer - MacBook Air M1 - is quite old so I would indeed have to pay a fee to diff with AI. But I think what happens is that you get a desktop computer with a massive amount of AI processing power that typically just sits idle. In that case, why not just have your on-device AI do something rather than finding the right pre-written tool or writing up the appropriate tool? This will be especially true for occasional or one-off tasks.
I don't think we'll see a world where local llms are capable of diffing a file in the millisecond timescale that current diffing does.
Same for most things. How long does it take some juiced up llm to build an mp4 renderer? Can it do all of that in the milliseconds it takes mpv file.mp4 to display it's first rendered frames?
What about a video game? How long would it take the god of all AI, running on local hardware, to build and launch GTA 6? Could it do so in less than the ~4hour download and ~1 hour install? Could it do so in less than the hour install if I didn't have it installed? Could it build and launch in less than the ~minutes it would take to launch if I'd already downloaded and installed it?
Suppose in this future someone else runs their llms for many days/hours/weeks across many agents totalling perhaps years of clock time. They have a game/program that does something I'd like to do/play, and it sits in say a terabyte of space. The only way me using my agents instead of downloading their file makes sense is if I have so vastly much more compute than them that I can complete what they did in days/hours/weeks in around a day (assuming my internet somehow stays pegged to 100mb/s in this future), because I can download a terabyte in 24 hours.
Haskell, future, pure and LLMs in the same sentence can only describe a good joke. But that wasn't one. So I guess you're serious. It's basically a useless language that hasn't been used for doing anything meaningfull, I tried using functional languages for doing "pure game programming" performance and type gymnastic was a total disaster. Spaghettified code even before LLMs
There are good reasons why people have tried for decades to make functional programming practical. Haskell is a great specification language that is also executable. LLMs can make it practical IF they can use correctness preserving transforms to compile this specification into fast executable code.
You're being sarcastic but it's likely necessary. There will be so many vulnerabilities found in legacy codebases and not enough people who understand them to patch the holes. We'll have to rely on LLM to update them and the once well-understood codebase would be more and more like a vibe-coded one.
It's like watching a boat pull into port. You can see the boat on the horizon, but people are saying the boat won't be arriving any time soon.
The boat's here. Nobody's going to be programming the old fashioned way anymore (except as a hobby or a specialist). You should probably get ready for the boat.
They have to go. You must keep feeble human hands off the code as much as possible. Ideally, all coding will be done in a safe and secure proprietary binary format owned by Anthropic, obfuscated and rich with LLM-poison booby traps to ensure that nobody can attempt to modify it using inefficient meatcoding or dangerous and unsafe bootleg Chinese LLMs. Only then can we achieve maximally aligned and efficient GSI and infinite prosperity for all.
Can it? Have we seen anything created by ai work at scale? Like the rust Postgres clone or the photoshop one.
This thread is completely detached from AIs real capabilities
LLM are great at prototypes and starting things but once you get to a certain level of complexity they cannot make progress without stumbling over itself
Most people on HN are completely detached from the reality of AI it seems. They see all these AI this AI that headlines, discovered vulnerabilities and for some reason think "yea AI can fix that".
Established programming languages have tons of material in the training sets. Any new language invented specifically for slop machines is at a huge disadvantage due to that. Also, what's optimized for today's models may not work too nicely with next month's models.
Emitting a executable binary (eg. ELF) will never make sense over using assembly language. Adding/removing some code to the middle of a large program shifts tons of addresses like jump targets and variables. It makes no sense to have LLM spend tokens on making edits to (potentially) every single call instruction in an existing binary, instead of letting assembler do it.
Second, it seems to have claimed the FreeBSD binary is for Linux. So it is not reading the platform value from the ELF headers, or looking closely at the syscall ABI in the static libc. Which is to say it makes mistakes.
Back in the day, thats how we got executables from one system to another, over ASCII-tuned serial lines. I had a colleague in the 80’s who could muscle-memory a “hx.c” which would produce the requisite hex listing given an input, and if that input was a hex listing, reproduce the binary. It was something like 12 lines of C code and it saved our asses so many times, he eventually made a DOS compatible equivalent that could be piped into DEBUG.com to produce the same results.
IIRC there was a bastard mini version of uudecode that was written so that all the opcodes were printable ascii, so you could save the text block, mark it as executable, and off you went... to get a uuencode of the full uudecode binary, decode it with your bastard mini uudecode, and then you were all set.
Not quite sure if this was the same thing, but I recall getting small utilities off some FidoNet echo for dos users which used DEBUG.com to turn it from ASCII into an executable binary.
As far as I know my colleagues tools were never pushed online, we were air-gapped from Fido and USENET and so on back then, but they might’ve made it out there .. he was mighty good at typing it in anew when it was needed though, literally a minute and we’d have bins available to do the rest. I used to use it to get zmodem.exe and kermit.exe onboard remotely managed DOS systems that didn’t have compilers .. I also had a 20 line debug.com assembly program that would do sector copies of floppy disks that I kept around/retyped when needed for such purposes. Those were fun times…
This can be useful in the reverse engineer / malware analysis case where there is existing machine code and no source but I don't think that AI "writing machine code directly rather than high level languages" makes sense. Code generation by LLMs is a non deterministic process so it makes far more sense to generate higher level compilable source code once and then compile it with a deterministic compiler for each target rather than generate machine code non deterministically several times seperately for each target.
An uninteresting case I am sure, but I have been leaning on LLMs to disassemble 6502 hex dumps and it's no doubt easy-lifting for them.
And the proof of course is that I can reassemble/link, with "modern" tools, their presented assembly and it combines right back into a hex dump mirroring the original.
Anyway, it's been a joy to take the long-out-of-print book, "The First Book of KIM", and re-typeset it (re-assemble it?) for modern hobbyists like myself. LLMs have allowed me to clean up the poor OCR-extracted text from the original. Claude has not only done that, but found missing labels and "equates" in the assembly, as legitimately printed in the book—not OCR errors.
Heck, Claude latched on to a bug in the classic "WUMPUS" game that apparently the author was not even aware of. (But I left that bug in for hobbyists to find—or exploit.)
I regularly hand a binary executable to chatgpt and ask it where it is seg faulting and why. It's a huge timesaver for me.
I also submit to it object files, and ask if it is well-formed. This is a big deal, because Apple's documentation on the object file format is extremely poorly documented. Sometimes my object files cause the Apple linker to crash.
It makes sense to let the LLM's work as close to an ideal machine as possible. In a year they are not going to need the high-level abstractions and languages. An instant stream of opcodes from prompts.
Here is an example of Astra reimplementing jq in pure .ll, the intermediate representation LLVM uses. It manages to bridge the gap and create architectural structure to solve the challenge without libraries, which GPT-5.x could not.
It feels like image models which create images and video by bitbanging pure pixels. This is almost never the way people would approach the task of creating images barring pixel graphics. It also means that using image models is inconvinient. I always likened it to having programming AI that would go from prompt to binary. I thought we don’t program like that because programmers understand the problem area better and like to go through source code.
This has been true for some time now. I can't remember details, but about a year ago, I was debugging some boot issues on an AVR microcontroller, and just pasting the HEX of the bootloader (that I didn't have access to the source of) into Claude's web interface got me the answers I needed.
Yeah sure, people have experimented with giving them binaries since the first instruction-tuned models. They do it poorly though, compilers and decompilers are still necessary.
They keep getting better. And they know the system. They can literally build and validate reverse engineering tools all by themselves, and they know enough of the machine to literally extrapolate. You could basically tell them machine format and he’ll climb them to a completely valid C compiler for it. That’s the power of Them speaking in machine code. It’s just another language. It’s actually a bounded and somewhat consistent language… and they speak it.
My point is that the data is much worse for machine code than for high-level languages, and it will always stay that way unless you generate more. Which doesn't make much sense as HLLs are just more efficient, being designed around more expressive semantics and formal constraints that can be automatically verified. LLMs are always going to suck at machine code compared to HLLs for the same reason as humans do.
Everything that can be offloaded to dumb tools should be offloaded. You don't do long division when you have a calculator. It's a neat trick, you should know how to do it, but it's not very practical.
A hex dump is just extra bytes of ascii filling up context... having to deal with "20 e3 8f" is 8 characters of text, compared to 3 bytes. LLMs can already parse binary files without the text overhead.
https://tokenizer.model.box/o200k_base says that's 6 tokens in o200k_base, which is still more than the 3 tokens you get for the raw bytes but not by as much as you suggest.
The incredible part is that an LLM that wasn't explicitly trained for this, can analyze bytes as if they were natural language, and surgically modify and/or insert bytes directly without going through intermediate steps (source -> ASM -> ...).
This process would normally require a human to lift bytes -> ASM, ideally ASM -> obfuscated source, making sense of it, and doing the chain backwards, and it requires coordinating a lot of knowledge. Which of course is no problem for an LLM.
I mean it's not 'limited circumstances'. If you work on this daily it's not that big a deal. I'm a compiler engineer, and while it's not my favorite I can 'read' machine code. I have a fairly good intuition for x86 and some more esoteric architectures I've worked on
What I don't have is the encyclopedic knowledge that comes from being trained on the processor manuals.
Yeeeeep. Been saying this for months. Go paste them Base 64…. Dozens of other formats. Use an open model. Watch thinking traces: “i am now mentally decoding X” O_o
Funny I was doing this the other day. Write an HTTP server and some routes directly as a FreeBSD binary, no intermediaries, no tools. It works and you know what, I am not even surprised.
Hmm, is this new or surprising? When testing my home computer emulator code for correctness, Opus had no trouble taking C64 .prg files apart and analyse them the same way it would analyse .asm source files.
PS: and it also has no problem writing a simple 'bouncing sprite' C64 demo without going through assembly and wrap that in a .prg file (of course this is basically the 'hello world' of C64 demos and could be as-is in the training set).
I recently watched a video of a guy trying to unbrick his Harley Davidson Livewire electric motorbike. Harley Davidson were unable to do it. Perplexity created an app to sniff the canbus and talk to the battery and other components on the bike. Soon it'll be possible to write your own firmware for things like this.
[0] is the episode I'm referring to, but the whole series of videos about his woes with this bike are quite interesting.
I could be wrong, but this is really silly:
1. Read byte by byte, well yes, executables are files how else do you want them to read and write them.
2. Binaries are 1 to 1 direct representations of the assembly code they are created from. If you know the instructions set it isn't black magic to figure what byte means what.
Why on earth would it be better for llms to program in binary or assembly. We have optimizing compilers so the effort doesn't need to be spent there.
Maybe my assumptions are wrong, so please correct me.
Edit: but yes, in general it's utterly amazing I can ask computer to tell what a computer program does, even a binary file.
Since disassembly is a solved problem and it's deterministic with a 1:1 solution, why throw a nondeterministic LLM at it? Why not give the disassembly output to the LLM and let it do the decompilation, which is not 1:1?
Pet peeve: Using AI to solve problems that existing algorithms can dependably solve with 10^-6 the CPU resources.
Disassembly is not 100% solved, at least not in a way that is easy for an LLM to use.
Disassembling directly an executable program, especially when it is some proprietary program for which there is no debugging information, is likely to produce a lot of garbage that does not correspond to correct assembly instructions.
Correct disassembly can be obtained only by simulating the execution of the program and disassembling only the instructions that are reached in this way. This is still not completely foolproof, because during simulation some parts of the code might be reached only for some specific input data, which is unknown. Sometimes one may guess all the possible inputs and disassemble anything reached in this way, but in reality some of those inputs can never occur, which can lead to erroneously disassembling parts that should not be disassembled, as they are not instructions.
That said, most executable programs are easy to disassemble, but some may be intentionally or unintentionally very obfuscated, and for such cases any disassembling tool would have to be guided by a human to achieve a complete disassembly.
If you have bytes that are known to be CPU instructions, disassembling them is indeed trivial.
But from where can you get such bytes?
Normally such bytes can be obtained only from a binary executable file, by what you called "reverse engineering", though complete reverse engineering includes a lot of work beyond identifying the machine instructions inside a binary file and disassembling/decompiling them.
I cannot imagine what source of bytes that are machine instructions do you have in mind, otherwise than those that must be obtained through reverse engineering.
If you generated yourself the binary file, there is no need to disassemble it, because any high-level language compiler provides upon request the corresponding assembly language program.
No, with reverse engineering I meant arriving at some readable code in a programming language.
But I had both misread what you wrote and was mistaken, even simply translating bytes in an executable to assembly is not trivial; I last dealt with it long enough that I misremembered it, and its issues didn't come to mind.
I think they're worried about self-modifying code, compressed code, intermingled code and data, hand-crafted overlapping instruction sequences (for systems with non-fixed instruction lengths) and other esoteric cases like that.
I am not aware of any other situation where you have a sequence of bytes that are machine instructions and that must be disassembled, except when you have extracted them from a binary executable file after identifying their location, which is indeed a part of reverse engineering.
If you do not extract the bytes to be disassembled from a binary file that you have generated yourself, then that is a part of reverse engineering.
If you generated the binary file yourself, you have no need to disassemble it. If you want to look at the assembly language program, you do not need to disassemble anything, because whatever tools you used to generate the binary will provide the assembly language equivalent.
The part about "erroneously disassembling parts that should not be disassembled, as they are not instructions" makes me think of const data in a text section.
I'll admit I expanded on the possibilities after that, because otherwise it didn't make much sense. Nobody cares if an instruction is run or not. You can still disassemble the instructions. In fact reading non-reachable code might reveal vulnerabilities you could reach if you altered the stack via a buffer overflow... only evaluating actual program flow seems like a way to overlook things.
Disassembling a binary file is never needed, unless it is a file that you have not generated yourself, in which case this is indeed a part of reverse engineering.
If you have no debugging information for the file, you cannot know that some byte sequence is unreachable code.
To identify what such a sequence of bytes is, you can try to decode it in various ways, e.g. by disassembling it, by interpreting it as ASCII or Unicode characters, by interpreting it as a sequence of floating-point or integer numbers, etc. If anyone of these decoding attempts looks plausible, you can guess that it is correct. So if it disassembles into a sequence of plausible instructions, you can conclude that it is unreachable code, but your guess is seldom certain.
In any case, the disassembling may start only by executing the program and following the control flow, and also noting all memory addresses that are accessed by loads and stores.
If after that there remain memory regions that have not been accessed in any way, possibly because you could not simulate all the possible program inputs, e.g. because the program accessed a hardware dongle that you do not possess, then you can start to decode in various ways the regions with unknown content, attempting to guess their purpose.
That's a bad and wrong point. You don't need expertise to throw the disassembly output at the LLM either. The LLM itself could do this as a first step.
Doing it completely in the LLM would do nothing but add orders of magnitude less efficiency.
Like the weight space stuff, I think there's a small misdirection in how this is presented: Phrases like "write directly in machine code" seem to imply that ordinary LLMs intuitively understand machine code as just another language, like English or JavaScript, that they read from context and write directly as tokens, and that this intuitive understanding could mean increased efficiency or capabilities if you ask the LLM to just use the bytes rather than asking it to use the tools that human reverse engineers use.
If that's what you assumed, the DeepSeek reasoning trace shatters that illusion. It's just disassembling by hand like what a human reverse engineer would do if you (sadistically?) denied them a disassembler. Yeah, the human reverse engineer would need an excellent memory of the x86_64 instruction reference, but we already know LLMs can recall a breadth of information. What we don't know, that this article also provides no evidence for, is if there will be any benefit in using the LLM for this over objdump.
The article seems to intentionally conflate disassembly (computationally cheap) with decompilation (can be computationally expensive) to imply that there's some benefit to asking the LLM to work with machine code (seems impressive to the RE layman) rather than working in assembly (something you probably already assumed LLMs can do,) but the DeepSeek trace has a disassembly in it, and it's hard to imagine that inference will ever be cheaper or more reliable per-token than objdump is at producing the same output.
I’m seeing the conflation between disassemble and decompile in other posts here too. I don’t know why you’d waste time and money using an LLM to disassemble an executable when it’s been trivial to do that with existing (free) tools for years!
Thats because those authors dont know the difference between the two terms, which is precisely the problem.
I wrote my own disassembler for object files. They're easy to write. Doing it for executables is even easier, though I haven't needed to do them.
I've found the decompiled code from LLM's to be much more similar to human-written code (and therefore easier for me to understand) than standard Ghidra/other decompiler output. Plus LLM's for this kind of work are pretty cheap.
The parent comment was making the point that disassembling and decompiling are not the same. There is no room for interpretation on disassembling, it's purely mechanical.
There's a little room for interpretation in disassembling. You can choose label names, and in instruction sets like amd64, you have to guess where the instructions begin. Also ideally you won't disassemble ARM literal pools as instructions, even though it's possible to branch, call, or return into them by computing their addresses at runtime. In binary formats that don't separate code from .rodata the problem can get arbitrarily hairy.
> in instruction sets like amd64, you have to guess where the instructions begin
Can't you just trace through all reachable instructions starting from a complete binary's entry point(s), as long as the code doesn't do unusual things like jumping into the middle of instructions, i.e. intentionally exploiting variable-length instruction ambiguity?
Not necessarily, because there is often code that is only accessible through function pointers; for example, C++ virtual methods. (In theory you can compute function pointers dynamically at runtime, too, which is less common.)
In https://canonical.org/~kragen/sw/dev3/qfitzah.s a lot of the function calls are conditionally compiled by the macro `do` (search for `.macro do`) to index into a table of function pointers pointed to by EBP, because that makes the call instructions 3 bytes instead of 5 bytes. Here I've disassembled one of the functions from the parser code from an unstripped version that has debugging symbols:
The actual source code to that subroutine looks like this: I didn't do this to make it harder to disassemble, but it does have that drawback.For what it's worth, Claude Opus 5.5 had no trouble disassembling it and making a list of its bugs, including six I already knew about and one I didn't. It commented:
> Reverse-engineering a program this small is satisfying in a way bigger programs rarely are, because you can hold the whole thing in your head. Working out the scheme of keeping type tags in the low pointer bits from nothing but `test al,3` and `test cl,2` scattered through the code was the best part. Watching `(D x) (x x)` print 4 MB correctly, and then segfault on the next query, was a close second.
In 32-bit ARM code you have the additional problem that you have to know whether the T bit is intended to be set when the CPU is decoding a particular subroutine. The low bit of the branch destination tells you — which is fine if it's statically reachable, but you may have to guess if it's only dynamically reachable.
disassembly (computationally cheap)
Only if you do "straight line" disassembly. On the other hand, code/data separation is extremely intensive, especially when indirect branches are involved, and approaches a decompiler in computational effort.
IIRC even disassembly is undecidable. The hardness indeed comes from jumps, you don’t know where instructions start (they don’t all have the same number of bytes).
Edit: I think the problem I have in mind which is undecidable is whether a byte is (the start of) an instruction.
Not all ISAs are variable-length.
Ooh I'm not sure that I would buy into that! I've been experimenting with giving small binaries without strings or other direct clues, and asking them "So tell me what you think this does" in a bare container without disassembly tools.
So I'm actually fairly confident that the ability does come straight from the ability to read the bytes into context (usually as hex bash/perl/python3 is almost always available), and Then to directly interpret what the binary does from the code. And yes, when I do use open models, I can see from their reasoning traces that they do "hand-translate" the hex into assembly, but I kind of suspect that the translation stage into assembly may just be a narration, and the "understanding" of what is going on is actually a pure latent skill. That's just my guess, though.
The reason that I think this is because I've been using minimal implementations of inference engines. And frontier closed-weight models and open-weight models are able to observe a binary with no strings and the weights not attached, and they can infer the exact architecture and model (eg qwen3 0.6b, qwen3.5-0.8b, whatever) from the fixed shape of the inference kernels.
So in effect, over the course of 30 seconds, they go from looking at 500 instructions of x86 code and reply, “I know what this is. It's an inference engine for Qwen 3 0.6B!”
And I'm fairly sure that 30 seconds or not, it's a skill that no single human has.
Assembly code is almost just machine code, which itself is nothing magical. Human cannot work with machine code because of the logical complexity, not because it's hex.
Matching decompilation is very tricky, but the decompiled output that ghidra spits out is good enough for many cases and is not significantly more computationally than getting a disassembly.
I've suspected for years that AI companies have been using commercial software to train their models including games, audio plugings, DAWs, etc. It would not surprise me one bit if we find out they used torrented pirated apps like they did with books and music.
There's certainly eg open source audio dev material out there but not nearly as much as web dev. How else do you explain all the avalanche of vibe coded audio plugins and DAWs appearing this year?
And (from a training material point of view) it makes perfect sense.
I would assume that to a LLM, a binary executable is not much worse than actual source code. And it's much denser, just missing the semantics of human-written identifiers. It's the code's pure logic, condensed into machine language.
And there's tons of it, many vendors supply it "freely" (you don't need license keys to train on a binary) It's often battle-tested code that you wouldn't get hold of otherwise, and compared to ingested source code its unlikely that Codex regurgitates a recognizable part of a Windows 2000 DLL into your C++ source code.
That's not quite right - source expresses the idea relative to a language syntax and grammar, whereas machine code (or assembler) implements the idea against a specific target machine, for a specific set of circumstances.
So as an example, there are a bunch of different ways a for or while loop can be implemented in compiled code, but there are also situations where two quite different source constructs can result in the same machine code sequence.
Machine code certainly won't teach an LLM much about how to write good source code.
Do you think a programmer that knows the low level assembly that will be generated for any source, and necessarily the reverse possibilities, would be a better or worse programmer?
As a programmer who implemented multiple compilers, yes, I'm pretty good at knowing what the resulting instruction stream looks like.
Being deterministic, a binary executable is just a compressed version of source code.
Compressed doesn't sound quite right. Source code contains abstractions which are often then expanded out into many variants in an executable (most obviously by function inlining, but at a smaller scope also by things like loop unrolling etc)
Lossy compression.
No, it goes the other way. Source code is a lossy compression of the billions of machine operations it can be compiled into. Abstraction is a way of compressing, both in the mechanical LoC sense, as well as in comprehension.
gainy expansion?
Many things also expand significantly in binary form. Heavy use of templating, for example.
Good observation!
Another one would be range-based foreach loops.
Is plate of spaghetti a compressed version of a recipe?
Executables are often larger than their source code.
A toolchain that creates repeatable binary builds is difficult to create. And a nonrepeatable build is potentially nondeterministic, in the sense that uncontrolled or hidden inputs impact the output.
This is very analogous to vector vs raster graphics. In a way, the vector format is strictly more powerful than the raster format as it can be rendered smoothly at any size. Thus the raster graphic, despite being larger in size, is a lossy distillation of the vector graphic.
Also it _is_ possible to gzip a file and get a larger file, there’s an inflection point based on the distribution of symbols in the source where the same compression algorithm produces a larger file than the original
A raster is not always larger than a vector; if you try to precisely vectorize an image of the real world (modeling each pixel), the vector version will be larger
Let me make a more rigorous claim: For any image, there is always a vector encoding that is at least as small as the smallest possible raster encoding. The converse is not true.
>Executables are often larger than their source code.
Not because they contain more information. Just stuff like inlining or different expression, plus image and other similar assets that we can ignore for this.
The term I would use is “projection” rather than compression.
Code belongs to the domain of source code. Compilation is a projection of that code into the domain of executables, and that involves enough types of transformation and mapping that it is a one-way operation.
From an information standpoint, some information is added (e.g. optimizations that encode knowledge about efficient bytecode) and some information is lost (e.g. the use of types that adapt to the native word size).
Projection as an instantiation, makes sense.
My hunch is that one of the handicaps Google is dealing with (vs. OpenAI and Anthropics) is that Google is not training on pirated media. They are using YouTube, Google book project scans, and purchased copies of media.
Perhaps they are not actively collecting it like Ant/OAI but YouTube and Google Books are full of user-uploaded pirated media.
YouTube has a massive system in place to find and block pirated material. I can't remember the last time I saw a simple video rip on YouTube.
Nowadays you're getting gray area stuff like new AI generated stories based in the Harry Potter universe, which is probably copyright violation but it's not actually pirated video, so Content ID doesn't spot it.
Nowadays you're getting gray area stuff like new AI generated stories based in the Harry Potter universe
Fanfiction has always existed, albeit in written form, but copyright takedowns are almost unheard of. The gray area is definitely getting bigger, and AI is effectively showing everyone that everything is a derivative work to some extent. I suspect we'll see a massive shift in the general sentiment around IP in the near future.
They might still train on stuff that's not publicly available on YouTube anymore
OP is talking about binaries
Its the doctor google problem, some people take any random page with health advise as authentic and authorative. My guess Google and its training also has a similar problem, how can they tell which sources are correct and which are the opposite. Of course there are some easy wins, such as checking the domain but considering that the majority of the internet is filled with random pages of stuff its difficult to judge those.
I suspect that Anthropic, being public, and under scrutiny, will get sued into compliance from all over the world. It's not a foregone conclusion that the data they trained on remains fair game over the long term.
At this point they'll get nothing more than a slap on the wrist. No judge wants to be the one to wipe hundreds of billions of dollars off the stock market.
There is zero chance models are not trained on pirated media and games, especially most popular ones.
Makes complete sense to study things people actually want to use and consume
I agree 100% with sentiment. It is inconceivable to me that models were not trained, either directly, or indirectly through other aggregates, on code and binaries and, of course, media, fetched from torrent or other filesharing sites.
At least for Google, they have almost every game worth playing documented in precise frame by frame detail on YouTube. They don't need access to the source code, or compiled binaries to reverse engineer game mechanics. They built vision models perfectly capable of ingesting those game play videos and derive game play understanding from them.
Plus, there's so much open source, and game engine SDKs, freely available, that with a few game engineers at Deep Mind, they can write training systems that connect the dots.
It's definitely a gray area because these models are doing what an indie game dev would do, but at a scale and precision that's basically pirating the game, and getting around the rules of copyright to do it legally.
wouldn't that information leak out or somehow found out? too many people already involved. With some, like Seedance it's obvious of course.
I thought the fact they fed copyrighted material into the models has been established.
It has leaked out as part of court cases that Meta has trained LLMs using pirate text torrents.
No reason to belive none of the labs have similarly trained on binaries.
Meta dropped out of the LLM race for a while shortly after the court rulings came out. (and their LLM sucked relatively to other offerings) Then Zuckerberg installed a new AI guy who specialized in things like data labelling. Then Meta came back with a better LLM.
Can't say for sure, but the coincidences align pretty well.
It will once all those involved cash out from the AI companies they’re working with…
Not saying you're wrong, but LLMs can work in areas they've never been exposed to in languages they've never been exposed to. Training is not needed for something to be possible
They also use synthetic generated data to train on too
If it works and it's verified in nine different ways, throw it in the training pile, and then automate the process.
That is how you end up with model collapse which, I assume, we will see more and more of.
There were fears about this 3-4 years ago but they never manifested. All the labs train heavily on synthetic data today
Up to a point I imagine. There's very domain specific stuff that I seriously doubt an LLM can create from scratch. Eg: how a DAW balances plugins/audio channels/etc with CPU cores.
If I made up the syntax for a language, and an algorithm that hasn't been used before, to solve a novel problem.
An LLM can give me a solution, in my hypothetical language, without knowledge of any of those things?
I'm not sure I understand what you're trying to say, because that claim seems impossible.
He’s just saying that it can generalize. In your hypothetical, if it was given the rules of your syntax it could write programs in it despite not ever being trained on any actual programs in your syntax
There is absolutely no way to prove that this claim is true because no model advanced enough to do so at a useful level has any fine grained report of its training data that could be used to verify this.
This is trivially true. Why can they work with a tool or API I’ve written or something internal to a company? They definitely have not trained on my local machine from 2020.
The only reason at this point that there is no way to prove the claim is true is that we cannot inspect the weights, nor query for copyright violations.
I imagine that someone will eventually figure out how to query black-box LLMs for copyright violations without triggering their guardrails. This is an existential threat for anyone running a black-box LLM with stolen data.
We already know how to and have proved it, before the guardrails got "better" (mostly at hiding it). Early versions of Claude and ChatGPT were fed with a paragraph or two of popular fiction and spat out a half dozen paragraphs more, sometimes verbatim. They're significantly lossy compression algorithms. Not judging just based on that, but it's clear that their creators engaged in far more than the already outrageous amounts of copyright infringement that they've admitted and been so gently slapped on the wrist for.
I'm pretty sure companies are not throwing away their training data each time they build a model.
I think the vibe coded plugins/DAWs are multiplying because electronic musicians with good ideas but lacking programming skills now have the tools to make things. SynthEdit, SynthMaster and Kontakt scripting engine were tools of choice for this previously.
I’ve noticed this at KVRaudio.com, people vibecode mostly for the love of the tools and curiosity about making machines that make noise. The ship for making real money from electronic music/tools pretty much sailed long ago unless you’re a brilliant dev who also understands the business, QC, and marketing side of things and had the foresight/integrity not to hit The Devil’s Johnson of VC/PE $$$ - say u-he, Ableton, Valhalla DSP, Cytomic, Madrona Labs, FabFilter and a few others.
It should be theoretically possible to use C/etc. source code, assembly output, and compiled output in the training set with equivalence mapping between them. That would make it possible for the LLM to learn disassembly and decompiling to some degree.
You don't need software for that aside from the compiler/linker tools (which are free for C/C++ compilers, rust, etc.). You can then permutate/generate the source code using tools/techniques like fuzzing.
There's also things like Intel's instruction set documentation.
An LLM trained on decompilation would be a game changer, but there's no way they'd allow us to use it.
Maybe you could use a regular decompiler. It makes bad variable and function names. Then get an existing LLM to correct the names.
It's been a thing for a couple of years now e.g.
https://arxiv.org/html/2403.05286v2
> How else do you explain all the avalanche of vibe coded audio plugins and DAWs
By them being nothing special?
> appearing this year?
Vibe coding in general became viable last year, Opus 4.5 is Nov 24, 2025.
Also the 1 Million token context window helped make it more viable.
I've made a bunch of unreleased vibe-coded VST plugins to replace the hundreds that I've bought over the years - and many of the barriers to development weren't the model. Just having belief that the model was capable of the work, a bit like the 4-minute mile. Also knowing to persevere and iterate with the model, not giving up because the first UI is inevitably so amateurish.
There's also a flywheel happening. If you make permanent notes on the process, your rules for UI design, a library of re-usable components and algorithms, tools so the model can judge its own work... the whole process for the next VST just gets much faster & much easier. Same goes for any software you make with AI too.
Yes, plus there is the fact that there is a HUGE amount of open source audio code out there, many, many DAW’s and so on.
I chalk it up to synth and audio guys being among the more innovative users, personally - its a market of folks who have always wanted to design their own DSP/audio-graphs, and were primed for the opportunity that AI/ML presented to do so.
The same goes for the vibe-coded game communities, and as well all the retro guys.
Folks tend to not realize just how much open source is out there in fields/subjects they’re not really familiar with.
> By them being nothing special?
I would agree about a simple reverb or a distortion but there are full vibe coded DAWs out there.
so weird how AI boosters act like it hasn't been less than a year, isn't it?
It hasn't been. Since Opus 4.5 "letting Dario take the wheel" became useful for the vast majority of programmers.
As of this fall, you are being professionally irresponsible and SHOULD NOT be working in the field if AI is not integral to your SDLC. At a minimum it should be checking your code for bugs and vulnerabilities.
There are legal issues, ethical issues, moral issues, aesthetic issues, economic issues, financial issues, availability issues, longevity issues, consistency issues, reproducibility issues, sovereignty issues, regulatory issues, licensing issues, terms of service issues, and data retention issues that need to be addressed before AI can even be permitted in my software development life cycle - much less made mandatory on pain of unemployment.
What about security issues? What about the fact that if AI isn't hardening your code under your control, it's hacking it under an adversary's?
Did it come across as dismissive of security issues? I apologise. The security issues are exactly what’s being weighed against those other issues. Both the security issues and the other issues are valid issues and it’s valid to weigh them against each other.
And I sort of understand what you’re trying to say in that second line, something along the lines of “the only thing that stops a bad guy with a gun, is a good guy with a gun”, right? Unfortunately it’s poorly phrased.
If AI isn’t hardening my code, then my code is very vulnerable to AI hacking from an adversary, I’ll grant that. But an absence of my AI does not cause the presence of an adversary’s AI in the way that the binary construction claims. (Compare “if you’re not with us, you’re against us”, which is a threat that says the absence of your fealty does cause the presence of my hostility.)
Likewise, I’ll grant that AI hardening my code does mitigate the damage from AI hacking, but again its presence does not cause the absence of an adversary’s AI hacking in the way your phrasing says it does.
this is what i'm talking about
Audio plugins are something that lots of people wanted to try making, but few (until now) could. Now they all can, and they do.
They're about to learn the hard way that the market wasn't profitable even without 10x the number of new entrants.
And that a third of the paying customer base (which was not that large to begin with) gave up on plug-ins anyway to generate finished music tracks from prompts.
AIs are quite good at using existing binary analysis tools; they usually do not just operate on binaries in “prompt space” like this
torrented pirated apps may contain all sorts of nasties, it would be amusing if the nastiness gets embedded in the LLMs
In that case it would be movie-plot worthy when the LLM was trained on all those embedded viruses and malware. "Believing" it is intentional and then start embedding it into all software it creates for us.
report shows AI botnet is using your fridge as distributed command and control server
"Hello, we're here to see whether or not your fridge is runni-Hey, no not like that, this is serious!"
I want to see stuxnet for my LG fridge
I don't think it'll make sense for an LLM to program directly on the cpu opcode level due to the overhead.
But I hope you all realize that the programming language of the future is going to be for LLMs. It'll tightly optimized for LLMs and we'll get nice tools to understand them. They'll probably be great, with loads of fancy tooling.
The programming language of the future will be a purely functional language, where the LLM acts as the compiler using correctness preserving transforms.
Any other way of coding will be considered ridiculous (except maybe if you do it for a hobby).
Better start learning Haskell, folks.
In its purest form there would be no programming at all, you would just be interacting directly with the AI model. The interface would be like an amorphous digital goo that can instantly shape-shift into any state.
A learned comment in this forum stuck with me: suppose diff has not been written. You can use an AI to write diff. Or you can use AI to do a diff. The latter might be easier, but now you have to pay every time you diff files.
But as AI proliferates and gets cheaper, why not pay to diff files. It will be sufficiently cheap.
At that point, just ask AI to write the Holodeck and we won't ever need to use computers anymore.
I will never touch a computer again the moment I must pay a rental fee for diffing.
But you pay an electricity fee for diffing. and/or you pay a battery life fee. You pay a timeslice of your finite human life. You pay an amount of depreciation cost of your computer hardware - if you bought a $1000 computer and all you did was diff, that would be $1000 to diff.
My daily computer - MacBook Air M1 - is quite old so I would indeed have to pay a fee to diff with AI. But I think what happens is that you get a desktop computer with a massive amount of AI processing power that typically just sits idle. In that case, why not just have your on-device AI do something rather than finding the right pre-written tool or writing up the appropriate tool? This will be especially true for occasional or one-off tasks.
I don't think we'll see a world where local llms are capable of diffing a file in the millisecond timescale that current diffing does.
Same for most things. How long does it take some juiced up llm to build an mp4 renderer? Can it do all of that in the milliseconds it takes mpv file.mp4 to display it's first rendered frames?
What about a video game? How long would it take the god of all AI, running on local hardware, to build and launch GTA 6? Could it do so in less than the ~4hour download and ~1 hour install? Could it do so in less than the hour install if I didn't have it installed? Could it build and launch in less than the ~minutes it would take to launch if I'd already downloaded and installed it?
Suppose in this future someone else runs their llms for many days/hours/weeks across many agents totalling perhaps years of clock time. They have a game/program that does something I'd like to do/play, and it sits in say a terabyte of space. The only way me using my agents instead of downloading their file makes sense is if I have so vastly much more compute than them that I can complete what they did in days/hours/weeks in around a day (assuming my internet somehow stays pegged to 100mb/s in this future), because I can download a terabyte in 24 hours.
Speaking of GTA 6 ... I'd buy that game if the non-player characters had capable AI, like an LLM agent.
Imagine going into a bar:
Yeah, I'll have a beer and also find a firmware RCE for my car please.
So, Star Trek
Haskell, future, pure and LLMs in the same sentence can only describe a good joke. But that wasn't one. So I guess you're serious. It's basically a useless language that hasn't been used for doing anything meaningfull, I tried using functional languages for doing "pure game programming" performance and type gymnastic was a total disaster. Spaghettified code even before LLMs
There are good reasons why people have tried for decades to make functional programming practical. Haskell is a great specification language that is also executable. LLMs can make it practical IF they can use correctness preserving transforms to compile this specification into fast executable code.
shellcheck is written in Haskell
OCaml, F#, or even Rust have functional cores that verify correctness. They’re wire token efficient, too.
Sufficiently massaged OCaml/F# approaches straightforward readability in plain English.
And, for people who have been doing it for a minute, the other ways of programming do seem a little ridiculous ;)
Yea lets just trash all these legacy codebases the world depends on, I'm sure this will work out
You're being sarcastic but it's likely necessary. There will be so many vulnerabilities found in legacy codebases and not enough people who understand them to patch the holes. We'll have to rely on LLM to update them and the once well-understood codebase would be more and more like a vibe-coded one.
Some people, when confronted with a problem, think “I know, I’ll use AI.” Now they have many problems. They think “I know, I’ll use many AIs.”
Legacy codebases are toast. This is where the software world is heading:
https://www.youtube.com/watch?v=Bo-vESS4jt0
https://www.youtube.com/watch?v=UCL4K36SHp0
It's like watching a boat pull into port. You can see the boat on the horizon, but people are saying the boat won't be arriving any time soon.
The boat's here. Nobody's going to be programming the old fashioned way anymore (except as a hobby or a specialist). You should probably get ready for the boat.
They have to go. You must keep feeble human hands off the code as much as possible. Ideally, all coding will be done in a safe and secure proprietary binary format owned by Anthropic, obfuscated and rich with LLM-poison booby traps to ensure that nobody can attempt to modify it using inefficient meatcoding or dangerous and unsafe bootleg Chinese LLMs. Only then can we achieve maximally aligned and efficient GSI and infinite prosperity for all.
Claude can rewrite those codebases in Rust in an eyeblink. Let them all burn.
So why doesnt Microsoft rewrite a fully backcompat "close to perfect" Windows thats requires less memory and lots of other good stuff ?
Can it? Have we seen anything created by ai work at scale? Like the rust Postgres clone or the photoshop one.
This thread is completely detached from AIs real capabilities
LLM are great at prototypes and starting things but once you get to a certain level of complexity they cannot make progress without stumbling over itself
Most people on HN are completely detached from the reality of AI it seems. They see all these AI this AI that headlines, discovered vulnerabilities and for some reason think "yea AI can fix that".
What overhead? More tokens for same result?
I haven't looked into it properly but, it looks like it might actually be the same amount of tokens? (Excepting stdlib boilerplate etc.)
Established programming languages have tons of material in the training sets. Any new language invented specifically for slop machines is at a huge disadvantage due to that. Also, what's optimized for today's models may not work too nicely with next month's models.
It depends. If you design and train a tokenizer for that, I don't see why it wouldn't work, and I'm not even an "AI psychosis" victim :D
Emitting a executable binary (eg. ELF) will never make sense over using assembly language. Adding/removing some code to the middle of a large program shifts tons of addresses like jump targets and variables. It makes no sense to have LLM spend tokens on making edits to (potentially) every single call instruction in an existing binary, instead of letting assembler do it.
First, I'm not sure this result is very shocking.
Second, it seems to have claimed the FreeBSD binary is for Linux. So it is not reading the platform value from the ELF headers, or looking closely at the syscall ABI in the static libc. Which is to say it makes mistakes.
Back in the day, thats how we got executables from one system to another, over ASCII-tuned serial lines. I had a colleague in the 80’s who could muscle-memory a “hx.c” which would produce the requisite hex listing given an input, and if that input was a hex listing, reproduce the binary. It was something like 12 lines of C code and it saved our asses so many times, he eventually made a DOS compatible equivalent that could be piped into DEBUG.com to produce the same results.
IIRC there was a bastard mini version of uudecode that was written so that all the opcodes were printable ascii, so you could save the text block, mark it as executable, and off you went... to get a uuencode of the full uudecode binary, decode it with your bastard mini uudecode, and then you were all set.
Not quite sure if this was the same thing, but I recall getting small utilities off some FidoNet echo for dos users which used DEBUG.com to turn it from ASCII into an executable binary.
As far as I know my colleagues tools were never pushed online, we were air-gapped from Fido and USENET and so on back then, but they might’ve made it out there .. he was mighty good at typing it in anew when it was needed though, literally a minute and we’d have bins available to do the rest. I used to use it to get zmodem.exe and kermit.exe onboard remotely managed DOS systems that didn’t have compilers .. I also had a 20 line debug.com assembly program that would do sector copies of floppy disks that I kept around/retyped when needed for such purposes. Those were fun times…
This can be useful in the reverse engineer / malware analysis case where there is existing machine code and no source but I don't think that AI "writing machine code directly rather than high level languages" makes sense. Code generation by LLMs is a non deterministic process so it makes far more sense to generate higher level compilable source code once and then compile it with a deterministic compiler for each target rather than generate machine code non deterministically several times seperately for each target.
We also have more or less deterministic tools to lift code as well, which are usually more efficient to use.
An uninteresting case I am sure, but I have been leaning on LLMs to disassemble 6502 hex dumps and it's no doubt easy-lifting for them.
And the proof of course is that I can reassemble/link, with "modern" tools, their presented assembly and it combines right back into a hex dump mirroring the original.
Anyway, it's been a joy to take the long-out-of-print book, "The First Book of KIM", and re-typeset it (re-assemble it?) for modern hobbyists like myself. LLMs have allowed me to clean up the poor OCR-extracted text from the original. Claude has not only done that, but found missing labels and "equates" in the assembly, as legitimately printed in the book—not OCR errors.
Heck, Claude latched on to a bug in the classic "WUMPUS" game that apparently the author was not even aware of. (But I left that bug in for hobbyists to find—or exploit.)
You don’t need an LLM to disassemble code though especially for 6502. Disassembly has been a solved problem for decades.
Sure. It's practically a lookup table, right?
It's all the other stuff though—when your OCR is shit, for example. And then to add labels, equates, comments(!). Not bad.
As mentioned the other day [1], a couple of HNers including me are getting LLMS to modify Amiga games from the binary.
I’ve added websocket and webrtc networking to Lotus Turbo Challenge II
and I’m in the middle of adding 8x1 player multiplayer… it originally only supported 2x2 over serial link
[1] (me, tezza) https://news.ycombinator.com/item?id=50537075
&
(Kurtz79) https://news.ycombinator.com/item?id=50545239
I regularly hand a binary executable to chatgpt and ask it where it is seg faulting and why. It's a huge timesaver for me.
I also submit to it object files, and ask if it is well-formed. This is a big deal, because Apple's documentation on the object file format is extremely poorly documented. Sometimes my object files cause the Apple linker to crash.
Would be the best way for LLM's to create programs, we need a new way to interface, a new language.
It makes sense to let the LLM's work as close to an ideal machine as possible. In a year they are not going to need the high-level abstractions and languages. An instant stream of opcodes from prompts.
Here is an example of Astra reimplementing jq in pure .ll, the intermediate representation LLVM uses. It manages to bridge the gap and create architectural structure to solve the challenge without libraries, which GPT-5.x could not.
https://nial.se/blog/57-billion-tokens-and-nothin-on/
Interesting thanks!
> There are now 15 competing standards
It feels like image models which create images and video by bitbanging pure pixels. This is almost never the way people would approach the task of creating images barring pixel graphics. It also means that using image models is inconvinient. I always likened it to having programming AI that would go from prompt to binary. I thought we don’t program like that because programmers understand the problem area better and like to go through source code.
000000?
Nope.
000001?
Nope.
...
000000 00001 00010 00110 00000 100000?
Hey, we've got addition!
This has been true for some time now. I can't remember details, but about a year ago, I was debugging some boot issues on an AVR microcontroller, and just pasting the HEX of the bootloader (that I didn't have access to the source of) into Claude's web interface got me the answers I needed.
They can recompile games from pure windows/intel to mac silicone right now, pretty sure they can do that into binary
Yeah sure, people have experimented with giving them binaries since the first instruction-tuned models. They do it poorly though, compilers and decompilers are still necessary.
They keep getting better. And they know the system. They can literally build and validate reverse engineering tools all by themselves, and they know enough of the machine to literally extrapolate. You could basically tell them machine format and he’ll climb them to a completely valid C compiler for it. That’s the power of Them speaking in machine code. It’s just another language. It’s actually a bounded and somewhat consistent language… and they speak it.
My point is that the data is much worse for machine code than for high-level languages, and it will always stay that way unless you generate more. Which doesn't make much sense as HLLs are just more efficient, being designed around more expressive semantics and formal constraints that can be automatically verified. LLMs are always going to suck at machine code compared to HLLs for the same reason as humans do.
Everything that can be offloaded to dumb tools should be offloaded. You don't do long division when you have a calculator. It's a neat trick, you should know how to do it, but it's not very practical.
My point was they can make their own better tools. Just kinda neat.
A hex dump is just extra bytes of ascii filling up context... having to deal with "20 e3 8f" is 8 characters of text, compared to 3 bytes. LLMs can already parse binary files without the text overhead.
https://tokenizer.model.box/o200k_base says that's 6 tokens in o200k_base, which is still more than the 3 tokens you get for the raw bytes but not by as much as you suggest.
Okay, now multiply it by a 256kb rom file. /shrug
I wonder if prompt injection via an executable is possible. Especially in the context of having AI perform malware analysis.
Yes but models are fairly resistant to simple prompt injection these days. Having them hit guardrails is a more common strategy
Wow. It can read bytes from a byte output? That's incredible
The incredible part is that an LLM that wasn't explicitly trained for this, can analyze bytes as if they were natural language, and surgically modify and/or insert bytes directly without going through intermediate steps (source -> ASM -> ...).
This process would normally require a human to lift bytes -> ASM, ideally ASM -> obfuscated source, making sense of it, and doing the chain backwards, and it requires coordinating a lot of knowledge. Which of course is no problem for an LLM.
People can do this in some limited circumstances, LLMs are just better at it
I mean it's not 'limited circumstances'. If you work on this daily it's not that big a deal. I'm a compiler engineer, and while it's not my favorite I can 'read' machine code. I have a fairly good intuition for x86 and some more esoteric architectures I've worked on
What I don't have is the encyclopedic knowledge that comes from being trained on the processor manuals.
Yeeeeep. Been saying this for months. Go paste them Base 64…. Dozens of other formats. Use an open model. Watch thinking traces: “i am now mentally decoding X” O_o
Now think about homomorphism and safety filters……
So can I. CD 20
Couldn't they do this before?
They don't list any model size or version that failed, right?
Funny I was doing this the other day. Write an HTTP server and some routes directly as a FreeBSD binary, no intermediaries, no tools. It works and you know what, I am not even surprised.
If you ask the Claude chat web interface to do this, it might just compile it with clang and not bother to mention it.
Hmm, is this new or surprising? When testing my home computer emulator code for correctness, Opus had no trouble taking C64 .prg files apart and analyse them the same way it would analyse .asm source files.
PS: and it also has no problem writing a simple 'bouncing sprite' C64 demo without going through assembly and wrap that in a .prg file (of course this is basically the 'hello world' of C64 demos and could be as-is in the training set).
Let's see if the data URL survives HN:
https://floooh.github.io/tiny8bit/c64.html?file=data%3Aappli...
I recently watched a video of a guy trying to unbrick his Harley Davidson Livewire electric motorbike. Harley Davidson were unable to do it. Perplexity created an app to sniff the canbus and talk to the battery and other components on the bike. Soon it'll be possible to write your own firmware for things like this.
[0] is the episode I'm referring to, but the whole series of videos about his woes with this bike are quite interesting.
[0] https://www.youtube.com/watch?v=Z4Ty0oqA_Bk
I could be wrong, but this is really silly: 1. Read byte by byte, well yes, executables are files how else do you want them to read and write them.
2. Binaries are 1 to 1 direct representations of the assembly code they are created from. If you know the instructions set it isn't black magic to figure what byte means what.
Why on earth would it be better for llms to program in binary or assembly. We have optimizing compilers so the effort doesn't need to be spent there.
Maybe my assumptions are wrong, so please correct me.
Edit: but yes, in general it's utterly amazing I can ask computer to tell what a computer program does, even a binary file.
Since disassembly is a solved problem and it's deterministic with a 1:1 solution, why throw a nondeterministic LLM at it? Why not give the disassembly output to the LLM and let it do the decompilation, which is not 1:1?
Pet peeve: Using AI to solve problems that existing algorithms can dependably solve with 10^-6 the CPU resources.
Disassembly is not 100% solved, at least not in a way that is easy for an LLM to use.
Disassembling directly an executable program, especially when it is some proprietary program for which there is no debugging information, is likely to produce a lot of garbage that does not correspond to correct assembly instructions.
Correct disassembly can be obtained only by simulating the execution of the program and disassembling only the instructions that are reached in this way. This is still not completely foolproof, because during simulation some parts of the code might be reached only for some specific input data, which is unknown. Sometimes one may guess all the possible inputs and disassemble anything reached in this way, but in reality some of those inputs can never occur, which can lead to erroneously disassembling parts that should not be disassembled, as they are not instructions.
That said, most executable programs are easy to disassemble, but some may be intentionally or unintentionally very obfuscated, and for such cases any disassembling tool would have to be guided by a human to achieve a complete disassembly.
Disassembly means going from bytes to assembly, it's not only "solved" but trivial.
In your comment you're talking about reverse engineering.
If you have bytes that are known to be CPU instructions, disassembling them is indeed trivial.
But from where can you get such bytes?
Normally such bytes can be obtained only from a binary executable file, by what you called "reverse engineering", though complete reverse engineering includes a lot of work beyond identifying the machine instructions inside a binary file and disassembling/decompiling them.
I cannot imagine what source of bytes that are machine instructions do you have in mind, otherwise than those that must be obtained through reverse engineering.
If you generated yourself the binary file, there is no need to disassemble it, because any high-level language compiler provides upon request the corresponding assembly language program.
> by what you called "reverse engineering"
No, with reverse engineering I meant arriving at some readable code in a programming language.
But I had both misread what you wrote and was mistaken, even simply translating bytes in an executable to assembly is not trivial; I last dealt with it long enough that I misremembered it, and its issues didn't come to mind.
I think they're worried about self-modifying code, compressed code, intermingled code and data, hand-crafted overlapping instruction sequences (for systems with non-fixed instruction lengths) and other esoteric cases like that.
No, to me it seems that they were really just talking of reverse engineering
Yes I was talking about reverse engineering.
I am not aware of any other situation where you have a sequence of bytes that are machine instructions and that must be disassembled, except when you have extracted them from a binary executable file after identifying their location, which is indeed a part of reverse engineering.
If you do not extract the bytes to be disassembled from a binary file that you have generated yourself, then that is a part of reverse engineering.
If you generated the binary file yourself, you have no need to disassemble it. If you want to look at the assembly language program, you do not need to disassemble anything, because whatever tools you used to generate the binary will provide the assembly language equivalent.
The part about "erroneously disassembling parts that should not be disassembled, as they are not instructions" makes me think of const data in a text section.
I'll admit I expanded on the possibilities after that, because otherwise it didn't make much sense. Nobody cares if an instruction is run or not. You can still disassemble the instructions. In fact reading non-reachable code might reveal vulnerabilities you could reach if you altered the stack via a buffer overflow... only evaluating actual program flow seems like a way to overlook things.
Disassembling a binary file is never needed, unless it is a file that you have not generated yourself, in which case this is indeed a part of reverse engineering.
If you have no debugging information for the file, you cannot know that some byte sequence is unreachable code.
To identify what such a sequence of bytes is, you can try to decode it in various ways, e.g. by disassembling it, by interpreting it as ASCII or Unicode characters, by interpreting it as a sequence of floating-point or integer numbers, etc. If anyone of these decoding attempts looks plausible, you can guess that it is correct. So if it disassembles into a sequence of plausible instructions, you can conclude that it is unreachable code, but your guess is seldom certain.
In any case, the disassembling may start only by executing the program and following the control flow, and also noting all memory addresses that are accessed by loads and stores.
If after that there remain memory regions that have not been accessed in any way, possibly because you could not simulate all the possible program inputs, e.g. because the program accessed a hardware dongle that you do not possess, then you can start to decode in various ways the regions with unknown content, attempting to guess their purpose.
Well, now I read a "disassembly instructions" in his comment, did I really miss it or he edited it?
It was probably me who misunderstood, and I was wrong anyway, apologies.
No, I was mistaken, I apologize
The point is you no longer need expertise as a prerequisite
That's a bad and wrong point. You don't need expertise to throw the disassembly output at the LLM either. The LLM itself could do this as a first step.
Doing it completely in the LLM would do nothing but add orders of magnitude less efficiency.
I wonder if you trained a LLM on the data that has passed through my router throughout my whole life how smart it would be.
I mean decrypted data of course, because it surely couldn't learn directly from encrypted data, right? Right?
Woah...
Oh no, how long until a Snow Crash prompt injection turns LLMs into babbling tools of the oligarchs? /s