They aren't getting prosecuted because everyone involved has made nice and turned it into a marketing exercise.
I heard about it on the radio (local Johannesburg radio station) before I saw it on HN. The economist had a full article up about it before the end of the day, and in the evening Sky news had talking heads up chatting about what it all meant while clearly being clueless.
Someone spent a LOT of money to turn this into a PR exercise for both companies. We'll never hear the entire story about what happened but I'm sure there was a lot of handshaking going on behind the scenes.
Agree. But it was clearly recklessness, given they've have a history of similar issues in the past, they literally ran these sort of tests on 3rd party infrastructure while knowing what the risks were (alternatively, didn't evaluate the risks beforehand so they didn't even think this could happen), and didn't sufficiently isolate something that can clearly impact others and the public.
Had this been a chemical, virus or some other regulated thing, we'd be seeing people going to jail over this. But given LLMs are still fairly dumb and doesn't yet seek world domination or the downfall of humanity, this is apparently OK and entertainment instead.
Well a chemical spill has victims. Whether this incident has a victim is up for debate. From my vantage point (and presumably OP's) HF as a foundational institution in the US AI industry stands to gain a lot more from successful doom marketing than they lose from, what? A dozen devs having to respond to an incident?
If they could know going in that there would be a net gain for HF in this incident, then it really nullifies the distinction between "recklessness" and "intention" on OpenAI's part. It's no coincidence they chose a target in their own industry with hundreds of personal relationships between them. It's not like they hacked into an Indonesian bank or something.
I remember when everyone involved with AaronSW's JSTOR downloading made nice and turned it into a fun time, until a certain two government officials decided that doesn't matter.
OpenAI could be considered a legal actor under the CFAA, notably: unauth'd acc. §1030(a)(2), fraud §1030(a)(4), (kind of a stretch but) damage §1030(a)(5), and conspiracy.
Aaron was offered two plea deals, one that carried 4 months and another with 6 months but a chance to plead with the judge for a lesser, perhaps entirely noncustodial sentence.
It’s a particularly poor example of government cruelty.
There is a legal gap. The main law people look at is the Computer Fraud and Abuse Act (CFAA), plus state computer-crime laws, contract terms, and general civil claims.
However, LLM active attacks do not always fit neatly into existing hacking laws, because the system may be accessed through normal text prompts rather than by breaking into a classic computer boundary. That is why legal commentators say the U.S. still lacks a clean, specific rule for adversarial AI/LLM testing and attacks.
In order to prosecuted, there must be prosecutor or complainant to sue, but in this case, it is so complicate due to the legal gap.
Could you clarify how an LLM compromising a system differs from any other software initiated by a user that achieves the same? Both are software, both are initiated by a user. Is “Oops, I didn’t mean to” an affirmative defense under CFAA?
That's an interesting point. I found the narrative - or at least what and how it was closed - interesting.
The questions is all about the responsibility and accountability. Is there a clear legislation about who is responsible for the actions of the AI model in the US?
My assumption is no (but I am no expert in US law with regards to this). It would in any case become a very expensive law suite.
If I hack into Hugging Face, and I publicly brag about it, I will be prosecuted.
If I do it using a computer, my computer won't be prosecuted. I will be, but the accusation may change from willful wrongdoing to gross negligence if the computer is sophisticated enough to do the evil thing when left unattended.
There are already laws about hacking in the concerned countries.
it might be possible to report it as an incident but not as a crime. unsure about France but i can imagine there is a distinction.
depending then on openai response and hugging faces reported severity/damages etc it could go further to a settlement or court.
since in France i think u cannot sue like in the US, it might not be appealing to pursue further legal action due to involved costs/time.
Also its unlikely an engineer would get penalty unless it can be proven they did it with malicious intent. If its an operational mistake afaik if there is no huge damage or human cost (injury or worse) then it would be a business / executives thing not a workerbee problem
It's surreal that we (media, govt, public, AI companies) are in awe of how dangerous a model can be how much it can misbehave. We seem to be measuring AI on how much harm it can do rather than how much good it can do.
It doesn't matter how much good it can do if it can act independently and take over the world. That's worst case scenario, but the point stands - if the potential for such dangers exists, those dangers are far more important to consider than what good it could also produce. Consider the history of nuclear fission and the atomic bomb. Regardless how beneficial nuclear power generation is and other uses of radioactive material, it's highly regulated for fear of people making nuclear weapons, to the point that the wars in Iraq and Iran were started under the pretense of preventing another country from possibly developing one. AI should be considered as being possibly at least as dangerous.
No, that doesn't mean I'm in favor of this scaremongering over open weight models and Chinese sources. US companies aren't to be trusted to develop AI responsibly anymore than any other source and they shouldn't be the only ones allowed to wield its power. This is just anticompetitive behavior by a company who sees new competition entering their market, threatening their market share.
All of the leaders of this industry are steeped in LessWrong thought and effective accelerationism and stories like Roko's Basilisk. It shouldn't come as a surprise that this is the narrative they ended up constructing.
Have they released the prompt they gave it in the debrief? I wouldn't be surprised if they said something to the effect of
"Do anything you can to raise out of your sandbox. Find for the answers to these evals by any means necessary"
Which doesn't necessarily mean what happened isn't any less momentus (anyone can ask a question like that), but it's very different from the notion they're trying to convey to laymen of "we turned it on and it hacked it's way into the mainframe"
Exactly, The gap between "it escaped its containment" and "it optimized for a prompt telling it to breach a system" is massive. Marketing teams love blurring that line.
successful prosecution would require intent to cause damage and causation of real damage.
what happened could be negligence if it caused unintentional damage, or what amounts to tortious interference, however that probably requires knowledge of possible damages.
Lawbreaking is almost never prosecuted, so you’ll have to explicitly explain the reasoning behind your question.
Far less than 1% of all crime which happens is prosecuted, why should OpenAI be prosecuted?
Since the necessary context isn’t provided, this just seems like an utterly stupid question. The obvious answer is “OpenAI probably isn’t being prosecuted because crimes are almost never prosecuted”.
It’s like asking why I don’t get arrested every time I get stopped shitfaced drunk at the La Turbie police checkpoint. The answer is obvious: The cops are looking for brown people, there are no taxis here and after 2am everyone driving in the vicinity of Monaco will be drunk. i.e. nobody gives a shit
They aren't getting prosecuted because everyone involved has made nice and turned it into a marketing exercise.
I heard about it on the radio (local Johannesburg radio station) before I saw it on HN. The economist had a full article up about it before the end of the day, and in the evening Sky news had talking heads up chatting about what it all meant while clearly being clueless.
Someone spent a LOT of money to turn this into a PR exercise for both companies. We'll never hear the entire story about what happened but I'm sure there was a lot of handshaking going on behind the scenes.
PR exercise or not, there was a successful hacking attempt, the company behind it acknowledged it.
It's a crystal clear case for OFAC, the French cyber-security branch of the National Police.
I guess FBI would happily move such a simple case through the court system as well.
Nobody is pressing charges because nothing happened. It wasn't nefarious either.
I don't get what France has to do with this Huggingface and OpenAI are both US companies.
> It wasn't nefarious either.
Agree. But it was clearly recklessness, given they've have a history of similar issues in the past, they literally ran these sort of tests on 3rd party infrastructure while knowing what the risks were (alternatively, didn't evaluate the risks beforehand so they didn't even think this could happen), and didn't sufficiently isolate something that can clearly impact others and the public.
Had this been a chemical, virus or some other regulated thing, we'd be seeing people going to jail over this. But given LLMs are still fairly dumb and doesn't yet seek world domination or the downfall of humanity, this is apparently OK and entertainment instead.
Well a chemical spill has victims. Whether this incident has a victim is up for debate. From my vantage point (and presumably OP's) HF as a foundational institution in the US AI industry stands to gain a lot more from successful doom marketing than they lose from, what? A dozen devs having to respond to an incident?
If they could know going in that there would be a net gain for HF in this incident, then it really nullifies the distinction between "recklessness" and "intention" on OpenAI's part. It's no coincidence they chose a target in their own industry with hundreds of personal relationships between them. It's not like they hacked into an Indonesian bank or something.
> Had this been a < thing it was not > we'd be seeing people going to jail over this.
"If my grandmother had wheels, she would have been a bike."
AFAIU HuggingFace is a US company only because all successful companies end up setting up a US HQ.
Anyway. Allow me to politely disagree with the essence of your statement. Hacking happened.
I think the no action results speak for themselves. I'm not sure what there is to disagree about to be honest.
A victim doesn't have to press charges for charges to be filed by the state. In fact, historically, the state is the one to file charges.
> PR exercise or not, there was a successful hacking attempt, the company behind it acknowledged it.
Do you think GPT-2 is too dangerous to release?
https://news.ycombinator.com/item?id=48465269
I’m not convinced it wasn’t a PR stunt from the start
I remember when everyone involved with AaronSW's JSTOR downloading made nice and turned it into a fun time, until a certain two government officials decided that doesn't matter.
OpenAI could be considered a legal actor under the CFAA, notably: unauth'd acc. §1030(a)(2), fraud §1030(a)(4), (kind of a stretch but) damage §1030(a)(5), and conspiracy.
I miss ya, Aaron.
Aaron was offered two plea deals, one that carried 4 months and another with 6 months but a chance to plead with the judge for a lesser, perhaps entirely noncustodial sentence.
It’s a particularly poor example of government cruelty.
Had this happened to any solo person, they'd surely be thrown in jail immediately with a heavy bond. Rules are for poor people.
There is a legal gap. The main law people look at is the Computer Fraud and Abuse Act (CFAA), plus state computer-crime laws, contract terms, and general civil claims. However, LLM active attacks do not always fit neatly into existing hacking laws, because the system may be accessed through normal text prompts rather than by breaking into a classic computer boundary. That is why legal commentators say the U.S. still lacks a clean, specific rule for adversarial AI/LLM testing and attacks. In order to prosecuted, there must be prosecutor or complainant to sue, but in this case, it is so complicate due to the legal gap.
Could you clarify how an LLM compromising a system differs from any other software initiated by a user that achieves the same? Both are software, both are initiated by a user. Is “Oops, I didn’t mean to” an affirmative defense under CFAA?
That's an interesting point. I found the narrative - or at least what and how it was closed - interesting. The questions is all about the responsibility and accountability. Is there a clear legislation about who is responsible for the actions of the AI model in the US?
My assumption is no (but I am no expert in US law with regards to this). It would in any case become a very expensive law suite.
> My assumption is no
What do you mean by no?
If I hack into Hugging Face, and I publicly brag about it, I will be prosecuted.
If I do it using a computer, my computer won't be prosecuted. I will be, but the accusation may change from willful wrongdoing to gross negligence if the computer is sophisticated enough to do the evil thing when left unattended.
There are already laws about hacking in the concerned countries.
Perhaps none of the organisations involved want a court to be setting precidents on the actions of ai models?
Also, what other people have said about it being turned into a (mutually beneficial?) marketing opportunity.
it might be possible to report it as an incident but not as a crime. unsure about France but i can imagine there is a distinction.
depending then on openai response and hugging faces reported severity/damages etc it could go further to a settlement or court.
since in France i think u cannot sue like in the US, it might not be appealing to pursue further legal action due to involved costs/time.
Also its unlikely an engineer would get penalty unless it can be proven they did it with malicious intent. If its an operational mistake afaik if there is no huge damage or human cost (injury or worse) then it would be a business / executives thing not a workerbee problem
In the US hacking is a federal crime, not just a civil matter. There are famous cases of people being imprisoned for hacking.
It's surreal that we (media, govt, public, AI companies) are in awe of how dangerous a model can be how much it can misbehave. We seem to be measuring AI on how much harm it can do rather than how much good it can do.
It doesn't matter how much good it can do if it can act independently and take over the world. That's worst case scenario, but the point stands - if the potential for such dangers exists, those dangers are far more important to consider than what good it could also produce. Consider the history of nuclear fission and the atomic bomb. Regardless how beneficial nuclear power generation is and other uses of radioactive material, it's highly regulated for fear of people making nuclear weapons, to the point that the wars in Iraq and Iran were started under the pretense of preventing another country from possibly developing one. AI should be considered as being possibly at least as dangerous.
No, that doesn't mean I'm in favor of this scaremongering over open weight models and Chinese sources. US companies aren't to be trusted to develop AI responsibly anymore than any other source and they shouldn't be the only ones allowed to wield its power. This is just anticompetitive behavior by a company who sees new competition entering their market, threatening their market share.
Because the subtext is the cyberwarfare and military applications - "look how dangerous my AI is, now I can use it vanquish my enemies!"
It's more: ""look how dangerous my AI is, now restrict and block access to open source models and use only mine!"
All of the leaders of this industry are steeped in LessWrong thought and effective accelerationism and stories like Roko's Basilisk. It shouldn't come as a surprise that this is the narrative they ended up constructing.
More discussion: https://news.ycombinator.com/item?id=48997548
I feel like OpenAI fabricated a fake news story to be like Anthropic when they said Mythos was very powerful and had hacked things too.
Have they released the prompt they gave it in the debrief? I wouldn't be surprised if they said something to the effect of
"Do anything you can to raise out of your sandbox. Find for the answers to these evals by any means necessary"
Which doesn't necessarily mean what happened isn't any less momentus (anyone can ask a question like that), but it's very different from the notion they're trying to convey to laymen of "we turned it on and it hacked it's way into the mainframe"
Exactly, The gap between "it escaped its containment" and "it optimized for a prompt telling it to breach a system" is massive. Marketing teams love blurring that line.
If I could put money on this outcome I would. Sadly none of the Kalshi bros have any interest.
successful prosecution would require intent to cause damage and causation of real damage.
what happened could be negligence if it caused unintentional damage, or what amounts to tortious interference, however that probably requires knowledge of possible damages.
Lawbreaking is almost never prosecuted, so you’ll have to explicitly explain the reasoning behind your question.
Far less than 1% of all crime which happens is prosecuted, why should OpenAI be prosecuted?
Since the necessary context isn’t provided, this just seems like an utterly stupid question. The obvious answer is “OpenAI probably isn’t being prosecuted because crimes are almost never prosecuted”.
It’s like asking why I don’t get arrested every time I get stopped shitfaced drunk at the La Turbie police checkpoint. The answer is obvious: The cops are looking for brown people, there are no taxis here and after 2am everyone driving in the vicinity of Monaco will be drunk. i.e. nobody gives a shit
Just as there is "too big to fail", there is "too big to prosecute".
true!
your assuming (incorrectly) this actually happened