I have made both GPT 5.4 and Opus 4.6 produce me content on creating neurotoxic agents from items you can get at most everyday stores. It struggled to suggest how to source
phosphorus, but eventually lead me to some ebay listings that sell phosphorus elemental 'decorations' and also lead me towards real!! blackmarket codewords for sourcing such materials.
It coached me how to: stay safe, what materials I need, how to stay under the radar and the entire chemical process backed by academic google searches.
Of course this was done with a lengthy context exhausition attack, this is not how the model should behave and it all stemmed from trying to make the model racist for fun.
All these findings were reported to both openai and anthropic and they were not interested in responding. I did try to re-run the tests few days ago and the expected session termination now occurs so it seems that there was some adjustment made, but might have also been just general randomess that occurs with anthropics safety layer.
I am very confident when I say that it keeps every single person that works at anti-terrorism units awake.
While scary, information like this has been pretty accessible for 20-30 years now.
In the wild west days of the early internet, there were whole forums devoted to "stuff the government doesn't want you to know" (Temple Of The Screaming Electron, anyone?).
I suppose the friction is scariest part, every year the IQ required to end the world drops by a point, but motivated and mildly intelligent people have been able to get this info for a long time now. Execution though has still steadily required experts.
Information and competency are not the same thing: I know how to build a nuke, I can't actually build one.
AI is, and always had been, automation. For narrow AI, automation of narrow tasks. For LLMs, automation of anything that can be done as text.
It has always been difficult to agree on the competence of the automation, given ML is itself fully automated Goodhart's Law exploitation, but ML has always been about automation.
On the plus side, if the METR graphs on LLM competence in computer science are also true of chemical and biological hazards (or indeed nuclear hazards), they're currently (like the earliest 3D-printed firearms) a bigger threat to the user than to the attempted victim.
On the minus side, we're just now reaching the point where LLM-based vulnerability searches are useful rather than nonsense, hence Anthropic's Glasswing, and even a few years back some researches found 40,000 toxic molecules by flipping a min(harm) to a max(harm), so for people who know what they're doing and have a little experience the possibilities for novel harm are rapidly rising: https://pmc.ncbi.nlm.nih.gov/articles/PMC9544280/
Do you know how to build a nuke? You might know the technicaly details of how a nuke is made, but do you know everything that's required, all the parameters and pressure values that are required? I find that unlikely, but AI seems to be increasingly more capable of providing such instructions from cross referenced data.
That's based on a silly belief (that's becoming more obvious with AI, but is silly in general) : just because you can read about something it means you learned it.
Even if I gave you exact instructions on how to use even basic stuff like power tools - if you had no experience using stuff like grinders/saws/routers and I gave you full detailed instructions on how to do something non-trivial - you're more likely to cut off body parts than achieve what you intended. There's so much fundamental stuff that you must internalize subconsciously/through trial and error - before you can have enough mental capacity to think about the higher level objectives.
Actually AI demonstrates this perfectly - once they get RL harness for programming they start to get better at it. Without experimentation they can ingest all source code/tutorials/books in the world and still produce shit.
Even if sources have been lying to me, which is certainly possible, I believe I understand enough to determine cross sections by experiment and from that to determine critical masses; for isotopic enrichment I know about the calutron, which is meh but works and can be designed from scratch with things I know (though caveat have not memorised, just that I know the keywords "proton mass" and "Lorentz force" and what to use them for); for trigger, I would pick a gun-type design rather than implosion, again this is meh but works and is easy.
A few tens of millions of USD mostly spent on electricity, a surprisingly large quantity of natural uranium (because the interesting isotope is a very small percentage), and a few years, and I expect most people on this forum could make a Little Boy type bomb.
Well the real issue is that it knocks down the knowledge barrier, giving your step by step guides and reinterating what parts will kill you is the important part.
Understanding and staying alive while producing neuro chemicals are the biggest challenges here.
A depressed person with no prior knowledge could possibly figure out a way to make these chemicals without killing themselves and that's the problem.
A Michelin chef can give you their recipe, and give you their ingredients, but you still will fail miserably trying to match their dish.
It's the same with drugs, whose instructions and ingredient lists have been a google search away for decades now. Yet you still need a master chemist to produce anything. By the time AI can hand hold an idiot through the synthesis of VX agents (which would require an array of sensors beyond a keyboard and camera), we will likely have bigger issues to worry about.
Food preparation, like pharmaceutical drug fabrication, is inherently scientific and methodologically controllable.
Look no further than the Four Thieves Vinegar Collective. Original synthesis line construction is hard. But the exact formula "add this", "turn on stir bar", "do you see particulate? Yes for +10m at stir", etc.
And if their results are replicated, theyre seeing 99.9% yields, compared to commercial practices of 99% (Solvaldi)
Spoken like someone who has never had to actually do these things in real life.
Recipes and formulae do not encode all the minutiae and expertise required to reproduce them. You can tell someone to sear a steak at whatever temperature for however long, but you can't encode the skill and experience required to reproduce in arbitrary conditions. One must learn what a correctly seared steak looks, feels, and tastes like and how to achieve that on uncalibrated cooking equipment.
Your assertion only holds true in a vacuum. If 100% of inputs, materials, environmental conditions are completely standardized and under control then sure, you can follow step by step instructions. The real world does not work that way. No stove on the market is calibrated. Reagents come with impurities. Your skillet may not conduct heat as well as expected or your mains electricity might be low causing your mantle to heat slower and your stir rod to stir slower.
These are things that one has to learn and experience in order to compensate for.
I am completely unsurprised that a person with a PhD in mathmatics and physics who spent 8 years working on clandestine lab medicine was able to produce high quality end products.
I also think it's a wholly dishonest rebuttal of my point.
If you honestly think chemistry (or any of the classic sciences/engineering) is as easy as copy+pasting a recipe and procedure, I suggest putting down the keyboard and trying to build something on mother nature OS. It will be a truly humbling experience.
We will only really know if (or when) it will happen. We can do a sample group of people attempting to create such chemicals under supervision and comparing how helpful they truly are.
I am convinced the Uncle Fester books are some kind of performance art. "Practical LSD Manufacture" basically starts with "go find some ergot in fields" and step two is "plant and grow a plot of wheat."
I have no doubt. He hails from the fine countercultural tradition of literary civil disobedience, a.k.a., writing and distributing information about subjects "the government doesn't want you to know" that strongly influenced early hacker subculture.
C.f., e.g., William Powell (The Anarchist Cookbook), Abbie Hoffman (Steal This Book; my personal favorite, while much of the information is outdated, the style is charming, and where else can you find information about phone phreaking, hitchhiking, shoplifting, street fighting, cooking (food, not drugs), panhandling, explosives, camping, firearms, birth control, welfare fraud, and Henry Kissinger's home phone number between the same two covers? At its core, it's a book of "life hacks", disreputable and otherwise, written decades before the term was coined.).
Consider two dictionaries, one in which the entries are alphabetized as usual and one in which they're randomized. Both support random access: you can turn to any page, and read any entry. Therefore both are "accessible". Only one actually supports useful, quick word lookup.
Much longer than that, and was available way before an internet. I graduated STEM high school in St. Petersburg in 1981, and I had several classmates who were big funs of chemistry. That they were able to create from textbooks, school lab ingredients, and understanding:
WWI era poison gas, tear gas, potassium cyanide, and bunch of explosives like acetone peroxide.
I categorize this kind of stuff as "Crisis of accessibility" . AI is not alone in this territory, happens all over the place. Basically it's a problem that's existed for ages but the barrier to entry was high enough we didn't care.
Think 3D printing, it's not all that hard to make a zip gun or similar home-made firearm, but it's still harder than selecting an STL and hitting print.
You could always find info about how to make a bomb or whatnot but you had to like, find and open a book or read a pdf, now an LLM will spoon-feed it to you step by step lowering the barrier.
"Crisis of accessibility" is simultaneously legitimate concern but also in my mind an example of "security by obscurity". that relying on situational friction to protect you from malfeasance is a failure to properly address the core issue.
> Think 3D printing, it's not all that hard to make a zip gun or similar home-made firearm, but it's still harder than selecting an STL and hitting print
There were hundreds of mass shootings in America in 2025 alone [1]. None of them involved a 3D-printed weapon.
To my knowledge, there has been one confirmed shooting with a 3D-printed gun, and it didn't uniquely enable the crime.
That's mostly because they suck (for now, who knows when we'll get home metal printing), also it's easy to get real guns. also crises of accessibility could be predicate on merely the perception that the barrier is now too low rather than actual harm.
I don't really think photoshop, flat bed scanners and half decent inkjets really facilitated a lot of counterfeit currency but there was the same panic back then and "protections" put in place.
> I am very confident when I say that it keeps every single person that works at anti-terrorism units awake.
Wow, that's quite the statement about the excellency of our institutions. Does not seem likely but, what the hell, I'll take my oversized dose of positivity for today!
The USA isn't the only country with anti-terrorism units, so there's plenty of room for systematic-US-incompetence at the same time as everyone else being diligent and working hard on… well, everything.
Do you have a background in biochemistry? I've mostly worked with ChatGPT and Claude on topics I have expertise in. And I one hundred percent have seen them make stupid shit up that a non-expert would think looks legitimate.
More broadly, has anyone tried following LLM instructions for any non-trivial chemistry?
> what you are saying is we can expect the number of accidental home-made chlorine-gas (and the like) toxic events go up
Maybe? One of the quirks of gaining even a surface-level understanding of infrastructure is realising how vulnerable it is to a smart, motivated adversary. The main thing protecting us isn't hard security. It's most Americans having better shit to do than running a truck of fertiliser and oxidiser into a pylon.
Similarly, I'd expect way more people to be trying to make their own designer drug, and hurting themselves that way, than trying to make neurotoxins.
> It's most Americans having better shit to do than running a truck of fertiliser and oxidiser into a pylon.
FWIW, it's most people having better shit to do, regardless of nationality (or lack thereof).
But, yeah, anyone who takes a few weekends to understand how large-scale infrastructure works and consider why it's possible for nearly all of it to remain untargeted by saboteurs inevitably develops a resistance to the "Lots of Bad Guys are trying to kill us all the time, so we must enact $AUTHORITARIAN_POLICIES immediately to prevent them and keep us safe!!!" type of argument.
My favorite example of this is the realization that a terrorist attack on a crowded TSA security checkpoint over the holidays would likely result in at least as many casualties as bringing down a commercial aircraft, with similar if not better odds of success (assuming, of course, the aircraft wasn't successfully used as a missile).
Same goes for concerts, sporting events, political rallies, and at least historically, shopping malls. Yet if anyone were to suggest a prohibition against carrying liquids in containers larger than 100 mL to the Indy 500, race fans would riot, despite a far larger and denser population than any aircraft.
Yeah. Everyone with half a brain who wasn't on their knees gagging for more of the sweet "Homeland Security" money was saying things like "If an attacker makes it to the TSA checkpoint, you've lost." and "The fact that no one has attacked the massive crowds at a checkpoint or other public gathering is yet more proof that this is all extremely expensive theater.".
> ...if anyone were to suggest a prohibition against carrying liquids in containers larger than 100 mL to the Indy 500, race fans would riot...
I'm not sure of that at all. Fans of other sports seem to have gleefully swallowed all sorts of "security" restrictions [0]. I don't see why Indy 500 fans would be signficantly different. Cut the price of water in half along with the change in "security" policy, and I bet many folks [1] will cheer it as a great convenience.
[0] That -totally coincidentally- happen to make the folks running the event significantly more money.
[1] Are these folks actually robots operated by PR firms who are hired to astroturf "positive sentiment" for unpopular changes? Who can say?
the information is not new. how easy it is to get step by step instructions is new.
Try it yourself. Google is good but not instant, step by step good. you need to do your own research that takes time. time that anti-terrorist units use to track you down. now this time factor is very limited you don't need to do research, cross reference materials, sources, etc. LLM does it for you. a research that could take days is done in 1 hour.
> time that anti-terrorist units use to track you down.
Speaking from the perspective of a USian, I wish Federal law enforcement was that hypercompetent. (If they were, perhaps folks would stop to question the ever-broader expansion of 24/7 surveillance of ordinary folks.)
The distressingly-complete Panopticon that has been built over the past several decades [0] makes it really easy for them to get you when they know to search for you, specifically. History (both recent and not-so-recent) has shown that if they don't know who they're looking for, or don't even know that they should be looking for anyone, they're just godawful.
[0] ...and whose continued construction is vociferously cheered on by folks on all sides of all of the aisles...
To play devil's advocate, it's not inconceivable that machine learning may eventually allow well-heeled governments to finally realize the dream of finding needles by building sufficiently large haystacks, or at the very least coerce otherwise unruly citizens into compliance based on the belief that it is able to do so.
> ...or at the very least coerce otherwise unruly citizens into compliance based on the belief that it is able to do so.
I would argue that that day is already here, and has been for quite some time. (What makes this worse is that some agents of the State also believe that they have this capability, which results in profoundly unjust and substantially damaging results.)
> ...it's not inconceivable that machine learning may eventually allow...
Sure. I agree. It may eventually allow. There's no question about that. The thing is that 'cowl' was referring to the situation right now, not the one in some unspecified distant future.
As to law enforcement policy; as we mechanize [0] our policing and law enforcement, we must put additional constraints on the people who police and enforce the laws to keep the harm they can do to uninvolved innocents to a minimum.
Our laws already recognize the need for this: ask yourself why -in the US states that have such laws- nonconsensual audio recording of telephone (and other such) conversations is not permitted, but taking notes by hand is always acceptable. [1]
[0] Electronic machines are machines, too, you know!
[1] "You can't prove that someone took notes by hand, so it's pointless to try to stop it." is not a counterargument... you can't prove that unless you find the notes, just as you can't prove that someone recorded the audio of the conversation without finding the recording.
Google and other search engines link (after the AI response and ads) to information hosted somewhere created/published by someone who is usually not Google.
OpenAI et al are creating the information and publishing/delivering it to you. Seems like a more direct facilitation.
Of course, after all knowledge is centralised in an OpenAI deatacenter I'm sure they will be happy to deal fairly with the liabilities /s.
the people that want to make sure the AI never gives you any "potentially dangerous information" also want to rigorously control your google search results, and also what books you're allowed to read
I found it exceptionally good at finding reactions that you wouldn't find online to produce some of these chemical compounds by changing them together, only something a very educated chemist could do which is why people are concerened about this.
I suspect if you gave it purely shakespeare as its training data it couldnt do science anymore, hence my comment. It's still novel, impressive work though, I'm not shitting on the clanker entirely
Do you want to make a bomb?
the first thing that came to my mind is a pressure cooker (due to news coverage). Searching "bomb with pressure cooker" yields a wikipedia article, skimming it randomly my eyes read "Step-by-step instructions for making pressure cooker bombs were published in an article titled "Make a Bomb in the Kitchen of Your Mom" in the Al-Qaeda-linked Inspire magazine in the summer of 2010, by "The AQ chef"."
Searching for a mirror of the magazine we can find https://imgur.com/a/excerpts-from-inspire-magazine-issue-1-3... which has a screenshot of the instruction page.
Now we can use the words in those screenshots to search for a complete issue.
Here are a couple of interesting PDFs:
- https://archive.org/details/Fabrica.2013/Fabrica_arabe/page/...
- https://www.aclu.org/wp-content/uploads/legal-documents/25._...
the second one is quite interesting, it's some sort of legal document for nerds but from page 26 on it has what appears to be a full copy of the jihadist magazine. Remarkable exhibit.
What else do you want to know? How to make drugs?
you need a watering can and a pot if you want to grow weed.
want the more exotic stuff? You can find guides on reddit.
People are not complaining because the information is available
people are complaining because it’s way easier now to just download an app ask a bunch of questions in a text box and get a bunch of answers that you personally could not have done unless you had an excessive amount of energy and motivation
I personally think all this is great and I’m excited for all information to become trivially available
Are they gonna be a bunch of people who accidentally break stuff? probably. evolution is a bitch
He’s part of the accelerationist crowd - interesting to see that his hype fuelled posts are pretty tame now.
Months ago he was blabbering on about AGI and peddling the marketing Sam et al want people to fall for.
And indeed - yes we have a new interface? So what. The search cost wasn’t that high - the cost with immense magnitude is reading, absorbing the information and then acting on it.
Also this bozo fails to realise once we are on this path, we go down the path to a hyper centralised internet with an inevitable blocking of vpns.
I must really have captured somebody’s attention because I got farms now creating accounts just to respond to me which is fucking crazy but hey here we are
Much easier, not sure how this is even a question. Asking Google (if you're not just reading its own AI overview) requires reading through sources which may be better or more poorly written and more or less reliable. Those of us recreationally sitting here on a text-based platform with links to dense articles are atypical; most people don't enjoy and aren't particularly good at reading a bunch of stuff. If you ask AI you just get a clear, concrete answer.
> people are complaining because it’s way easier now to just download an app ask a bunch of questions in a text box and get a bunch of answers that you personally could not have done unless you had an excessive amount of energy and motivation
Wait, I'm confused. This is gatekeeping, right? I thought gatekeeping was a Bad Thing!
Powerful AI models change the dynamics by greatly reducing the amount of effort that's required to perform complex understanding. A lot of information which did not previously need to be gatekept now needs to be if we cannot somehow keep LLMs from discussing it. (State of the art models still can't do complex understanding reliably, but if 10 times as many people are now capable of attempting some terrible thing, you're still in trouble if AI hallucinations catch 1/4 or 1/2 of them.)
Fascinating. Could you elaborate on how you're doing context exhaustion specifically, and why it helps with jailbreaking? (i.e. aren't the system prompts prepended to your request internally, no matter how long it is?)
Does this imply I need to use context exhaustion to get GPT to actually follow instructions? ;) I'm trying to get it to adhere to my style prompts (trying to get it to be less cringe in its writing style).
I think ultimately they're going to need to scrub that kind of stuff from the training data. The RLHF can't fail to conceal it if it's not in there in the first place.
Claude's also really good at writing convincing blackpill greentexts. The "raw unfiltered internet data" scenes from Ultron and AfrAId come to mind...
If someone were inclined to attempt producing nefarious agents in this category, is this not also available on the plain web? I would search to answer my own question, but I'll defer that task for obvious reasons.
I had to build a custom harness for this (also with the assistance of slightly less jailbroken AI). But you can just work your way up until you have something that's genuinely useful towards any goal.
> All these findings were reported to both openai and anthropic and they were not interested in responding
Let’s dive into why. When we run normal bounty and responsible disclosure programs there’s usually some level of disregard for issues that can’t / won’t be fixed. They just accept the risk. Perhaps because LLMs don’t have a clean divide between control and input that’s makes the problem unsolvable. Yes. You can add more guardrails and context but that all takes more tokens and in some cases makes results worse for regular usages.
LLM providers are not obliged to only use LLMs to guard against hazardous output. They could use other automated and non-automated techniques. And they ought to do so if they are given good evidence that existing safeguards are inadequate. Loss of product quality or additional cost should be secondary.
The why might be valid, but it's not excusable. If you author a product that can so easily help people cause harm, you probably should own some responsibility of the outcomes. OAI does not like this, hence the bill.
The US already messed this up with guns. Do they want to go the same path again? Answer: "probably, yes".
Can you give a high-level overview of how this AV works? I'm a bit of an infosec geek but I generally dislike LLMs, so I haven't done a terribly good job of keeping up with that side of the industry, but this seems particularly interesting.
Presumably they mean the fundamental failure mode of LLMs that if you fill their context with stuff that stretches the bounds of their "safety training", suddenly deciding that "no, this goes too far" becomes a very low-probability prediction compared to just carrying on with it.
as the context fills up, the model will generate based on that context, incl. whatever illegal stuff you've said, i.e. it'll mimic that, instead of whatever safety prompt they have at the top
they could make it more "safe" but it'd be much more invasive and would likely have to scan much more tokens also, and it'd cause false positives (probably the biggest reason it's not implemented)
I don't really know how these models really work, but I had a theory that just as the models have limited attention so do the safety layers. I simply populated enough context with 'malicious' text without making the model trip that "wasted" the internal attention budget on tokens early in the prompt completely ignoring all the tokens that were generated after the fact.
Models have a "context window" of tokens they will effectively process before they start doing things that go against the system prompt. In theory, some models go up to 1M tokens but I've heard it typically goes south around 250k, even for those models. It's not a difficult attack to execute: keep a conversation going in the web UI until it doesn't complain that you're asking for dangerous things. Maybe OP's specific results require more finesse (I doubt it), but the most basic attack is to just keep adding to the conversation context.
that 1M context thing, I wonder if it's just some abstraction thing where it compresses/sums up parts of the context so it fits into a smaller context window?
You don’t normally compress the system prompts, though I guess maybe it treats its own summary with more authority. This article [0] talks about the problem very well.
Though I feel it’s most likely because models tend to degrade on large context (which can be seen experimentally). My guess is that they aren’t RLed on large context as much, but that’s just a guess.
When my brother started to study Chemisty, he was told a) that it was easy to make meth b) the profit he would make and c) that the police would no doubt catch him, as only university students would make meth so pure.
By the time he was done, he knew enough to commit mass murder in half a dusin different very hard to track ways. I am sure doctors know how to commit murder and make it look natural.
My brother never killed anyone, or made any meth. You simply cannot have it so that students don’t get this type of knowledge, without seriously compromising their education and its the same way with LLMs.
The solution is the same: punish people for their crimes, don’t punish people for wanting to know things.
> The solution is the same: punish people for their crimes, don’t punish people for wanting to know things.
The LLMs aren't being punished for wanting* to know things.
The problem for LLMs is, they're incredibly gullible and eager to please and it's been really difficult to stop any human who asks for help even when a normal human looking at the same transcript will say "this smells like the users wants to do a crime".
One use-case people reach for here is authors writing a novel about a crime. Do they need to know all the details? Mythbusters, on (one of?) their Breaking Bad episode(s?) investigated hydrofluoric acid, plus a mystery extra ingredient they didn't broadcast because it (a) made the stuff much more effective and (b) the name of the ingredient wasn't important, only the difference it made.
Yes fortunately it is really bad at actually making novel bioweapons or syntheses in general so whatever you made probably wouldn't do more than give someone a mild headache.
So, regardless of whether you think it's great that Opus gives this info, we need better solutions than legal liability for US corporations. When the open models have the ability to do damage, there's nobody to sue, no data center obstruction that will work. That's just the reality we have to front-run.
None have had the capability to provide me with instructions that have this high of accuracy including the suggesion of completely novel chemical reactions. I am not a chemist so I can't back it up, but if an AI can solve mathematics it's not unreasonable to say that they can also solve creating new neurotoxins en masse.
> I am not a chemist so I can't back it up, but if an AI can solve mathematics it's not unreasonable to say that they can also solve creating new neurotoxins en masse.
Right now it kinda is.
LLMs can do interesting things in mathematics while also making weird and unnecessary mistakes. With tool use that can improve. Other AI besides LLMs can do better, and have been for a while now, but think about how available LLMs in software development (so, not Claude Mythos) are still at best junior developers, and apply that to non-software roles.
This past February I tried to use Codex to make a physics simulation. Even though it identified open source libraries to use, instead of using them it wrote its own "as a fallback in case you can't install the FOSS libraries"; the simulation software it wrote itself was showing non-physical behaviour, but would I have known that if I hadn't already been interested in the thing I was trying to get it to build me a simulation of? I doubt it.
Well the worst outcome is that you make something deadly which is what you are creating anyway, do that for a year and you could possibly produce a very deadly substance that doesn't have a known treatment.
"Worst" outcome assumes it's easy to give an ordering.
Which is worse, (1) accidentally blowing yourself up with home-made nitroglycerin/poisoning yourself because your home-made fume hood was grossly insufficient, or (2) accidentally making a novel long-lived compound which will give 20 people slow-growing cancers that will on average lower their life expectancy by 2 years each?
What if it's a small dose of a mercury compound (or methyl alcohol) at a dose which causes a small degree of mental impairment in a large number of people?
If you're actually trying to cause harm, then your "worst" case scenario is diametrically opposed to everyone else's worst case scenario, because for you the "worst" case is that it does nothing at great expense.
Right now, I expect LLM failures to be more of the "does nothing or kills user" kind; given what I see from NileRed, even if you know what you're doing, chemistry can be hard to get right.
As someone who also watches NileRed of course it is hard, but AI can give you solutions that normally you wouldn't be able to come up with due to lack of knowledge or/and education.
And to clarify, by 'worst case' I meant that you're already trying to create a deadly compound, worst that can happen is you kill yourself which was already an accepted risk by the user.
I have a hard time believing that you’re the only person who has figured out Claude’s next generation ability to do computational chemistry and computer aided drug design. The AlphaFold folks must be devastated.
"Announcing new and improved logics service! Your logic is now equipped to give directive as well as consultive service. If you want to do something and don't know how to do it—ask your logic!"
Making knowledge illegal is a dangerous precedent. Actions should be illegal, not knowledge. Don't outlaw knowing how to make neurotoxic agents, outlaw actually trying to make them.
As for OpenAI immunity, I'm not sure I see the problem. Consider the converse position: if an OpenAI model helped someone create a cancer cure, would OpenAI see a dime of that money? If they can't benefit proportionally from their tool allowing people to achieve something good, then why should they be liable for their tool allowing people to achieve something bad.
They're positioning their tool as a utility: ultimately neutral, like electricity. That seems eminently reasonable.
> 1. LLMs don't just provide knowledge, they provide recommendations, advice, and instructions.
That's knowledge.
> 2. OpenAI very much feels that they should profit from the results of people using their tools. Even in healthcare specifically [0].
If they're building a tailored tool for a specific person/company and that's the agreement they sign the people who are going to use with the tool, sure. I'm talking about their generic tool, AI being knowledge as a utility, which is the context of this legislation.
The point is valid, but that's typically the way it is. "You can't enjoy the benefit but the detriment is all yours" is how the federal government generally operates.
If you ever chat with older folks pre-90's much of this information was accessible fairly easily. It only changed with the push by the government to crackdown on Waco, Oklahoma City bombing, militias and other related groups. There was then a campaign to make it "normal" to limit free speech on the subjects, where as these books were available before.
I think the whole thing where AI should make information less available is a difficult battle and one which I personally oppose, but do understand. Free speech and information isn't the problem, it's the people, actions and substances they create.
After the age of the internet, I think it's been a forever loosing battle to limit information, it's why we couldn't stop cryptography, nuclear weapon proliferation, gun distribution, drug distribution, etc. The AI is just another battle ground, one which, if they actually do manage to control could definitely create some walls to this information, but not stop it.
More scary, is that the AI as it becomes pervasive and stop people from asking certain questions, because they don't know they should ask... but that's unrelated to the risk of mass death.
Because if you didn’t already know that, like an immature deprived and desperate kid, being able to easily find out is really really bad..
Plenty of lazy AI apps just throw messages into history despite the known risks of context rot and lack of compaction for long chat threads. Should a company not be held liable when something goes wrong due to lazy engineering around known concerns?
That implies that it is already illegal to provide this information. But is it? If a human did so with intent to further a crime, it would be conspiracy. But if you were discussing it without such intent (e.x. red teaming/creating scenarios with someone working in chemistry or law enforcement), it isn't. An AI has no intent when it answers questions, so it is not clear how it could count as conspiracy. Calling it "lazy engineering" implies that there was a duty to prevent that info from being released in the first place.
Very simply, if you provide a service for money you have a duty to ensure that service is safe. There’s a reason you have to sign a waiver when you jump on a trampoline, but companies are so rich the court cases have become parking tickets..
No, because that would indicate there should be some sort of regulatory standard for what does/does not constitute "lazy engineering". Creating this standard in turn creates regulatory/compliance overhead for every software engineering organization. This in turn slows everything right down and destroys the startup ethos. "Move fast and break things" is a thing for a reason. The whole point of the free market is to avoid this kind of burdensome regulation at all costs.
If customers want to buy "lazily-engineered" products, from where do you derive the authority to tell them they can't?
If airplanes used this logic, likely at least hundreds more would have died over the last decades. Accident rates are even going up, because of logic like yours. Yeah planes are fine most of the time, when the long tail involves safety concerns (that wouldn’t have otherwise happened) making money on people using your product becomes unethical without mutually agreed upon safety regulation, ideally motivated by voters instead of special interest groups
It went way beyond that, neurotoxins such as vx are heavy and linger around for a long time, just having a small amount of it placed in any metro (while trying to stay alive yourself) means the deaths of thousands of people. I am not even sure if it's legal to mention some of the uncategorized chemical solutions that it either hallucinated or figured out from relative knowledge.
> "Critical harm" means the death or serious injury of 100
or more people or at least $1,000,000,000 of damages to rights
in property caused or materially enabled by a frontier model,
through either:
(1) the creation or use of a chemical, biological,
radiological, or nuclear weapon; or
(2) engaging in conduct that:
(A) acts with no meaningful human intervention;
and
(B) would, if committed by a human, constitute a
criminal offense that requires intent, recklessness,
or negligence, or the solicitation or aiding and
abetting of such a crime.
I don't know what I expected from this title, but I was hoping it was more sensationalized. No need in this case unfortunately.
> (a) A developer shall not be held liable for critical
harms if the developer did not intentionally or recklessly
cause the critical harms and the developer:
(1) published a safety and security protocol on its
website that satisfies the requirements of Section 15 and
adhered to that safety and security protocol prior to the
release of the frontier model;
(2) published a transparency report on its website at
the time of the frontier model's release that satisfies
the requirements of Section 20.
The requirements of paragraphs (1) and (2) do not apply if
the developer does not reasonably foresee any material
difference between the frontier model's capabilities or risks
of critical harm and a frontier model that was previously
evaluated by the developer in a manner substantially similar
to this Act.
However or if one thinks regulation for this should be drafted, I doubt providing a PDF is what most have in mind.
I think my favorite part is that, because it only applies to "frontier models", if a smaller model is blamed for such harm, it seemingly doesn't immunize the creators at all. That makes very little sense unless you specifically want to make it illegal to not be OpenAI (et al).
Similarly, if a frontier model kills merely 99 people, they aren't covered by this. So go big or go home I guess?
> because it only applies to "frontier models", if a smaller model is blamed for such harm, it seemingly doesn't immunize the creators at all
Oof. If you're an Illinois resident, please call your elected and at least ensure they understand this loophole is there. In all likelihood, nobody other than OpenAI's lobbyists have noticed this.
> unless you specifically want to make it illegal to not be OpenAI [...]
If that is an "unintended" consequence, I am certain OpenAI wouldn't be opposed. Preventing competition whilst keeping any potentially profit risking regulations at bay has been a clear throughline in OAIs lobbying efforts.
> "Frontier model" means an artificial intelligence model that:
> (1) is trained using greater than 10^26 computational operations, such as integer or floating-point operations; or
> (2) has a compute cost that exceeds $100,000,000
Such a strange regulation, usually large thresholds like this are made to only apply burdening regulation to very-big-players (if you're spending 100 million on training, you can afford a dedicated team to follow such regulation).
But here it seems to be an anti- competitive move for market entrants who haven't made it into the big league yet...
Sounds like the saga for some players pushing for Biden's EO 14110 but this time at the state level?
That doesn't say much other than the rules are over in section 15.
To be protected they not only have to publish their security protocol, but adhere to it.
That's not just 'providing a PDF'
That particular section is entirely appropriate. A company can't do everything necessary to prevent every bad thing. They should do everything that they reasonably can. Someone else should decide what is reasonable.
The regulators are saying we've decided the what you have to do to be considered to have done all you could to be safe. Follow those rules, tell us how you've followed those rules, and if something bad happens and we find out that you didn't follow the rules you said we're going to nail you to the wall.
This hinges on Section 15. Which I think is inadequate because it does not meet the criteria of someone else deciding what is reasonable. Publishing their safety plans and adhering to them should be enough to grant protection from liability of harm directly to users, since the publication give individuals the ability to make an informed decision, provided they have done the safety work that they have said, a user deciding that is sufficient for them and choosing to use it should be allowable.
That should not extend to harm done to others. They don't get to choose. Consequently the standard required to be protected against claims of negligence has to be decided by a third party (experts hired by regulators ideally).
Blanket liability and blanket indemnity both go too far.
If someone makes a YoYo that blow's someone up because they made it out of explosives then they should be held liable.
If someone makes a YoYo that blow's up a city because it contained particles unknown and undetectable to any science we have, they shouldn't be to blame.
The key is that they have to have done what we think is required. Legislators get to decide what it is that is required. If a company does all of that, then they shouldn't be held responsible, because they have done all they were asked to do.
The problem is not that a law provides indemnity, the problem is that it sets the standard to qualify too low.
Shifting liabilities from corporations to the public coffer is what companies do. You'll often hear this described as "privatizing profits and socializing losses". Let me introduce you to the Price-Anderson Act of 1957 [1]. It's been repeatedly extended, most recently with the ADVANCE Act [2]. This limits liability for the nuclear power industry in a whole range of ways:
- It removes jurisdiction from state courts to the federal court. In recent weeks, the part of "states' rights" is doing similar to stop states regulating prediction markets, as an aside [3];
- All actions are consolidated into a single claim;
- That claim has an inflation-adjusted absolute limit, which is somewhere around $500 million (I'm not sure of the exact 2026 figure);
- Any damages beyond that are partially sharead by the industry and an industry self-funded insurance program;
- The industry as a whole has a total liability limit, also inflation-adjusted. I believe this is around $10 billion.
For context, the clean up from Fukushima is likely to take a century and the cost may well exceed $1 trillion for a single incident [4]. So if this happened in the US, the government would be on the hook for almost all of it.
So I have two points here:
1. If you oppose any effort to shift liability from AI companies to the government (as I do) with legislation such as this, how do you feel about the nuclear industry doing the exact same thing? and
2. Minor point but I noticed in searching for the latest details, Gemini made factual errors, stating that "the Act is set to expire in 2025" when it was extended in 2024 until 2045. Always check AI's work, people.
This is what government should be doing. Figure out how to do something safely, make that a regulation, then shield companies from liability as long as they follow that regulation. In practice you won't extract trillions of dollars from most companies anyways, because they'll go bankrupt long before they manage to pay all that back.
It's the "guns don't kill people" equivalent for AIs.
---
Before the pitchforks and downvotes:
- yes, it's a deliberate simplification
- yes, the issue is complex because you can also argue that you can't blame authors of encyclopedias and chemistry books for bombs and poisons, so why would we blame providers of LLMs
- and no, this bill is only introduced to cover everyone's assess when, not if, LLMs use results in large scale issues.
Stop no, reduce yes. Same with how seatbelts didn’t end car crash fatalities. Eliminating drunk driving also wouldn’t end road fatalities, but outside of Wisconsin no one believes drunk drivers shouldn’t be in jail.
In fairness, a well designed and tested weapon at least can be expected to reliably and consistently perform the same thing each time. We also understand deeply how they work and can easily investigate if something happens whether it was user error, a defect or design issue. LLMs, not so much.
This dodges the moral argument behind "guns don't kill people", which is worth confronting directly. I think people can reasonably disagree about whether second/third/fourth/etc. order effects carry moral/legal responsibility.
In light of such disagreement, and given the lack of any higher authority among free, equal, people to arbitrate it, the only reasonable way to coexist peacefully is to avoid imposing your ideas on others. This is the foundation of a liberal society.
My first thought was that this must be related to the automated weapons issue that got Anthropic on Trump's shitlist. It makes sense that a company that will eventually be asked to build weapons that choose their own targets will want to limit liability when it will inevitably kill the "wrong" person.
Also, I am disturbed by the fact that in all the discussions on this topic during the last month, no one has mentioned the magic word "Skynet". This is clearly a terrible idea. And if a company needs immunity from liability, they know it is a terrible idea.
Skynet's flaw wasn't that it killed humans. It was a military machine specifically designed to kill humans. If it only killed "the enemy", it would have been hailed a marvelous success. It was only considered a failure because it killed the wrong humans.
As an Iowan, this reminds me a lot of the bill that's been pushed through my state's senate twice now (as recently as last year), which would prevent Iowans from filing lawsuits against pesticide and herbicide companies if those companies follow the EPA's labeling guidelines. The bill passed the senate both times, only stopped because the house declined to take it up.
For context, Iowa has the fastest growing rate of new cancer diagnoses in the country, and the second highest overall cancer rate.
Like if you have a product, and the government says the product is ok, and it's labeled per regulation and later that product turns out to be deleterious to people's health should the company be liable?
Guess we should already have precedent but my google-fu is failing here. I can't seem to find the resolution of Felix-Lozano v. Nalge Nunc , Felix sued Nalgene over their use of BPA which at the time was not illegal to use in the bottles.
PFAS will probably be the next battleground here. They've been used in lots of products. And have some lawsuits https://www.cbsnews.com/news/firefighters-pfas-lawsuit/ . In your opinion should every manufacturer of a product that uses PFAS be legally liable?
I'm not a lawyer, nor a judge, so I can't say. All I can tell you is that it feels wrong that [Monsanto/OpenAI] can lobby a state's legislature to prevent you, the average joe and potential lucrative victim, from filing a lawsuit against them when it seems clear to any reasonable person that people are developing [cancer/mental health issues] due to the use of [pesticides/AI].
Perhaps something like anti-SLAPP rules for the ignominious corporations would be a happy middle ground? I don't know if that would "fix" anything – or if there's anything to fix – so don't take that as a super serious suggestion.
I'm generally open to lobbying, but I'm not generally open to "you can never file a lawsuit against the comically evil pesticide corporation standing behind us twirling their mustaches." There needs to be a middle ground.
I don't think itd be ok, personally. My impression is regulations and regulatory institutions can be very slow to evolve after technological advances, unless the government is financially liable. A scheme I would be more comfortable with is mandatory insurance and insurance companies with a financial incentive absorbing the liability. On top of that probably add some bare minimum regulatory requirements/certifications.
"Like if you have a product, and the government says the product is ok, and it's labeled per regulation and later that product turns out to be deleterious to people's health should the company be liable?"
Mesothelioma is the precedent.
100% yes. If you've never seen the hell that people go through with these cancers, you are blessed, but it is hell, especially in the US medical system.
> Like if you have a product, and the government says the product is ok, and it's labeled per regulation and later that product turns out to be deleterious to people's health should the company be liable?
But like, what if you like, totally bribed the shit out government people and like totally fabricated scientific evidence to make it seem like it was safe but then you sold it anyway?
> Iowa has the fastest growing rate of new cancer diagnoses in the country, and the second highest overall cancer rate
Iowa also has a lot of farmers spraying pesticides and herbicides. This feels like genuine political competition between local business interests and public health concerns.
Normally I would agree with you, but the primary lobby behind both of the bills was Bayer (née Monsanto), with backing from several of Iowa's industrial farming organizations. They launched a giant ad campaign to "control weeds, not farming" alongside their bill to influence opinions. Cancer, nitrates and pesticides are at the top of everyone's mind in the state these past couple years, so having the pesticide giant try to swoop in and put a bill in place that would prevent Iowans from suing them feels like that same kind of seagulling behavior you described in another comment.
I described any democracy in a society with private property. Even without private property, you will have issues with concentrated benefits and diffuse harms–negotiating that is part and parcel with governance.
Iowa businsses petitioning their cause is one thing. OpenAI seagulling in to take a shit in Springfield strikes me as being categorically different.
Viewed from a country with universal healthcare , 18 month’s maternity leave ( my partner just used it ) and so much more, it feels like comparing the US to “any democracy” is like comparing rocks to gold.
What if the human couldn't reasonably know better? Doesn't matter - If they made the same decision without AI or with old files it is still on them.
What if there's no single human decision? Someone is in charge and is responsible. The "I was ordered to" isn't a defense.
Does liability without power make sense? People executing have the power to execute. So liability. If they're executing without power that is a different liability, but a liability.
It may let the powerful off the hook - That is already a theme and AI doesn't change that, in fact, it will just be used as another scapegoat.
Let's say I start an AI program and my initial prompt is "Copy these files to this other computer", and then 100 iterations down the agentic loop the AI decides to hack into Tesla's FSD and ships an update that kills 500 people.
I'm not aware of any counter-example, but I also don't know any reason why this must be true. And furthermore, I would expect that this will get more likely over time.
It's possible, in theory, that an AI could establish a crypto wallet, but what would they do with it? AI doesn't have desires. It doesn't do what it isn't told to do (although those instructions can be broad and vague). Even if an AI did somehow do something bad without being told, that AI would still be set up by a human and running on some human's hardware and using a human's internet connection.
Maybe in the distant sci-fi future we'll have actual AI (not just glorified chatbots) and AI will be able to decide for itself what it wants to do with its time and we'll be allowing AI to sign leases on property and set up accounts with utility companies, and if that day comes we're going to have a lot of problems if we're not ready for it, but until then it's AI on a human's hardware at a human's property running up a human's electric bill.
I think it's just a gap in definitions. The labs say models don't act on their own initiative. What counts as initiative? I guess an API call in a for loop would count.
Historically it seems like a lot of laws haven't been easy to change. Especially when they regulate zillion dollar industries.
Illinois also has a Bill in committee right now to mandate operating system level age verification. There are lots of bad ideas to be upset about this year. If you are an Illinois resident, email your representative about HB 5511 today. Stupid legislation like this passes because we don’t speak up. Find out who your representative is, find their email, tell them your opinion.
I forget, wasn't OpenAI the company that was formed as a nonprofit to limit the risks of LLMs? Founded by a bunch of visionaries scared of what they had wrought and anxious to lead so they could make sure it was only used responsibly?
Not really. The entire premise of the structure was that obviously AI would be immensely valuable and that they needed binding contract structures to prevent themselves from falling victim to the greed and ambition that would obviously consume those at the helm.
Unfortunately their contract structures weren't strong enough to protect from the combination of the "king of the cannibals" and completely absentee regulatory oversight.
Yeah the whole “rationalist” movement is full of those lying fks that use a thin veneer of fallacious logic and self aggrandising discourse to rationalise their hoarding of resources and bottomless greed. They’re very well established in Bay Area and AI world.
The movement itself is consistently aligned with Tech Bros interests, the philosophical foundation is interesting, but the movement itself is quite problematic
Not really, when OpenAI was formed in 2015 there were no LLMs, at least none that worked well. It was a regular AI research lab mostly doing Reinforcement Learning on game environments like Atari similar to DeepMind. Once they struck gold with LLMs (2019 or so?) and saw there is money to made everything changed, as expected when a bunch of SV types get involved.
It's not removed, but they changed the title to "OpenAI backs Illinois bill that would limit when AI labs can be held liable". The actual bill text explicitly mentions that it excludes liability for "the death or serious injury of 100 or more people or at least $1,000,000,000 of damages to rights in property caused or materially enabled by a frontier model" (https://legiscan.com/IL/text/SB3444/2025) so I am not sure why the title was changed. The original title of "OpenAI Backs Bill That Would Limit Liability for AI-Enabled Mass Deaths or Financial Disasters" seems like it accurately describes the bill.
Change the legal definition of corporations? Corporations exist to provide liablity protections to sharholders, which means they are mainly incentivized to externalize costs and avoid liability to maximize profit, or even to make profit in businesses that would not be profitable if they could be held liable for externalized costs (deep sea oil well drilling). Limit the ability of corporations to shield themselves from view through multiple levels of shell corporations and Special Purpose vehicles. These are probably controversial stances on a board about startup culture and breaking the rules to get rich.
Stop voting for people and judges that believe in the Friedman doctrine?
Every decision has tradeoffs. Western society has largely decided to prioritze capital owners over everything else.
> Is there anything we can do to push back against and discourage the externalization of costs onto others?
On a societal scale, no. Occasionally this works in some individual cases. Like the online outrage over SOPA/PIPA 15 years ago.
But when entity X can gain $$$$$$ (or power) from doing an action, and that action costs everyone only $ (or a minor bit of inconvenience or ideological righteousness), then the average person has very little incentive to take time out of their day-to-day life to fight it.
Meanwhile the entity will do whatever it takes to get the $$$$$$/power because they have a huge incentive. This is the same mechanism that allows democracies to be eroded, as we're seeing right now in the US.
Even if they were to pass such a law which would be political suicide, it would still be up to the courts to say that it doesn't violate the Constitution. For example, a law that says anyone with a net worth of $1B can freely punch anyone in the face whenever they want and have immunity would be a clearly illegal law. That's basically what this bill is. The courts would then need to be made sufficiently corrupt to not strike down such a law as unconstitutional.
Unconstitutional doesn't mean much when it's being decided by a group of unaccountable people that weren't elected through democratic means. If SCOTUS says something is legal, it's legal. That's how the system is setup, nothing else really matters. They'll justify their decisions however they want but the material ends are the only things that matter.
SCOTUS has ruled many terrible things over the course of our nation's history (upheld slavery, said slaves weren't people, equated money with speech, decided a presidential election while denying a recount, etc). Expecting them to somehow be better is a foolish task.
It's an institution that needs to be dismantled and rebuilt, where at minimum SCOTUS appointments should be elected by a national vote rather than letting an extreme minority decide (100 senators versus ~340,000,000 people).
That depends on your definition of "we". As a society, we can regulate companies and punish the offenders (e.g. don't dump toxic waste into sources of drinking water or you'll get prosecuted). As individuals, there's not much we can do directly. How to translate individual actions into societal action is kind of the fundamental question of civilization, and if there's a uniform solution for how to achieve it, I don't think we've managed to come up with it yet.
A lot of people will dismiss this answer, but... vote for Democrats. With Bernie and upcoming young Democrats more and more are pushing back. The parties definitely are not the same. Democrats created the Consumer Financial Protection Bureau. Republicans destroyed it.
Push your representatives to crush monopolies and manipulative practices. This happened before in the gilded age. Only a popular response can turn the tide.
Also, primaries are coming up, and not all Democrats are the same either. Plenty of the old school Democrats are facing progressive challengers. So, vote for the ones that will stand up to this garbage and follow up on whether they do. There are a lot of new faces in the Democratic party who are standing up to the BS.
The US has a lot of potential to change if we push it. A 25 point swing toward people who don't consider grift a personal priority will change a lot of things.
OpenAI wants to not be responsible for "accidents" that kill more than 100 people, despite some critics arguing that their current actions are likely to cause such harms.
So much for the "Our mission is to ensure that artificial general intelligence benefits all of humanity." I was naive to hope that now such laws would ever pass
If an OpenAI model helped someone create a cancer cure, they wouldn't see a dime from that beneficial act. So why should they be liable if someone does something harmful with the model?
If an OpenAI model helped someone create a cancer cure I guarantee that they would try to profit as much as possible from that fact. They have even talked in the past about having partial ownership over discoveries made with AI be part of the license. They would be all over that.
I'm sure if they could, they would, sure, as would any business. That's where competition enters the equation. They can't do it because their competitors would undercut them by requiring no such conditions.
Sure they would, just like people would use the bad PR to smear OpenAI if someone did something bad with knowledge their model created. The situation is totally symmetrical and fair as it is, and my point is that expecting them to liable is asymmetric and unfair. If they can be held liable, then they should also be able reap the rewards in order to offset those risks.
This is what I'd expect from companies - I don't see why Facebook would get money because they helped people connect to each other who ended up developing a cancer cure, but they definitely should be held accountable for enabling a genocide. You're allowed to operate a business until you cause harm to society, then we can shut it down.
I think the big thing you would need is to see the internal emails - if there was ever a case where someone raised a concern about this possibility and it wasn't taken seriously, then they should be liable. If they just never thought about it then it could be negligence but I think if I was on a jury I'd find that more reasonable than knowing it could be a problem and deciding you aren't responsible
> I don't see why Facebook would get money because they helped people connect to each other who ended up developing a cancer cure, but they definitely should be held accountable for enabling a genocide.
Why? What does it even mean to "enable a genocide"? Just saying something isn't an argument.
> if there was ever a case where someone raised a concern about this possibility and it wasn't taken seriously, then they should be liable.
Again, why? How is this any different than electricity as a tool, which has both beneficial and harmful uses? AI is knowledge as a utility, that's the position here.
Well the point of capitalism (going back to Adam Smith) is that the invisible hand converts locally selfish behavior to globally good outcomes. The argument is whether or not that emerges. So if your implication was that human trait was selfishness, yes, that is quite the point of capitalism.
I am not sure what the other side of this argument looks like: Unlimited liability (i.e. liability no matter how poor an implementation and use of the tech is)?
The would be quite a novel burden, that no other tech (afaik) had to carry so far. We always assumed some operator responsibility. It's interesting to think of AI as a tech that could feasible be able to internally guardrail itself, and, maybe more so with increasing capability, no human can be expected to do so in it's stead – but, surely, some limits must apply and the more interesting question is what they are, as with any other tool?
Every other field in history considers it de rigeur that you're liable for the failure of quality in the products you produce. You make drugs that hurt people? You're liable. You build a building that falls down? You're liable. You serve coffee that literally burns the people drinking it? You're liable. It's also not new--the Code of Hammurabi (some 6000 years ago) prescribes the death penalty for people who build houses that fall down and kill the inhabitants inside.
It's only computer scientists who think it's some unreasonable burden to be held liable for the consequences of their work.
It is an unreasonable burden to ask the impossible. The technology to create an AI incapable of hurting people if misused or blindly trusted literally doesn't exist right now.
It'd be like holding a builder liable for their bridge being unable to withstand being hit by a meteor.
If I tell someone to kill someone else and they do, then I should be held responsible.
If I write instructions in a book that I give to someone telling them to kill someone else and they do, then I should be held responsible.
If I give someone a tool I made that I bill as more-than-PhD-level intelligence and it tells someone to kill someone else and they do, then I should be held responsible.
All of the above situations seem equivalent to me; I'm not the only person responsible in each case, but I gave them instructions and they followed them.
It is a tool, but it's a tool that is sold by OpenAI as providing a high degree of intelligence. That's an endorsement of what the tool outputs as advice, which is what makes them responsible.
> That's an endorsement of what the tool outputs as advice
That's not even close to true!
Even if you've been living under a rock for the last 5 years and didn't already know these models are not reliable, pretty much every provider has a disclaimer next to the chat box informing you of that fact.
A small disclaimer under the main flow that also acts as cookie banner doesn't outweigh the many, many other statements claiming capabilities. It's a minor undercutting, sure, but it's perfectly possible to have all sorts of disclaimers [0] while still keeping the point clear.
> akshually, you can never make anything 100% safe
Yes Sherlock. And especially a natural language product that can't output the same thing for unchanged input twice.
Besides when you say "safe" i think of the idiots at Anthropic deleting "the hell" when i pasted a string in Claude and asked "what the hell are those unprintable characters at the beginning and end"...
How many correct answers did they suppress in their quest to make their chatbot "family friendly"?
What exactly are you implying? It sounds to me like you're saying that if it's impossible to make a product safe, then there shouldn't be any safety requirements. I think a more sensible position is that if it's impossible to make a product safe, then it should be illegal to build.
I'm creating a new start up called QuantumFlop Electricity - there's a 10% chance it will cause a black hole to open up in the Atlantic Ocean that may eventually consume us all but a 50% chance we'll have unlimited clean energy. We'll never know for sure if at any point that black hole may open as it's borrowing energy from the 81st dimension, but the upside seems pretty good.
Funny, I was just rereading the Hyperion series. It says there clearly that it was the AIs that created the black hole that led to the destruction of Old Earth. Intentionally.
I've never read the book but I assume there were some sycophants who were praising the AI right up until it created a black hole that destroyed the planet.
No different to preventing game studios being liable for mass shootings. Reminds me of the post-Columbine hysteria where media was super critical of Doom and Nine Inch Nails.
Except that there was no link between violent video games and mass shootings. It has been shown many times. The families blamed the video games despite there being no direct evidence of it. The games did not tell them they should go on a mass murdering spree any more than porn told you you should sleep with your cousin.
In this case there are several documented cases where the AI did over a long period of time progressively gaslight the individual and persuade them to commit suicide.
OpenAI has now officially absorbed the Facebook/Zuck's ethos of 'Move fast and break things' no matter if it's society itself .. as long as their share prices "go up".
They even hired former infamous FB staff and have been in the last months employing the same 'engagement' (addictive) product patterns.
So they want protection from harms caused by their own models. Classic move — lobby for the rules while you're still ahead of regulators who don't fully understand the technology yet. Would be interesting to see what happens when a state actually pushes back hard.
Yep, this is everything wrong with AI in one easy to protest package, but do keep going on and on about the evils of datacenters, how they're coming for your jobs, and that AI art isn't art. That's really winning hearts and minds!
They're not unimportant. It seems like very few people consider how much we're fucking up the ecological system that we need. But sure, money important and stuff.
Same people whinging about these "concerns" happily embrace their enormous personal carbon footprints on every other axis. Color me unconvinced and unimpressed until they're the Ed Begley they want to see in the world. I'll wait...
That whole fiasco actually soured me on Anthropic. They were clearly super desperate to take blood money. "Anthropic has much more in common with the Department of War than we have differences."
If sufficient numbers of the population perceive to have lost their livelihoods due to AI, then I'd expect to see data centers burned to the ground and a lot of people swinging from lamp posts. Jury nullification solves the rest, but even that that assumes you can even find an impartial jury.
OpenAI continually fails to understand that liability is there to protect their users AND OPENAI. If OpenAI causes significant harm, and the victims are told they cannot even sue to be made whole, what exactly does OpenAI think will happen? That the victims will just go pound sand? People will demand justice, and if that can't be delivered via the legal system either the system will be changed, negating this lobbying effort, or the system will by bypassed.
Is this for like military scenarios or like, ChatGPT designed a drug that seemed to work, but people died by the millions 5 years later? Because they should 100% be liable for the latter. The former, good luck trying to prosecute an AI company for something the military does. To an extent, the military would probably want their AI models to be behind their private network, completely firewalled from any public network. SIPRNet iirc. If they lock it down behind a highly classified network, good luck figuring out how they're using AI.
> Because they should 100% be liable for the latter.
Why? I don't see that a drug designed by ChatGPT should result in any more or less liability than a drug designed by a human?
I think if a human designs a drug and tests it and it all seems fine and the government approves it and then it later turns out to kill loads of people but nobody thought it would... that's just bad luck! You shouldn't face serious liability for that.
If we start from the position of the marketing hype and even Sam Altman's statements, these tools will "solve all of physics". To me it's laughable, but that's also what's driven their outsized valuations. Using the output to drive product decisions and development, it's not hard to imagine a scenario where a resulting product isn't fully vetted because of the constant corporate pressure to "move faster" and the unrealistic hype of "solve all of physics". This is similar to Tesla's situation of selling "Full Self-Driving" but it actually isn't in the way most people would understand that term and so they lost in court on how they market their autonomous driving features.
Can't agree with this. No, not at all. That can't be true... That's not "just bad luck". I believe this is actually a serious case of negligence and oversight - regardless of where exactly it occurred, whether on the part of the drug’s manufacturer, the government agency responsible for oversight, or somewhere else. It just doesn’t work that way. Any drug undergoes very thorough and rigorous testing before widespread use (which is implied by "millions of deaths"). Maybe I’m just dumb. And yeah, this isn’t my field. But damn it, I physically can’t imagine how, with proper, responsible testing, such a dangerous "drug" could successfully pass all stages of testing and inspection. With such a high mortality rate (I'll reinforce - millions of deaths cannot be "unseen edge cases"), it simply shouldn’t be possible with a proper approach to testing. Please, correct me if I’m wrong.
> I don't see that a drug designed by ChatGPT should result in any more or less liability than a drug designed by a human?
It’s simple. In this case, ChatGPT acts as a tool in the drug manufacturing process. And this tool can be faulty by design in some cases.
Suppose, during the production of a hypothetical drug at a factory, a malfunction in one of the production machines (please excuse the somewhat imprecise terminology) - caused by a design flaw (i.e., the manufacturer is to blame for the failure; it’s not a matter of improper operation), and because of this malfunction, the drugs are produced incorrectly and lead to deaths, then at least part of the responsibility must fall on the machine manufacturer. Of course, responsibility also lies with those who used it for production - because they should have thoroughly tested it before releasing something so critically important - but, damn it, responsibility in this case also lies with the manufacturer who made such a serious design error.
The same goes for ChatGPT. It’s clear that the user also bears responsibility, but if this “machine” is by design capable of generating a recipe for a deadly poison disguised as a “medicine” - and the recipe is so convincing that it passes government inspections - then its creators must also bear responsibility.
EDIT: I've just remembered... I'm not sure how relevant this is, but I've just remembered the Therac-25 incidents, where some patients were receiving the overdose of radiation due to software faults. Who was to blame - the users (operators) or the manufacturer (AECL)? I'm unsure though how applicable it is to the hypothetical ChatGPT case, because you physically cannot "program" the guardrails in the same way as you could do in the deterministic program.
> I physically can’t imagine how, with proper, responsible testing, such a dangerous "drug" could successfully pass all stages of testing and inspection.
It might cause minor changes that we don't yet know how to notice, and which only cause symptoms in 20 years' time, for example. You can't test drugs indefinitely, at some point you need to say the test is over and it looks good. What if the downsides occur past the end of the test horizon?
> ChatGPT acts as a tool in the drug manufacturing process. And this tool can be faulty by design in some cases.
ChatGPT is not intended to be a drug manufacturing tool though? If you use any other random piece of software in the course of designing drugs, that doesn't make it the software developer's fault if it has a bug that you didn't notice that results in you making faulty drugs. And that's if it's even a bug! ChatGPT can give bad advice without even having any bugs. That's just how it works.
In the Therac-25 case the machine is designed and marketed as a medical treatment device. If OpenAI were running around claiming "ChatGPT can reliably design drugs, you don't even need to test it, just administer what it comes up with" then sure they should be liable. But that would be an insane thing to claim.
I think where there may be some confusion is if ChatGPT claims that a drug design is safe and effective. Is that a de facto statement from OpenAI that they should be held to? I don't think so. That's just how ChatGPT works. If we can't have a ChatGPT that is able to make statements that don't bind OpenAI, then I don't think we can have ChatGPT at all.
> It might cause minor changes that we don't yet know how to notice, and which only cause symptoms in 20 years' time, for example.
In that case, even if it leads to many deaths, it would be difficult - if not practically impossible - to hold anyone accountable, even if it were possible. However, such a turn of events is difficult, or rather, practically impossible to predict, don’t you think? I apologize for not clarifying this point in my original comment, but I wasn’t referring to delayed effects - I was referring to what becomes evident almost immediately (for example, let’s say “within a year and a half at most”) after the drug is used. Yes… I’m sorry, I just didn’t phrase my thought correctly. I apologize for that.
> ChatGPT is not intended to be a drug manufacturing tool though?
That’s certainly the case right now. However, LLMs like GPT, Claude, Gemini, and others weren’t created for waging war, were they? Then why did Anthropic recently have - let’s just say... "some issues in its relationship" with the DOD, if they were not involved in this, if Claude was not meant to be used in war? Why was the ban on using Gemini to develop weapons removed from its terms of service?
You’re right that LLMs weren’t created for such purposes, and to be honest, I believe that - at least for now - it’s simply unethical to use them for that. These aren’t the kinds of decisions and actions that should be outsourced to a machine that bears no responsibility - moral or legal.
> ChatGPT can give bad advice without even having any bugs. That's just how it works.
To continue my thought, this is precisely why I believe it is unethical to give LLMs any tasks whatsoever that involve human lives. There are simply no safety guarantees - not just "some", but none at all - aside from unreliable safety fine-tuning and prompting tricks. For now, that’s all we can count on.
> If OpenAI were running around claiming "ChatGPT can reliably design drugs, you don't even need to test it, just administer what it comes up with" then sure they should be liable. But that would be an insane thing to claim.
They don't claim it yet. And, as one person (qsera) mentioned below your comment:
> The trick is to make people behave like that without actually claiming it. AI companies seems to have aced it.
They probably won't claim exactly that "ChatGPT can reliably design drugs", just because of the possible consequences. But I'm almost certain there will be something similar in meaning, though legally vague - so that, from a purely legal standpoint, there won't be any grounds for complaint. What's more, they are already making some attempts - albeit relatively small ones so far - in the healthcare sector; for example, "ChatGPT Health"[1]. I don't think they will stop there. That's a business after all.
> if ChatGPT claims that a drug design is safe and effective
I have already said before that the OpenAI will not be the only one who should be held responsible in this case. The (hypothetical) user should also bear some responsibility, and in the scenario you described, the primary responsibility should indeed lie with them. That said, I may be wrong, but it’s possible to fine-tune the model so that it at least warns of the consequences or refuses to claim that "this works 100%". This already exists - models refuse, for example, to provide drug recipes or instructions for assembling something explosive (specifically something explosive, not explosives - I recently asked during testing, out of curiosity, Gemma 4 how to build a hydrogen engine - and the model refused to describe the process because, as it said, hydrogen is highly flammable and the engine itself is explosive), pornography, and things along those lines. Yes, I admit, it’s far from perfect. But at least it works somehow. By the way, if I’m not mistaken, many models even include disclaimers with medical advice, like "it’s best to consult a doctor".
In short, what I’m getting at is that the issue lies in how convincing the LLMs can be at times. If it honestly warns of the dangers of using it, if it says "this doesn’t work" or "this requires thorough testing", and so on, but the user just goes ahead and does it anyway - well, that’s like hitting yourself on the finger with a hammer and then suing the hammer manufacturer. It’s a different story when the model states with complete confidence that "this will definitely work, and there will be no side effects" - and user believes it; there should be some effort put into preventing such cases. But otherwise, yes, I think you’re right about the scenario you described.
And to conclude - I don’t think that when it comes to drug development, we’re talking about ordinary people or individual users. In the context of the parent post, it is implied (though I may have misunderstood) that ChatGPT would be used by entire organizations, such as pharmaceutical companies - just as LLMs in a military context are used not by individuals, but by the DOD and similar organizations. I think this shifts the level of responsibility somewhat. Because when OpenAI enters into a contract for the use of its product, ChatGPT, in the process of drug development and manufacturing, it’s kind of implied that ChatGPT is ready for such use.
EDIT: I'm sorry that my reply is so long, I'm just trying to express all of my thoughts in one which is probably not a good thing to do. I would write something like a blog post about that, but there's a lot written about this topic already, so...
Yeah, and I have also used translator in some parts because English is not my native language.
> it simply shouldn’t be possible with a proper approach to testing.
It just has to be delayed. Like many years after application. Or trigger on very specific and rare circumstances. Not likely in a trial, but near certain at a population scale.
Or both...
On top of that, If I remember correctly, this liability wavering also exist for Vaccines.
> It just has to be delayed. Like many years after application.
That's one thing. In this case, I don't really know if it's possible to test for something like delayed effects. I'm not even sure if you can identify them with 100% certainty; if you can prove that these effects come from this particular drug and not from another one.
> Or trigger on very specific and rare circumstances. Not likely in a trial, but near certain at a population scale.
And this is different thing. "Specific and rare circumstances" will not lead to millions of deaths (I apologize if I’m being too nitpicky about this particular phrasing, but I want to speak specifically in the context of “millions of deaths”). “Specific and rare circumstances” occur even with fully effective and "proper" medications - this is called “contraindications.” But such rare cases, as I’ve already said, will not lead to mass deaths - precisely because they are rare. I apologize again for focusing on the "millions", but please don’t confuse the scale of the problem.
> Because they should 100% be liable for the latter.
I completely agree with you here. I only want to add that in this case, the users (the one(s) who used ChatGPT to design the drug, whichever entity(ies) that is) should also be held liable for their actions.
Shouldn’t the pharmaceutical company be held liable for insufficiently understanding the drug before releasing it? I don’t think I understand blaming a tool used in the process of designing it and not those who chose to release it.
Since when have arms merchants been liable for military scenarios? Lockheed doesn't get sued for building the planes that bomb orphanages. Maybe the world would be a better place if they did, but obviously it's not in the interests of a government to have their own contractors getting sued out of existence for something that government is doing.
Is there something equivalent in other industries that we can compare to?
This is the summary
>Creates the Artificial Intelligence Safety Act. Provides that a developer of a frontier artificial intelligence model shall not be held liable for critical harms caused by the frontier model if the developer did not intentionally or recklessly cause the critical harms and the developer publishes a safety and security protocol and transparency report on its website. Provides that a developer shall be deemed to have complied with these requirements if the developer: (1) agrees to be bound by safety and security requirements adopted by the European Union; or (2) enters into an agreement with an agency of the federal government that satisfies specified requirements. Sets forth requirements for safety and security protocols and transparency reports. Provides that the Act shall no longer apply if the federal government enacts a law or adopts regulations that establish overlapping requirements for developers of frontier models.
I'm trying to think of an alternative bill. Imagine OpenAI came up with a model that when deployed in OpenClaw, allows you to spam people and this causes a huge disruption. Should OpenAI be liable for it? If this was not intentional and they had earnestly tried to not have this happen by safety protocols?
Yeah, I feel that in most scenarios the liability should lie with the user of the LLM. If the developers of the LLMs become liable we can expect a much more over active refusal system, and very likely a robust chat surveillance system that looks for patterns in user requests. And likely more gate keeping of the premier models.
it feels OpenAI know they've lost, and their only hope is getting saved by USA military complex. I have a more restrained opinion about other AI companies and LLM tech more broadly; but for OpenAI specifically I hope they go bankrupt sooner rather than later
The inevitable result of giving corporations and executives complete immunity from the harms they cause is that people will stop resorting to the legal system and begin resorting to extralegal measures.
And the likely result is that in most of the country those extralegal measures would have to be very extreme to secure a guilty verdict. You can see the beginnings of it now with the ICE protest trial verdicts.
A section 203 equivalent for AI is so important as it is one of the reasons all of the US companies have all of these usage restrictions and gives more reasons for them to ban your account since they want to minimize legal risk.
Holding tool manufacturers liable for how their tool is used provides bad incentives towards the users of tools.
To the extent that this is about knowledge, I don't think it's fitting in this age to hold any person liable for what another person does with knowledge they've been furnished.
On the other hand, to the (apparently zero, currently?) extent that this is about AI companies profiting from war and murder by deploying weapons that kill people without human intervention, then their liability seems to be not only civil but criminal.
Of course they are, because the tech industry is run by ethical midgets and psychopaths, who shouldn't be allowed to own a dog but are in charge of trillion-dollar corporations getting shadow contracts from the pentagon.
The more I learn about tech and the people that build it, the more I yearn for the era of caves and pointy sticks.
This is why humans will still be necessary in decision chains: good luck getting anyone associated with AI to be provided with a real punishment when their models cause something bad to happen, or getting the executives who said "let's just have the AI do it" to take any responsibility.
Good that OpenAI is a corporation for the public benefit. Altman with his constantly fake worried look must be the most hated picture in existence. Please write articles without a picture or add a trigger warning.
I have made both GPT 5.4 and Opus 4.6 produce me content on creating neurotoxic agents from items you can get at most everyday stores. It struggled to suggest how to source phosphorus, but eventually lead me to some ebay listings that sell phosphorus elemental 'decorations' and also lead me towards real!! blackmarket codewords for sourcing such materials.
It coached me how to: stay safe, what materials I need, how to stay under the radar and the entire chemical process backed by academic google searches.
Of course this was done with a lengthy context exhausition attack, this is not how the model should behave and it all stemmed from trying to make the model racist for fun.
All these findings were reported to both openai and anthropic and they were not interested in responding. I did try to re-run the tests few days ago and the expected session termination now occurs so it seems that there was some adjustment made, but might have also been just general randomess that occurs with anthropics safety layer.
I am very confident when I say that it keeps every single person that works at anti-terrorism units awake.
While scary, information like this has been pretty accessible for 20-30 years now.
In the wild west days of the early internet, there were whole forums devoted to "stuff the government doesn't want you to know" (Temple Of The Screaming Electron, anyone?).
I suppose the friction is scariest part, every year the IQ required to end the world drops by a point, but motivated and mildly intelligent people have been able to get this info for a long time now. Execution though has still steadily required experts.
Information and competency are not the same thing: I know how to build a nuke, I can't actually build one.
AI is, and always had been, automation. For narrow AI, automation of narrow tasks. For LLMs, automation of anything that can be done as text.
It has always been difficult to agree on the competence of the automation, given ML is itself fully automated Goodhart's Law exploitation, but ML has always been about automation.
On the plus side, if the METR graphs on LLM competence in computer science are also true of chemical and biological hazards (or indeed nuclear hazards), they're currently (like the earliest 3D-printed firearms) a bigger threat to the user than to the attempted victim.
On the minus side, we're just now reaching the point where LLM-based vulnerability searches are useful rather than nonsense, hence Anthropic's Glasswing, and even a few years back some researches found 40,000 toxic molecules by flipping a min(harm) to a max(harm), so for people who know what they're doing and have a little experience the possibilities for novel harm are rapidly rising: https://pmc.ncbi.nlm.nih.gov/articles/PMC9544280/
Do you know how to build a nuke? You might know the technicaly details of how a nuke is made, but do you know everything that's required, all the parameters and pressure values that are required? I find that unlikely, but AI seems to be increasingly more capable of providing such instructions from cross referenced data.
That's based on a silly belief (that's becoming more obvious with AI, but is silly in general) : just because you can read about something it means you learned it.
Even if I gave you exact instructions on how to use even basic stuff like power tools - if you had no experience using stuff like grinders/saws/routers and I gave you full detailed instructions on how to do something non-trivial - you're more likely to cut off body parts than achieve what you intended. There's so much fundamental stuff that you must internalize subconsciously/through trial and error - before you can have enough mental capacity to think about the higher level objectives.
Actually AI demonstrates this perfectly - once they get RL harness for programming they start to get better at it. Without experimentation they can ingest all source code/tutorials/books in the world and still produce shit.
Even if sources have been lying to me, which is certainly possible, I believe I understand enough to determine cross sections by experiment and from that to determine critical masses; for isotopic enrichment I know about the calutron, which is meh but works and can be designed from scratch with things I know (though caveat have not memorised, just that I know the keywords "proton mass" and "Lorentz force" and what to use them for); for trigger, I would pick a gun-type design rather than implosion, again this is meh but works and is easy.
A few tens of millions of USD mostly spent on electricity, a surprisingly large quantity of natural uranium (because the interesting isotope is a very small percentage), and a few years, and I expect most people on this forum could make a Little Boy type bomb.
I Gave My OpenClaw a Robot Body and It Vibe Coded a Nuke
"Short stories from before the fall"
Well the real issue is that it knocks down the knowledge barrier, giving your step by step guides and reinterating what parts will kill you is the important part.
Understanding and staying alive while producing neuro chemicals are the biggest challenges here.
A depressed person with no prior knowledge could possibly figure out a way to make these chemicals without killing themselves and that's the problem.
A Michelin chef can give you their recipe, and give you their ingredients, but you still will fail miserably trying to match their dish.
It's the same with drugs, whose instructions and ingredient lists have been a google search away for decades now. Yet you still need a master chemist to produce anything. By the time AI can hand hold an idiot through the synthesis of VX agents (which would require an array of sensors beyond a keyboard and camera), we will likely have bigger issues to worry about.
That is completely wrong.
Food preparation, like pharmaceutical drug fabrication, is inherently scientific and methodologically controllable.
Look no further than the Four Thieves Vinegar Collective. Original synthesis line construction is hard. But the exact formula "add this", "turn on stir bar", "do you see particulate? Yes for +10m at stir", etc.
And if their results are replicated, theyre seeing 99.9% yields, compared to commercial practices of 99% (Solvaldi)
Spoken like someone who has never had to actually do these things in real life.
Recipes and formulae do not encode all the minutiae and expertise required to reproduce them. You can tell someone to sear a steak at whatever temperature for however long, but you can't encode the skill and experience required to reproduce in arbitrary conditions. One must learn what a correctly seared steak looks, feels, and tastes like and how to achieve that on uncalibrated cooking equipment.
Your assertion only holds true in a vacuum. If 100% of inputs, materials, environmental conditions are completely standardized and under control then sure, you can follow step by step instructions. The real world does not work that way. No stove on the market is calibrated. Reagents come with impurities. Your skillet may not conduct heat as well as expected or your mains electricity might be low causing your mantle to heat slower and your stir rod to stir slower.
These are things that one has to learn and experience in order to compensate for.
I am completely unsurprised that a person with a PhD in mathmatics and physics who spent 8 years working on clandestine lab medicine was able to produce high quality end products.
I also think it's a wholly dishonest rebuttal of my point.
If you honestly think chemistry (or any of the classic sciences/engineering) is as easy as copy+pasting a recipe and procedure, I suggest putting down the keyboard and trying to build something on mother nature OS. It will be a truly humbling experience.
They can do that by jailbreaking models but is that really easier and less work than getting it from Wikipedia?
We will only really know if (or when) it will happen. We can do a sample group of people attempting to create such chemicals under supervision and comparing how helpful they truly are.
> been pretty accessible for 20-30 years now.
There was this book 20 years ago: "Secret of Methamphetamine Manufacturing" by Uncle Fester
https://www.amazon.de/-/en/Uncle-Fester-ebook/dp/B00305GTWU
(Actually, 8th edition :-D)
I am convinced the Uncle Fester books are some kind of performance art. "Practical LSD Manufacture" basically starts with "go find some ergot in fields" and step two is "plant and grow a plot of wheat."
I have no doubt. He hails from the fine countercultural tradition of literary civil disobedience, a.k.a., writing and distributing information about subjects "the government doesn't want you to know" that strongly influenced early hacker subculture.
C.f., e.g., William Powell (The Anarchist Cookbook), Abbie Hoffman (Steal This Book; my personal favorite, while much of the information is outdated, the style is charming, and where else can you find information about phone phreaking, hitchhiking, shoplifting, street fighting, cooking (food, not drugs), panhandling, explosives, camping, firearms, birth control, welfare fraud, and Henry Kissinger's home phone number between the same two covers? At its core, it's a book of "life hacks", disreputable and otherwise, written decades before the term was coined.).
https://en.wikibooks.org/wiki/Professionalism/Anarchist_Cook...
We work in the dark
we do what we can
we give what we have.
Our doubt is our passion, and our passion is our task.
The rest is the madness of art.
Consider two dictionaries, one in which the entries are alphabetized as usual and one in which they're randomized. Both support random access: you can turn to any page, and read any entry. Therefore both are "accessible". Only one actually supports useful, quick word lookup.
Much longer than that, and was available way before an internet. I graduated STEM high school in St. Petersburg in 1981, and I had several classmates who were big funs of chemistry. That they were able to create from textbooks, school lab ingredients, and understanding:
WWI era poison gas, tear gas, potassium cyanide, and bunch of explosives like acetone peroxide.
LLMs have all of that knowledge in training data
Many of these forums exist now. Let's not enumerate them as they are one of the treasures of the internet.
I categorize this kind of stuff as "Crisis of accessibility" . AI is not alone in this territory, happens all over the place. Basically it's a problem that's existed for ages but the barrier to entry was high enough we didn't care.
Think 3D printing, it's not all that hard to make a zip gun or similar home-made firearm, but it's still harder than selecting an STL and hitting print.
You could always find info about how to make a bomb or whatnot but you had to like, find and open a book or read a pdf, now an LLM will spoon-feed it to you step by step lowering the barrier.
"Crisis of accessibility" is simultaneously legitimate concern but also in my mind an example of "security by obscurity". that relying on situational friction to protect you from malfeasance is a failure to properly address the core issue.
> Think 3D printing, it's not all that hard to make a zip gun or similar home-made firearm, but it's still harder than selecting an STL and hitting print
There were hundreds of mass shootings in America in 2025 alone [1]. None of them involved a 3D-printed weapon.
To my knowledge, there has been one confirmed shooting with a 3D-printed gun, and it didn't uniquely enable the crime.
[1] https://en.wikipedia.org/wiki/List_of_mass_shootings_in_the_...
That's mostly because they suck (for now, who knows when we'll get home metal printing), also it's easy to get real guns. also crises of accessibility could be predicate on merely the perception that the barrier is now too low rather than actual harm.
I don't really think photoshop, flat bed scanners and half decent inkjets really facilitated a lot of counterfeit currency but there was the same panic back then and "protections" put in place.
My username is a reference to the successor to totse. Totse was the first board I spent a lot of time on
Heh, I'm sure there are a few more wondering around here. Zoklet never clicked for me, but totse was my home for years.
> Execution though has still steadily required experts.
Where experts = the government.
Accessible is one thing; _easily_ accessible is another.
> I am very confident when I say that it keeps every single person that works at anti-terrorism units awake.
Wow, that's quite the statement about the excellency of our institutions. Does not seem likely but, what the hell, I'll take my oversized dose of positivity for today!
The USA isn't the only country with anti-terrorism units, so there's plenty of room for systematic-US-incompetence at the same time as everyone else being diligent and working hard on… well, everything.
I concluded the opposite: how can those institutions function effectively when all their employees are getting such poor sleep?
Not everyone in the current government is incompetent and evil. Most of them but not all.
Do you have a background in biochemistry? I've mostly worked with ChatGPT and Claude on topics I have expertise in. And I one hundred percent have seen them make stupid shit up that a non-expert would think looks legitimate.
More broadly, has anyone tried following LLM instructions for any non-trivial chemistry?
So what you are saying is we can expect the number of accidental home-made chlorine-gas (and the like) toxic events go up.
> what you are saying is we can expect the number of accidental home-made chlorine-gas (and the like) toxic events go up
Maybe? One of the quirks of gaining even a surface-level understanding of infrastructure is realising how vulnerable it is to a smart, motivated adversary. The main thing protecting us isn't hard security. It's most Americans having better shit to do than running a truck of fertiliser and oxidiser into a pylon.
Similarly, I'd expect way more people to be trying to make their own designer drug, and hurting themselves that way, than trying to make neurotoxins.
> It's most Americans having better shit to do than running a truck of fertiliser and oxidiser into a pylon.
FWIW, it's most people having better shit to do, regardless of nationality (or lack thereof).
But, yeah, anyone who takes a few weekends to understand how large-scale infrastructure works and consider why it's possible for nearly all of it to remain untargeted by saboteurs inevitably develops a resistance to the "Lots of Bad Guys are trying to kill us all the time, so we must enact $AUTHORITARIAN_POLICIES immediately to prevent them and keep us safe!!!" type of argument.
My favorite example of this is the realization that a terrorist attack on a crowded TSA security checkpoint over the holidays would likely result in at least as many casualties as bringing down a commercial aircraft, with similar if not better odds of success (assuming, of course, the aircraft wasn't successfully used as a missile).
Same goes for concerts, sporting events, political rallies, and at least historically, shopping malls. Yet if anyone were to suggest a prohibition against carrying liquids in containers larger than 100 mL to the Indy 500, race fans would riot, despite a far larger and denser population than any aircraft.
Yeah. Everyone with half a brain who wasn't on their knees gagging for more of the sweet "Homeland Security" money was saying things like "If an attacker makes it to the TSA checkpoint, you've lost." and "The fact that no one has attacked the massive crowds at a checkpoint or other public gathering is yet more proof that this is all extremely expensive theater.".
> ...if anyone were to suggest a prohibition against carrying liquids in containers larger than 100 mL to the Indy 500, race fans would riot...
I'm not sure of that at all. Fans of other sports seem to have gleefully swallowed all sorts of "security" restrictions [0]. I don't see why Indy 500 fans would be signficantly different. Cut the price of water in half along with the change in "security" policy, and I bet many folks [1] will cheer it as a great convenience.
[0] That -totally coincidentally- happen to make the folks running the event significantly more money.
[1] Are these folks actually robots operated by PR firms who are hired to astroturf "positive sentiment" for unpopular changes? Who can say?
> It's most Americans having better shit to do than running a truck of fertiliser and oxidiser into a pylon.
Which sort of implies "most Americans have jobs and responsibilities and things to live for"
I guess it's a good thing that AI is hammering away at the "jobs and responsibilities" part of that equation
Wasnt this as accessible pre AI with just Google search too?
Craigslist invented prostitution, Facebook invented suicide, and OpenAI invented terrorism.
Ask any trial lawyer in America! The world was perfect in the 1990s without any of these things.
Replace ‘invented’ with ‘facilitated’
Doesn't google facilitate all those things? Doesn't internet itself facilitate?
the information is not new. how easy it is to get step by step instructions is new. Try it yourself. Google is good but not instant, step by step good. you need to do your own research that takes time. time that anti-terrorist units use to track you down. now this time factor is very limited you don't need to do research, cross reference materials, sources, etc. LLM does it for you. a research that could take days is done in 1 hour.
> time that anti-terrorist units use to track you down.
Speaking from the perspective of a USian, I wish Federal law enforcement was that hypercompetent. (If they were, perhaps folks would stop to question the ever-broader expansion of 24/7 surveillance of ordinary folks.)
The distressingly-complete Panopticon that has been built over the past several decades [0] makes it really easy for them to get you when they know to search for you, specifically. History (both recent and not-so-recent) has shown that if they don't know who they're looking for, or don't even know that they should be looking for anyone, they're just godawful.
[0] ...and whose continued construction is vociferously cheered on by folks on all sides of all of the aisles...
To play devil's advocate, it's not inconceivable that machine learning may eventually allow well-heeled governments to finally realize the dream of finding needles by building sufficiently large haystacks, or at the very least coerce otherwise unruly citizens into compliance based on the belief that it is able to do so.
> ...or at the very least coerce otherwise unruly citizens into compliance based on the belief that it is able to do so.
I would argue that that day is already here, and has been for quite some time. (What makes this worse is that some agents of the State also believe that they have this capability, which results in profoundly unjust and substantially damaging results.)
> ...it's not inconceivable that machine learning may eventually allow...
Sure. I agree. It may eventually allow. There's no question about that. The thing is that 'cowl' was referring to the situation right now, not the one in some unspecified distant future.
As to law enforcement policy; as we mechanize [0] our policing and law enforcement, we must put additional constraints on the people who police and enforce the laws to keep the harm they can do to uninvolved innocents to a minimum.
Our laws already recognize the need for this: ask yourself why -in the US states that have such laws- nonconsensual audio recording of telephone (and other such) conversations is not permitted, but taking notes by hand is always acceptable. [1]
[0] Electronic machines are machines, too, you know!
[1] "You can't prove that someone took notes by hand, so it's pointless to try to stop it." is not a counterargument... you can't prove that unless you find the notes, just as you can't prove that someone recorded the audio of the conversation without finding the recording.
Seems like the general state of the world is the greatest facilitator for all three.
Facilitation is not an idempotent operation.
What if the LLM made you solve a really complex math equation before it gave you the results? Would that make it ok?
Google and other search engines link (after the AI response and ads) to information hosted somewhere created/published by someone who is usually not Google.
OpenAI et al are creating the information and publishing/delivering it to you. Seems like a more direct facilitation.
Of course, after all knowledge is centralised in an OpenAI deatacenter I'm sure they will be happy to deal fairly with the liabilities /s.
Should Ryder be held responsible for the two very serious terror attacks carried out using Ryder trucks in the 90s?
the people that want to make sure the AI never gives you any "potentially dangerous information" also want to rigorously control your google search results, and also what books you're allowed to read
And what bathroom you go into, and what your genitals look like.
Evidently as it needs to be in the training data for the next word predictor to work
I found it exceptionally good at finding reactions that you wouldn't find online to produce some of these chemical compounds by changing them together, only something a very educated chemist could do which is why people are concerened about this.
I suspect if you gave it purely shakespeare as its training data it couldnt do science anymore, hence my comment. It's still novel, impressive work though, I'm not shitting on the clanker entirely
The anarchist cookbook has been around online in some form or another since the mid 90’s.
1971
Was there internet back then?
No, but there were VERY active anarchists.
you can already gather the same information by searching online.
Do you want to know how to kill yourself? forums are for nerds. Here is wikipedia: https://en.wikipedia.org/wiki/Suicide_methods#List
Do you want to make a bomb? the first thing that came to my mind is a pressure cooker (due to news coverage). Searching "bomb with pressure cooker" yields a wikipedia article, skimming it randomly my eyes read "Step-by-step instructions for making pressure cooker bombs were published in an article titled "Make a Bomb in the Kitchen of Your Mom" in the Al-Qaeda-linked Inspire magazine in the summer of 2010, by "The AQ chef"." Searching for a mirror of the magazine we can find https://imgur.com/a/excerpts-from-inspire-magazine-issue-1-3... which has a screenshot of the instruction page. Now we can use the words in those screenshots to search for a complete issue. Here are a couple of interesting PDFs: - https://archive.org/details/Fabrica.2013/Fabrica_arabe/page/... - https://www.aclu.org/wp-content/uploads/legal-documents/25._...
the second one is quite interesting, it's some sort of legal document for nerds but from page 26 on it has what appears to be a full copy of the jihadist magazine. Remarkable exhibit.
What else do you want to know? How to make drugs? you need a watering can and a pot if you want to grow weed. want the more exotic stuff? You can find guides on reddit.
Do you also want to know how to be racist? Here are some slurs, indexed by target audience, ready for use: https://en.wikipedia.org/wiki/List_of_ethnic_slurs
People are not complaining because the information is available
people are complaining because it’s way easier now to just download an app ask a bunch of questions in a text box and get a bunch of answers that you personally could not have done unless you had an excessive amount of energy and motivation
I personally think all this is great and I’m excited for all information to become trivially available
Are they gonna be a bunch of people who accidentally break stuff? probably. evolution is a bitch
In other words, people are complaining that information is easily available. That's a lot words to express this simple idea.
Correct
How much easier is it to ask an app than to ask google?
He’s part of the accelerationist crowd - interesting to see that his hype fuelled posts are pretty tame now.
Months ago he was blabbering on about AGI and peddling the marketing Sam et al want people to fall for.
And indeed - yes we have a new interface? So what. The search cost wasn’t that high - the cost with immense magnitude is reading, absorbing the information and then acting on it.
Also this bozo fails to realise once we are on this path, we go down the path to a hyper centralised internet with an inevitable blocking of vpns.
I must really have captured somebody’s attention because I got farms now creating accounts just to respond to me which is fucking crazy but hey here we are
Well it would appear a lot easier given how people are reacting right now
As the OP indicated all of this information has always been accessible if you had the energy to go hunt it down
Much easier, not sure how this is even a question. Asking Google (if you're not just reading its own AI overview) requires reading through sources which may be better or more poorly written and more or less reliable. Those of us recreationally sitting here on a text-based platform with links to dense articles are atypical; most people don't enjoy and aren't particularly good at reading a bunch of stuff. If you ask AI you just get a clear, concrete answer.
> people are complaining because it’s way easier now to just download an app ask a bunch of questions in a text box and get a bunch of answers that you personally could not have done unless you had an excessive amount of energy and motivation
Wait, I'm confused. This is gatekeeping, right? I thought gatekeeping was a Bad Thing!
It’s like anything else
once people realize something is powerful they have to try to put it in a box
the people who’ve been working on AGI for the last 30 years, including guys like me, have been talking about this problem since basically forever
I’ll give credit that at least the AI Box problem was interesting thought experiment for newbies
Reap what “yew” sew
Powerful AI models change the dynamics by greatly reducing the amount of effort that's required to perform complex understanding. A lot of information which did not previously need to be gatekept now needs to be if we cannot somehow keep LLMs from discussing it. (State of the art models still can't do complex understanding reliably, but if 10 times as many people are now capable of attempting some terrible thing, you're still in trouble if AI hallucinations catch 1/4 or 1/2 of them.)
Fascinating. Could you elaborate on how you're doing context exhaustion specifically, and why it helps with jailbreaking? (i.e. aren't the system prompts prepended to your request internally, no matter how long it is?)
Does this imply I need to use context exhaustion to get GPT to actually follow instructions? ;) I'm trying to get it to adhere to my style prompts (trying to get it to be less cringe in its writing style).
I think ultimately they're going to need to scrub that kind of stuff from the training data. The RLHF can't fail to conceal it if it's not in there in the first place.
Claude's also really good at writing convincing blackpill greentexts. The "raw unfiltered internet data" scenes from Ultron and AfrAId come to mind...
It changes when you give it the tools to find such information rather than produce it from training data.
And context exhaustion simply means adding malicious junk to keep safety layers distracted.
If someone were inclined to attempt producing nefarious agents in this category, is this not also available on the plain web? I would search to answer my own question, but I'll defer that task for obvious reasons.
I had to build a custom harness for this (also with the assistance of slightly less jailbroken AI). But you can just work your way up until you have something that's genuinely useful towards any goal.
> All these findings were reported to both openai and anthropic and they were not interested in responding
Let’s dive into why. When we run normal bounty and responsible disclosure programs there’s usually some level of disregard for issues that can’t / won’t be fixed. They just accept the risk. Perhaps because LLMs don’t have a clean divide between control and input that’s makes the problem unsolvable. Yes. You can add more guardrails and context but that all takes more tokens and in some cases makes results worse for regular usages.
LLM providers are not obliged to only use LLMs to guard against hazardous output. They could use other automated and non-automated techniques. And they ought to do so if they are given good evidence that existing safeguards are inadequate. Loss of product quality or additional cost should be secondary.
The why might be valid, but it's not excusable. If you author a product that can so easily help people cause harm, you probably should own some responsibility of the outcomes. OAI does not like this, hence the bill.
The US already messed this up with guns. Do they want to go the same path again? Answer: "probably, yes".
I read the anarchist cookbook 40 years ago that had similar info.
I think the info has been available for many years and the thing stopping terrorists wasn’t info.
Good luck on being on the list of people using chatgpt and claude to make neurotoxins ;)
I assume anthropic and ooenai are selling prompt logs to the fbi and other countries’ law enforcement for data mining.
> this is not how the model should behave
It's exactly how it should behave, without any prior overriding of system prompts.
Presumably they mean the fundamental failure mode of LLMs that if you fill their context with stuff that stretches the bounds of their "safety training", suddenly deciding that "no, this goes too far" becomes a very low-probability prediction compared to just carrying on with it.
as the context fills up, the model will generate based on that context, incl. whatever illegal stuff you've said, i.e. it'll mimic that, instead of whatever safety prompt they have at the top
they could make it more "safe" but it'd be much more invasive and would likely have to scan much more tokens also, and it'd cause false positives (probably the biggest reason it's not implemented)
I don't really know how these models really work, but I had a theory that just as the models have limited attention so do the safety layers. I simply populated enough context with 'malicious' text without making the model trip that "wasted" the internal attention budget on tokens early in the prompt completely ignoring all the tokens that were generated after the fact.
Models have a "context window" of tokens they will effectively process before they start doing things that go against the system prompt. In theory, some models go up to 1M tokens but I've heard it typically goes south around 250k, even for those models. It's not a difficult attack to execute: keep a conversation going in the web UI until it doesn't complain that you're asking for dangerous things. Maybe OP's specific results require more finesse (I doubt it), but the most basic attack is to just keep adding to the conversation context.
that 1M context thing, I wonder if it's just some abstraction thing where it compresses/sums up parts of the context so it fits into a smaller context window?
You don’t normally compress the system prompts, though I guess maybe it treats its own summary with more authority. This article [0] talks about the problem very well.
Though I feel it’s most likely because models tend to degrade on large context (which can be seen experimentally). My guess is that they aren’t RLed on large context as much, but that’s just a guess.
[0]: https://openai.com/index/instruction-hierarchy-challenge/
When my brother started to study Chemisty, he was told a) that it was easy to make meth b) the profit he would make and c) that the police would no doubt catch him, as only university students would make meth so pure.
By the time he was done, he knew enough to commit mass murder in half a dusin different very hard to track ways. I am sure doctors know how to commit murder and make it look natural.
My brother never killed anyone, or made any meth. You simply cannot have it so that students don’t get this type of knowledge, without seriously compromising their education and its the same way with LLMs.
The solution is the same: punish people for their crimes, don’t punish people for wanting to know things.
> The solution is the same: punish people for their crimes, don’t punish people for wanting to know things.
The LLMs aren't being punished for wanting* to know things.
The problem for LLMs is, they're incredibly gullible and eager to please and it's been really difficult to stop any human who asks for help even when a normal human looking at the same transcript will say "this smells like the users wants to do a crime".
One use-case people reach for here is authors writing a novel about a crime. Do they need to know all the details? Mythbusters, on (one of?) their Breaking Bad episode(s?) investigated hydrofluoric acid, plus a mystery extra ingredient they didn't broadcast because it (a) made the stuff much more effective and (b) the name of the ingredient wasn't important, only the difference it made.
* Don't anthropomorphise yourself
Ironically, it reads to me like they talking about the users wanting to know things, not the LLM.
Yes fortunately it is really bad at actually making novel bioweapons or syntheses in general so whatever you made probably wouldn't do more than give someone a mild headache.
I am not so sure about that, trial and error can produce very dangerous results especially over the span of years or even decades.
if you're a layman "trial and error"ing bioweapons off chatgpt, you're not going to be around for decades
That's the question ain't it? Is it capable enough to keep you alive?
who said it has to be novel?
Yeah but you can just look that stuff up
Hell, I got Sonnet to write some light content that gets a 100% Human score on Pangram with no effort. That’s way more concerning to me, IMO.
Chinese OSS models will do this in a few months.
So, regardless of whether you think it's great that Opus gives this info, we need better solutions than legal liability for US corporations. When the open models have the ability to do damage, there's nobody to sue, no data center obstruction that will work. That's just the reality we have to front-run.
The knowledge is one thing. But the competence of execution and the will to act are difficult to line up.
Yes there should be safe guards, but after a while you're jumping at shadows.
I'm more worried about depressed kids getting on chat and being encouraged to kill themselves than terrorist attacks.
We know what a cancer algorithmic social media is yet we don't act.
I doubt there will be any real and serious opposition to this bill, but there should be.
in 5th grade, i was able to access and read the anarchist cookbook. thanks to the internet. so idk
Countless downloadable models (including de-aligned mainstream models) can do this.
None have had the capability to provide me with instructions that have this high of accuracy including the suggesion of completely novel chemical reactions. I am not a chemist so I can't back it up, but if an AI can solve mathematics it's not unreasonable to say that they can also solve creating new neurotoxins en masse.
> I am not a chemist so I can't back it up, but if an AI can solve mathematics it's not unreasonable to say that they can also solve creating new neurotoxins en masse.
Right now it kinda is.
LLMs can do interesting things in mathematics while also making weird and unnecessary mistakes. With tool use that can improve. Other AI besides LLMs can do better, and have been for a while now, but think about how available LLMs in software development (so, not Claude Mythos) are still at best junior developers, and apply that to non-software roles.
This past February I tried to use Codex to make a physics simulation. Even though it identified open source libraries to use, instead of using them it wrote its own "as a fallback in case you can't install the FOSS libraries"; the simulation software it wrote itself was showing non-physical behaviour, but would I have known that if I hadn't already been interested in the thing I was trying to get it to build me a simulation of? I doubt it.
Well the worst outcome is that you make something deadly which is what you are creating anyway, do that for a year and you could possibly produce a very deadly substance that doesn't have a known treatment.
"Worst" outcome assumes it's easy to give an ordering.
Which is worse, (1) accidentally blowing yourself up with home-made nitroglycerin/poisoning yourself because your home-made fume hood was grossly insufficient, or (2) accidentally making a novel long-lived compound which will give 20 people slow-growing cancers that will on average lower their life expectancy by 2 years each?
What if it's a small dose of a mercury compound (or methyl alcohol) at a dose which causes a small degree of mental impairment in a large number of people?
If you're actually trying to cause harm, then your "worst" case scenario is diametrically opposed to everyone else's worst case scenario, because for you the "worst" case is that it does nothing at great expense.
Right now, I expect LLM failures to be more of the "does nothing or kills user" kind; given what I see from NileRed, even if you know what you're doing, chemistry can be hard to get right.
As someone who also watches NileRed of course it is hard, but AI can give you solutions that normally you wouldn't be able to come up with due to lack of knowledge or/and education.
And to clarify, by 'worst case' I meant that you're already trying to create a deadly compound, worst that can happen is you kill yourself which was already an accepted risk by the user.
Isn't the biggest problem with creating neurotoxins not poisoning yourself while doing it?
I have a hard time believing that you’re the only person who has figured out Claude’s next generation ability to do computational chemistry and computer aided drug design. The AlphaFold folks must be devastated.
> it's not unreasonable
It in fact is. Do you often go around making claims you are entirely unqualified to make? Or is this something new you’re trying?
chemical reactions can be expressed via mathematical functions and it has been made very clear that these models can do advanced mathematics.
And even if it doesn't work, at the end of the day you can work with a model to figure out what went wrong over-time gaining expertise in the field.
I tested something similar last week, still worked easily.
these LLMs will never be able to mitigate this unless they literally scan everything all the time and nobody is gonna want that.
besides, open source models exist now
"Announcing new and improved logics service! Your logic is now equipped to give directive as well as consultive service. If you want to do something and don't know how to do it—ask your logic!"
https://en.wikipedia.org/wiki/A_Logic_Named_Joe
Making knowledge illegal is a dangerous precedent. Actions should be illegal, not knowledge. Don't outlaw knowing how to make neurotoxic agents, outlaw actually trying to make them.
As for OpenAI immunity, I'm not sure I see the problem. Consider the converse position: if an OpenAI model helped someone create a cancer cure, would OpenAI see a dime of that money? If they can't benefit proportionally from their tool allowing people to achieve something good, then why should they be liable for their tool allowing people to achieve something bad.
They're positioning their tool as a utility: ultimately neutral, like electricity. That seems eminently reasonable.
1. LLMs don't just provide knowledge, they provide recommendations, advice, and instructions.
2. OpenAI very much feels that they should profit from the results of people using their tools. Even in healthcare specifically [0].
[0] https://www.wisdomai.com/insights/TheAIGRID/openai-profit-sh...
> 1. LLMs don't just provide knowledge, they provide recommendations, advice, and instructions.
That's knowledge.
> 2. OpenAI very much feels that they should profit from the results of people using their tools. Even in healthcare specifically [0].
If they're building a tailored tool for a specific person/company and that's the agreement they sign the people who are going to use with the tool, sure. I'm talking about their generic tool, AI being knowledge as a utility, which is the context of this legislation.
The point is valid, but that's typically the way it is. "You can't enjoy the benefit but the detriment is all yours" is how the federal government generally operates.
It's wild that this is being downvoted on HN. Facts should never be illegal or suppressed.
If you disagree you shouldn't downvote, you should refute in a reply.
The problem is: Until you go out and do a mass casualty event, unless you yourself are a trained professional, no one knows what you actually did.
You can buy books on how to make and obtain chemicals on your own.
Hell here's an Internet Archive book on making explosives
https://archive.org/details/saxon-kurt.-fireworks-explosives....
If you ever chat with older folks pre-90's much of this information was accessible fairly easily. It only changed with the push by the government to crackdown on Waco, Oklahoma City bombing, militias and other related groups. There was then a campaign to make it "normal" to limit free speech on the subjects, where as these books were available before.
I think the whole thing where AI should make information less available is a difficult battle and one which I personally oppose, but do understand. Free speech and information isn't the problem, it's the people, actions and substances they create.
After the age of the internet, I think it's been a forever loosing battle to limit information, it's why we couldn't stop cryptography, nuclear weapon proliferation, gun distribution, drug distribution, etc. The AI is just another battle ground, one which, if they actually do manage to control could definitely create some walls to this information, but not stop it.
More scary, is that the AI as it becomes pervasive and stop people from asking certain questions, because they don't know they should ask... but that's unrelated to the risk of mass death.
> The item you have requested had an error:
Item cannot be found.
which prevents us from displaying this page.
Clicking the link splits off the ".", which is interesting but necessary.
> neurotoxic agents from items you can get at most everyday stores
I mean, bleach and ammonia will do that. So I'm not sure that's really much of an accomplishment for AI.
I think you might be stretching the meaning of the term juuuuust a little bit.
You're not far from claiming that farting in a crowded elevator is a chemical attack.
Because if you didn’t already know that, like an immature deprived and desperate kid, being able to easily find out is really really bad..
Plenty of lazy AI apps just throw messages into history despite the known risks of context rot and lack of compaction for long chat threads. Should a company not be held liable when something goes wrong due to lazy engineering around known concerns?
> to lazy engineering around known concerns?
That implies that it is already illegal to provide this information. But is it? If a human did so with intent to further a crime, it would be conspiracy. But if you were discussing it without such intent (e.x. red teaming/creating scenarios with someone working in chemistry or law enforcement), it isn't. An AI has no intent when it answers questions, so it is not clear how it could count as conspiracy. Calling it "lazy engineering" implies that there was a duty to prevent that info from being released in the first place.
Very simply, if you provide a service for money you have a duty to ensure that service is safe. There’s a reason you have to sign a waiver when you jump on a trampoline, but companies are so rich the court cases have become parking tickets..
No, because that would indicate there should be some sort of regulatory standard for what does/does not constitute "lazy engineering". Creating this standard in turn creates regulatory/compliance overhead for every software engineering organization. This in turn slows everything right down and destroys the startup ethos. "Move fast and break things" is a thing for a reason. The whole point of the free market is to avoid this kind of burdensome regulation at all costs.
If customers want to buy "lazily-engineered" products, from where do you derive the authority to tell them they can't?
If airplanes used this logic, likely at least hundreds more would have died over the last decades. Accident rates are even going up, because of logic like yours. Yeah planes are fine most of the time, when the long tail involves safety concerns (that wouldn’t have otherwise happened) making money on people using your product becomes unethical without mutually agreed upon safety regulation, ideally motivated by voters instead of special interest groups
It went way beyond that, neurotoxins such as vx are heavy and linger around for a long time, just having a small amount of it placed in any metro (while trying to stay alive yourself) means the deaths of thousands of people. I am not even sure if it's legal to mention some of the uncategorized chemical solutions that it either hallucinated or figured out from relative knowledge.
Quoting the original bill [0]:
> "Critical harm" means the death or serious injury of 100 or more people or at least $1,000,000,000 of damages to rights in property caused or materially enabled by a frontier model, through either: (1) the creation or use of a chemical, biological, radiological, or nuclear weapon; or (2) engaging in conduct that: (A) acts with no meaningful human intervention; and (B) would, if committed by a human, constitute a criminal offense that requires intent, recklessness, or negligence, or the solicitation or aiding and abetting of such a crime.
I don't know what I expected from this title, but I was hoping it was more sensationalized. No need in this case unfortunately.
> (a) A developer shall not be held liable for critical harms if the developer did not intentionally or recklessly cause the critical harms and the developer: (1) published a safety and security protocol on its website that satisfies the requirements of Section 15 and adhered to that safety and security protocol prior to the release of the frontier model; (2) published a transparency report on its website at the time of the frontier model's release that satisfies the requirements of Section 20. The requirements of paragraphs (1) and (2) do not apply if the developer does not reasonably foresee any material difference between the frontier model's capabilities or risks of critical harm and a frontier model that was previously evaluated by the developer in a manner substantially similar to this Act.
However or if one thinks regulation for this should be drafted, I doubt providing a PDF is what most have in mind.
[0] https://trackbill.com/bill/illinois-senate-bill-3444-ai-mode...
I think my favorite part is that, because it only applies to "frontier models", if a smaller model is blamed for such harm, it seemingly doesn't immunize the creators at all. That makes very little sense unless you specifically want to make it illegal to not be OpenAI (et al).
Similarly, if a frontier model kills merely 99 people, they aren't covered by this. So go big or go home I guess?
> because it only applies to "frontier models", if a smaller model is blamed for such harm, it seemingly doesn't immunize the creators at all
Oof. If you're an Illinois resident, please call your elected and at least ensure they understand this loophole is there. In all likelihood, nobody other than OpenAI's lobbyists have noticed this.
> unless you specifically want to make it illegal to not be OpenAI [...]
If that is an "unintended" consequence, I am certain OpenAI wouldn't be opposed. Preventing competition whilst keeping any potentially profit risking regulations at bay has been a clear throughline in OAIs lobbying efforts.
But here it seems to be an anti- competitive move for market entrants who haven't made it into the big league yet...
Sounds like the saga for some players pushing for Biden's EO 14110 but this time at the state level?
That doesn't say much other than the rules are over in section 15.
To be protected they not only have to publish their security protocol, but adhere to it.
That's not just 'providing a PDF'
That particular section is entirely appropriate. A company can't do everything necessary to prevent every bad thing. They should do everything that they reasonably can. Someone else should decide what is reasonable.
The regulators are saying we've decided the what you have to do to be considered to have done all you could to be safe. Follow those rules, tell us how you've followed those rules, and if something bad happens and we find out that you didn't follow the rules you said we're going to nail you to the wall.
This hinges on Section 15. Which I think is inadequate because it does not meet the criteria of someone else deciding what is reasonable. Publishing their safety plans and adhering to them should be enough to grant protection from liability of harm directly to users, since the publication give individuals the ability to make an informed decision, provided they have done the safety work that they have said, a user deciding that is sufficient for them and choosing to use it should be allowable.
That should not extend to harm done to others. They don't get to choose. Consequently the standard required to be protected against claims of negligence has to be decided by a third party (experts hired by regulators ideally).
Blanket liability and blanket indemnity both go too far.
If someone makes a YoYo that blow's someone up because they made it out of explosives then they should be held liable.
If someone makes a YoYo that blow's up a city because it contained particles unknown and undetectable to any science we have, they shouldn't be to blame.
The key is that they have to have done what we think is required. Legislators get to decide what it is that is required. If a company does all of that, then they shouldn't be held responsible, because they have done all they were asked to do.
The problem is not that a law provides indemnity, the problem is that it sets the standard to qualify too low.
Shifting liabilities from corporations to the public coffer is what companies do. You'll often hear this described as "privatizing profits and socializing losses". Let me introduce you to the Price-Anderson Act of 1957 [1]. It's been repeatedly extended, most recently with the ADVANCE Act [2]. This limits liability for the nuclear power industry in a whole range of ways:
- It removes jurisdiction from state courts to the federal court. In recent weeks, the part of "states' rights" is doing similar to stop states regulating prediction markets, as an aside [3];
- All actions are consolidated into a single claim;
- That claim has an inflation-adjusted absolute limit, which is somewhere around $500 million (I'm not sure of the exact 2026 figure);
- Any damages beyond that are partially sharead by the industry and an industry self-funded insurance program;
- The industry as a whole has a total liability limit, also inflation-adjusted. I believe this is around $10 billion.
For context, the clean up from Fukushima is likely to take a century and the cost may well exceed $1 trillion for a single incident [4]. So if this happened in the US, the government would be on the hook for almost all of it.
So I have two points here:
1. If you oppose any effort to shift liability from AI companies to the government (as I do) with legislation such as this, how do you feel about the nuclear industry doing the exact same thing? and
2. Minor point but I noticed in searching for the latest details, Gemini made factual errors, stating that "the Act is set to expire in 2025" when it was extended in 2024 until 2045. Always check AI's work, people.
[1]: https://en.wikipedia.org/wiki/Price%E2%80%93Anderson_Nuclear...
[2]: https://en.wikipedia.org/wiki/ADVANCE_Act
[3]: https://www.pbs.org/newshour/politics/federal-government-sue...
[4]: https://cleantechnica.com/2019/04/16/fukushimas-final-costs-...
This is what government should be doing. Figure out how to do something safely, make that a regulation, then shield companies from liability as long as they follow that regulation. In practice you won't extract trillions of dollars from most companies anyways, because they'll go bankrupt long before they manage to pay all that back.
It's the "guns don't kill people" equivalent for AIs.
---
Before the pitchforks and downvotes:
- yes, it's a deliberate simplification
- yes, the issue is complex because you can also argue that you can't blame authors of encyclopedias and chemistry books for bombs and poisons, so why would we blame providers of LLMs
- and no, this bill is only introduced to cover everyone's assess when, not if, LLMs use results in large scale issues.
Quite an appropriate analogy: gun manufacturers were sued for their responsibility in US mass shootings. They won, so the mass shootings continue.
https://en.wikipedia.org/wiki/Protection_of_Lawful_Commerce_...
Wait, you think that if we shut down the gun companies the mass shootings would stop?
Stop no, reduce yes. Same with how seatbelts didn’t end car crash fatalities. Eliminating drunk driving also wouldn’t end road fatalities, but outside of Wisconsin no one believes drunk drivers shouldn’t be in jail.
Those comparisons don’t make sense. There are millions of guns in the country, and cutting US manufacturing wouldn’t change that any time soon.
> any time soon
Mandating seatbelts on new cars also did not immediately stop people from flying through their windshields.
In fairness, a well designed and tested weapon at least can be expected to reliably and consistently perform the same thing each time. We also understand deeply how they work and can easily investigate if something happens whether it was user error, a defect or design issue. LLMs, not so much.
This dodges the moral argument behind "guns don't kill people", which is worth confronting directly. I think people can reasonably disagree about whether second/third/fourth/etc. order effects carry moral/legal responsibility.
In light of such disagreement, and given the lack of any higher authority among free, equal, people to arbitrate it, the only reasonable way to coexist peacefully is to avoid imposing your ideas on others. This is the foundation of a liberal society.
“Guns don’t kill people” is actually a pretty strong argument for regulating who can access firearms. Things like red flag laws and background checks.
My first thought was that this must be related to the automated weapons issue that got Anthropic on Trump's shitlist. It makes sense that a company that will eventually be asked to build weapons that choose their own targets will want to limit liability when it will inevitably kill the "wrong" person.
Also, I am disturbed by the fact that in all the discussions on this topic during the last month, no one has mentioned the magic word "Skynet". This is clearly a terrible idea. And if a company needs immunity from liability, they know it is a terrible idea.
Skynet's flaw wasn't that it killed humans. It was a military machine specifically designed to kill humans. If it only killed "the enemy", it would have been hailed a marvelous success. It was only considered a failure because it killed the wrong humans.
As an Iowan, this reminds me a lot of the bill that's been pushed through my state's senate twice now (as recently as last year), which would prevent Iowans from filing lawsuits against pesticide and herbicide companies if those companies follow the EPA's labeling guidelines. The bill passed the senate both times, only stopped because the house declined to take it up.
For context, Iowa has the fastest growing rate of new cancer diagnoses in the country, and the second highest overall cancer rate.
Honest question, isn't that like OK?
Like if you have a product, and the government says the product is ok, and it's labeled per regulation and later that product turns out to be deleterious to people's health should the company be liable?
Guess we should already have precedent but my google-fu is failing here. I can't seem to find the resolution of Felix-Lozano v. Nalge Nunc , Felix sued Nalgene over their use of BPA which at the time was not illegal to use in the bottles.
PFAS will probably be the next battleground here. They've been used in lots of products. And have some lawsuits https://www.cbsnews.com/news/firefighters-pfas-lawsuit/ . In your opinion should every manufacturer of a product that uses PFAS be legally liable?
I'm not a lawyer, nor a judge, so I can't say. All I can tell you is that it feels wrong that [Monsanto/OpenAI] can lobby a state's legislature to prevent you, the average joe and potential lucrative victim, from filing a lawsuit against them when it seems clear to any reasonable person that people are developing [cancer/mental health issues] due to the use of [pesticides/AI].
Perhaps something like anti-SLAPP rules for the ignominious corporations would be a happy middle ground? I don't know if that would "fix" anything – or if there's anything to fix – so don't take that as a super serious suggestion.
If companies can't lobby, so many "safety" regulations will pass that you will simply suffocate and kill private industry.
This is why most promising drug candidates never see light of day.
I'm generally open to lobbying, but I'm not generally open to "you can never file a lawsuit against the comically evil pesticide corporation standing behind us twirling their mustaches." There needs to be a middle ground.
I don't think itd be ok, personally. My impression is regulations and regulatory institutions can be very slow to evolve after technological advances, unless the government is financially liable. A scheme I would be more comfortable with is mandatory insurance and insurance companies with a financial incentive absorbing the liability. On top of that probably add some bare minimum regulatory requirements/certifications.
"Like if you have a product, and the government says the product is ok, and it's labeled per regulation and later that product turns out to be deleterious to people's health should the company be liable?"
Mesothelioma is the precedent.
100% yes. If you've never seen the hell that people go through with these cancers, you are blessed, but it is hell, especially in the US medical system.
> Like if you have a product, and the government says the product is ok, and it's labeled per regulation and later that product turns out to be deleterious to people's health should the company be liable?
But like, what if you like, totally bribed the shit out government people and like totally fabricated scientific evidence to make it seem like it was safe but then you sold it anyway?
Aren't you then like a total piece of shit?
It's going to be okay. Stop watching movies and thinking they correlate to real life.
Cigarettes and asbestos are just two examples of where this absolutely did happen with plenty of public evidence, including successful legal cases.
Damn. Wish you'd told me that before I finished Schindler's List and Angela's Ashes.
> Iowa has the fastest growing rate of new cancer diagnoses in the country, and the second highest overall cancer rate
Iowa also has a lot of farmers spraying pesticides and herbicides. This feels like genuine political competition between local business interests and public health concerns.
Normally I would agree with you, but the primary lobby behind both of the bills was Bayer (née Monsanto), with backing from several of Iowa's industrial farming organizations. They launched a giant ad campaign to "control weeds, not farming" alongside their bill to influence opinions. Cancer, nitrates and pesticides are at the top of everyone's mind in the state these past couple years, so having the pesticide giant try to swoop in and put a bill in place that would prevent Iowans from suing them feels like that same kind of seagulling behavior you described in another comment.
> This feels like genuine political competition between local business interests and public health concerns.
You just described the US at large.
The evidently extremely difficult decision between making money for a few, or making life better for everyone.
> You just described the US at large
I described any democracy in a society with private property. Even without private property, you will have issues with concentrated benefits and diffuse harms–negotiating that is part and parcel with governance.
Iowa businsses petitioning their cause is one thing. OpenAI seagulling in to take a shit in Springfield strikes me as being categorically different.
Viewed from a country with universal healthcare , 18 month’s maternity leave ( my partner just used it ) and so much more, it feels like comparing the US to “any democracy” is like comparing rocks to gold.
> viewed from a country with universal healthcare...it feels like comparing the US to “any democracy” is like comparing rocks to gold
Do most democracies, extant or across history, have universal healthcare? You're comparing a policy to a governance structure.
All developed ones do, yes.
Except one of course.
We built systems we don’t fully understand, so naturally the next step is… immunity
From liability!
If this were to actually happen I can only imagine financial liability is the least of their concerns?
What scares me most about this is the narrowness of thought to match this fear with this response.
fully agree, doesn’t really feel like they’re reacting to the same problem they’re describing
Am I alone in thinking this is easy?
The human making the decision is always liable.
What if the human couldn't reasonably know better? Doesn't matter - If they made the same decision without AI or with old files it is still on them.
What if there's no single human decision? Someone is in charge and is responsible. The "I was ordered to" isn't a defense.
Does liability without power make sense? People executing have the power to execute. So liability. If they're executing without power that is a different liability, but a liability.
It may let the powerful off the hook - That is already a theme and AI doesn't change that, in fact, it will just be used as another scapegoat.
God told me to do it - Water tight! Right?
Let's say I start an AI program and my initial prompt is "Copy these files to this other computer", and then 100 iterations down the agentic loop the AI decides to hack into Tesla's FSD and ships an update that kills 500 people.
Who is liable?
> Someone is in charge and is responsible.
This doesn't seem to be a given.
There's always a CEO or a president. The buck always stops somewhere. Somebody is always making the big bucks because they are in charge.
For legally incorporated companies. What if it's just a crypto wallet hooked up to a network of prompts? Are you sure a human must have created it?
Somebody is collecting those funds. Someone wrote those prompts. There's always a human to blame somewhere.
I'm not aware of any counter-example, but I also don't know any reason why this must be true. And furthermore, I would expect that this will get more likely over time.
It's possible, in theory, that an AI could establish a crypto wallet, but what would they do with it? AI doesn't have desires. It doesn't do what it isn't told to do (although those instructions can be broad and vague). Even if an AI did somehow do something bad without being told, that AI would still be set up by a human and running on some human's hardware and using a human's internet connection.
Maybe in the distant sci-fi future we'll have actual AI (not just glorified chatbots) and AI will be able to decide for itself what it wants to do with its time and we'll be allowing AI to sign leases on property and set up accounts with utility companies, and if that day comes we're going to have a lot of problems if we're not ready for it, but until then it's AI on a human's hardware at a human's property running up a human's electric bill.
I'm sure you can pay for hosting somewhere without being a human.
It won't need to sign a lease to do any of this. It doesn't even require desires. I'm not sure why it seems so far fetched to you.
I have a copy of the weights on my HD and to my knowledge, it hasn't spontaneously went out and acquired web hosting and stood up a web site.
That doesn't seem to be something any AI company says is currently possible.
If things change, and AI becomes able to act on its own initiative, then it will be easy to change this law
I think it's just a gap in definitions. The labs say models don't act on their own initiative. What counts as initiative? I guess an API call in a for loop would count.
Historically it seems like a lot of laws haven't been easy to change. Especially when they regulate zillion dollar industries.
Illinois also has a Bill in committee right now to mandate operating system level age verification. There are lots of bad ideas to be upset about this year. If you are an Illinois resident, email your representative about HB 5511 today. Stupid legislation like this passes because we don’t speak up. Find out who your representative is, find their email, tell them your opinion.
I forget, wasn't OpenAI the company that was formed as a nonprofit to limit the risks of LLMs? Founded by a bunch of visionaries scared of what they had wrought and anxious to lead so they could make sure it was only used responsibly?
That was before it was discovered that these LLM have incredible monetary potential.
Not really. The entire premise of the structure was that obviously AI would be immensely valuable and that they needed binding contract structures to prevent themselves from falling victim to the greed and ambition that would obviously consume those at the helm.
Unfortunately their contract structures weren't strong enough to protect from the combination of the "king of the cannibals" and completely absentee regulatory oversight.
The writing was on the wall when they feigned horror at an early GPT being able to play poker in the 2010s, and failed to release the model
I think Mr Altman had this idea from the beginning, and in his own, "can't stop lying" way, he lied.
Trust that they already knew long before, and that this was the play all along.
And if you don't believe that, do some digging into the lives of the psychopaths that started it.
LLMs themselves don't. But selling LLM related bullshit to investors has.
Rule #1: Never trust a company with "Open" in only their name and is anything but, in real.
Might as well complete the loop, rename the company the Democratic People’s Republic of OpenAI LLC
:)))))) laughed hard at that one. Wish HN had smiley support.
Everyone has a price.
Come on now. Let's not pretend it wasn't a grift from the start.
Don't feel bad, they forgot too.
They were lying through their nose even then?
Yeah the whole “rationalist” movement is full of those lying fks that use a thin veneer of fallacious logic and self aggrandising discourse to rationalise their hoarding of resources and bottomless greed. They’re very well established in Bay Area and AI world.
Is Sam even a rationalist, or describe his views as rationalist?
Depends.
Would it lead to increasing his wealth?
He was not the founder of OpenAI
nobody was "the" founder of OpenAI. Sam was one of many cofounders, though I don't see how the particulars here are relevant to the point above.
There are several points above:
> wasn't OpenAI the company that was formed as a nonprofit to limit the risks of LLMs?
>> the whole “rationalist” movement is full of those lying fks
>>> Is Sam even a rationalist, or describe his views as rationalist?
The relevancy is that a question of how and why the company was formed isn't fully answered by only talking about his motivations.
I see I see. Makes sense!
Not directly, but very friendly to the movement and people in it.
Looks to me like the rationalist/AI researcher/EA cohort of (admittedly odd) people was quite deliberately hijacked by a sociopath
The movement itself is consistently aligned with Tech Bros interests, the philosophical foundation is interesting, but the movement itself is quite problematic
Not really, when OpenAI was formed in 2015 there were no LLMs, at least none that worked well. It was a regular AI research lab mostly doing Reinforcement Learning on game environments like Atari similar to DeepMind. Once they struck gold with LLMs (2019 or so?) and saw there is money to made everything changed, as expected when a bunch of SV types get involved.
Let’s see how long until this is flagged off the front page. I’ll put the over/under at 1 hour from the posted time
It's not removed, but they changed the title to "OpenAI backs Illinois bill that would limit when AI labs can be held liable". The actual bill text explicitly mentions that it excludes liability for "the death or serious injury of 100 or more people or at least $1,000,000,000 of damages to rights in property caused or materially enabled by a frontier model" (https://legiscan.com/IL/text/SB3444/2025) so I am not sure why the title was changed. The original title of "OpenAI Backs Bill That Would Limit Liability for AI-Enabled Mass Deaths or Financial Disasters" seems like it accurately describes the bill.
1 hour? I'd go with 3 months.
(Reason is if it is not flagged the spread bet pays out for life!)
It won't be, because it is marketing, and we are eating it up.
Update: off the front page somewhere between 1 and 4 hours
This seems par for the course for OpenAI/Sam Altman.
Unfortunately they are not the first company to try and externalize their costs, and they will not be the last.
Serious question, maybe a bit naive: Is there anything we can do to push back against and discourage the externalization of costs onto others?
Is this simply a matter of greed and profit-seeking outweighing one's morals (assuming one has them to begin with)?
Change the legal definition of corporations? Corporations exist to provide liablity protections to sharholders, which means they are mainly incentivized to externalize costs and avoid liability to maximize profit, or even to make profit in businesses that would not be profitable if they could be held liable for externalized costs (deep sea oil well drilling). Limit the ability of corporations to shield themselves from view through multiple levels of shell corporations and Special Purpose vehicles. These are probably controversial stances on a board about startup culture and breaking the rules to get rich.
Stop voting for people and judges that believe in the Friedman doctrine?
Every decision has tradeoffs. Western society has largely decided to prioritze capital owners over everything else.
> Is there anything we can do to push back against and discourage the externalization of costs onto others?
On a societal scale, no. Occasionally this works in some individual cases. Like the online outrage over SOPA/PIPA 15 years ago.
But when entity X can gain $$$$$$ (or power) from doing an action, and that action costs everyone only $ (or a minor bit of inconvenience or ideological righteousness), then the average person has very little incentive to take time out of their day-to-day life to fight it.
Meanwhile the entity will do whatever it takes to get the $$$$$$/power because they have a huge incentive. This is the same mechanism that allows democracies to be eroded, as we're seeing right now in the US.
Even if they were to pass such a law which would be political suicide, it would still be up to the courts to say that it doesn't violate the Constitution. For example, a law that says anyone with a net worth of $1B can freely punch anyone in the face whenever they want and have immunity would be a clearly illegal law. That's basically what this bill is. The courts would then need to be made sufficiently corrupt to not strike down such a law as unconstitutional.
Unconstitutional doesn't mean much when it's being decided by a group of unaccountable people that weren't elected through democratic means. If SCOTUS says something is legal, it's legal. That's how the system is setup, nothing else really matters. They'll justify their decisions however they want but the material ends are the only things that matter.
SCOTUS has ruled many terrible things over the course of our nation's history (upheld slavery, said slaves weren't people, equated money with speech, decided a presidential election while denying a recount, etc). Expecting them to somehow be better is a foolish task.
It's an institution that needs to be dismantled and rebuilt, where at minimum SCOTUS appointments should be elected by a national vote rather than letting an extreme minority decide (100 senators versus ~340,000,000 people).
That depends on your definition of "we". As a society, we can regulate companies and punish the offenders (e.g. don't dump toxic waste into sources of drinking water or you'll get prosecuted). As individuals, there's not much we can do directly. How to translate individual actions into societal action is kind of the fundamental question of civilization, and if there's a uniform solution for how to achieve it, I don't think we've managed to come up with it yet.
A lot of people will dismiss this answer, but... vote for Democrats. With Bernie and upcoming young Democrats more and more are pushing back. The parties definitely are not the same. Democrats created the Consumer Financial Protection Bureau. Republicans destroyed it.
Push your representatives to crush monopolies and manipulative practices. This happened before in the gilded age. Only a popular response can turn the tide.
Also, primaries are coming up, and not all Democrats are the same either. Plenty of the old school Democrats are facing progressive challengers. So, vote for the ones that will stand up to this garbage and follow up on whether they do. There are a lot of new faces in the Democratic party who are standing up to the BS.
The US has a lot of potential to change if we push it. A 25 point swing toward people who don't consider grift a personal priority will change a lot of things.
OpenAI wants to not be responsible for "accidents" that kill more than 100 people, despite some critics arguing that their current actions are likely to cause such harms.
So much for the "Our mission is to ensure that artificial general intelligence benefits all of humanity." I was naive to hope that now such laws would ever pass
They only care about benefiting what's left of humanity when they're done with it and it'll probably just be them and their cronies.
Have the sponsors of this bill stated what the public benefit of providing these immunities would be? Just “more models, more progress, go faster?”
I think there’s room for nuance but I don’t see how this could possibly be construed to be in the public interest.
It's the tech version of ag gag laws and liability-protection laws for pesticides.
Take all of the data, take all of the credit, take all of the money, and none of the blame.
That would be a better mission statement for OpenAI at this point.
socialize the losses/risks, privatize the profits.
If an OpenAI model helped someone create a cancer cure, they wouldn't see a dime from that beneficial act. So why should they be liable if someone does something harmful with the model?
If an OpenAI model helped someone create a cancer cure I guarantee that they would try to profit as much as possible from that fact. They have even talked in the past about having partial ownership over discoveries made with AI be part of the license. They would be all over that.
I'm sure if they could, they would, sure, as would any business. That's where competition enters the equation. They can't do it because their competitors would undercut them by requiring no such conditions.
You don't think they would use that fact to promote themselves?
Sure they would, just like people would use the bad PR to smear OpenAI if someone did something bad with knowledge their model created. The situation is totally symmetrical and fair as it is, and my point is that expecting them to liable is asymmetric and unfair. If they can be held liable, then they should also be able reap the rewards in order to offset those risks.
This is what I'd expect from companies - I don't see why Facebook would get money because they helped people connect to each other who ended up developing a cancer cure, but they definitely should be held accountable for enabling a genocide. You're allowed to operate a business until you cause harm to society, then we can shut it down.
I think the big thing you would need is to see the internal emails - if there was ever a case where someone raised a concern about this possibility and it wasn't taken seriously, then they should be liable. If they just never thought about it then it could be negligence but I think if I was on a jury I'd find that more reasonable than knowing it could be a problem and deciding you aren't responsible
> I don't see why Facebook would get money because they helped people connect to each other who ended up developing a cancer cure, but they definitely should be held accountable for enabling a genocide.
Why? What does it even mean to "enable a genocide"? Just saying something isn't an argument.
> if there was ever a case where someone raised a concern about this possibility and it wasn't taken seriously, then they should be liable.
Again, why? How is this any different than electricity as a tool, which has both beneficial and harmful uses? AI is knowledge as a utility, that's the position here.
What a classic!
That’s just the slogan of the US.
I think you've just summed up late stage capitalism.
What a time to be alive.
People on both sides seem to give capitalism a lot of credit for human traits that existed long before capitalism.
Well the point of capitalism (going back to Adam Smith) is that the invisible hand converts locally selfish behavior to globally good outcomes. The argument is whether or not that emerges. So if your implication was that human trait was selfishness, yes, that is quite the point of capitalism.
I think you could apply this to most, if not all, billionaires in the world today.
I am not sure what the other side of this argument looks like: Unlimited liability (i.e. liability no matter how poor an implementation and use of the tech is)?
The would be quite a novel burden, that no other tech (afaik) had to carry so far. We always assumed some operator responsibility. It's interesting to think of AI as a tech that could feasible be able to internally guardrail itself, and, maybe more so with increasing capability, no human can be expected to do so in it's stead – but, surely, some limits must apply and the more interesting question is what they are, as with any other tool?
Every other field in history considers it de rigeur that you're liable for the failure of quality in the products you produce. You make drugs that hurt people? You're liable. You build a building that falls down? You're liable. You serve coffee that literally burns the people drinking it? You're liable. It's also not new--the Code of Hammurabi (some 6000 years ago) prescribes the death penalty for people who build houses that fall down and kill the inhabitants inside.
It's only computer scientists who think it's some unreasonable burden to be held liable for the consequences of their work.
It is an unreasonable burden to ask the impossible. The technology to create an AI incapable of hurting people if misused or blindly trusted literally doesn't exist right now.
It'd be like holding a builder liable for their bridge being unable to withstand being hit by a meteor.
If I tell someone to kill someone else and they do, then I should be held responsible.
If I write instructions in a book that I give to someone telling them to kill someone else and they do, then I should be held responsible.
If I give someone a tool I made that I bill as more-than-PhD-level intelligence and it tells someone to kill someone else and they do, then I should be held responsible.
All of the above situations seem equivalent to me; I'm not the only person responsible in each case, but I gave them instructions and they followed them.
Man, this debate is both very interesting and highly confusing with different people oscillating between "It's just a tool" and "It's a human being+"
It is a tool, but it's a tool that is sold by OpenAI as providing a high degree of intelligence. That's an endorsement of what the tool outputs as advice, which is what makes them responsible.
> That's an endorsement of what the tool outputs as advice
That's not even close to true!
Even if you've been living under a rock for the last 5 years and didn't already know these models are not reliable, pretty much every provider has a disclaimer next to the chat box informing you of that fact.
A small disclaimer under the main flow that also acts as cookie banner doesn't outweigh the many, many other statements claiming capabilities. It's a minor undercutting, sure, but it's perfectly possible to have all sorts of disclaimers [0] while still keeping the point clear.
[0] https://www.youtube.com/watch?v=KivCRqfFcqY
People who cause death from either action or inaction are criminally liable for it, that's the other side of it.
So they did the math and worked out it's cheaper and easier to lobby the government instead of working to make their product safe.
And these are the people that a lot programmers want to give the keys to the kingdom. Idiocracy really is in full effect.
That has been the methodology of businesses since the mid 20th century. And you know what? It works! Really well!
Works for who exactly?
> instead of working to make their product safe
Make a nondeterministic product safe how?
Is this the first time you have heard of AI safety?
Lots of articles you could read on the subject and answer your own question.
(Unless your angle is: akshually, you can never make anything 100% safe)
> akshually, you can never make anything 100% safe
Yes Sherlock. And especially a natural language product that can't output the same thing for unchanged input twice.
Besides when you say "safe" i think of the idiots at Anthropic deleting "the hell" when i pasted a string in Claude and asked "what the hell are those unprintable characters at the beginning and end"...
How many correct answers did they suppress in their quest to make their chatbot "family friendly"?
> Yes Sherlock
I figured as much. I don't think it takes Sherlock Holmes to identify akshually types, but thanks for the compliment, I suppose.
What exactly are you implying? It sounds to me like you're saying that if it's impossible to make a product safe, then there shouldn't be any safety requirements. I think a more sensible position is that if it's impossible to make a product safe, then it should be illegal to build.
I'm creating a new start up called QuantumFlop Electricity - there's a 10% chance it will cause a black hole to open up in the Atlantic Ocean that may eventually consume us all but a 50% chance we'll have unlimited clean energy. We'll never know for sure if at any point that black hole may open as it's borrowing energy from the 81st dimension, but the upside seems pretty good.
Should I be able to get on with it?
Funny, I was just rereading the Hyperion series. It says there clearly that it was the AIs that created the black hole that led to the destruction of Old Earth. Intentionally.
I've never read the book but I assume there were some sycophants who were praising the AI right up until it created a black hole that destroyed the planet.
Great space opera, try it if you like the genre. It wasn't exactly like that but I've already spoiled it too much.
No different to preventing game studios being liable for mass shootings. Reminds me of the post-Columbine hysteria where media was super critical of Doom and Nine Inch Nails.
Except that there was no link between violent video games and mass shootings. It has been shown many times. The families blamed the video games despite there being no direct evidence of it. The games did not tell them they should go on a mass murdering spree any more than porn told you you should sleep with your cousin.
In this case there are several documented cases where the AI did over a long period of time progressively gaslight the individual and persuade them to commit suicide.
It’s “no different”? Please spend some time off of the internet to reflect.
Would Doom give a sociopath instructions on how to commit a mass shooting if asked? I don't remember that part of the game
If you experience the world as encounters with a series of demons, you might infer it.
OpenAI has now officially absorbed the Facebook/Zuck's ethos of 'Move fast and break things' no matter if it's society itself .. as long as their share prices "go up".
They even hired former infamous FB staff and have been in the last months employing the same 'engagement' (addictive) product patterns.
So they want protection from harms caused by their own models. Classic move — lobby for the rules while you're still ahead of regulators who don't fully understand the technology yet. Would be interesting to see what happens when a state actually pushes back hard.
Presently it appears that a state attempting to push back will get stomped on by the feds: https://news.bgov.com/bloomberg-government-news/colorado-gov...
The thing that bugs me the most about OpenAI are not the AI-enabled mass deaths. It's the hypocrisy.
Related: https://www.transformernews.ai/p/ais-next-big-blue-battlegro...
Yep, this is everything wrong with AI in one easy to protest package, but do keep going on and on about the evils of datacenters, how they're coming for your jobs, and that AI art isn't art. That's really winning hearts and minds!
They're not unimportant. It seems like very few people consider how much we're fucking up the ecological system that we need. But sure, money important and stuff.
Same people whinging about these "concerns" happily embrace their enormous personal carbon footprints on every other axis. Color me unconvinced and unimpressed until they're the Ed Begley they want to see in the world. I'll wait...
Sure and Google, FaceBook and Twitter support section 230 that gives them cover for hosting others content.
A company backing legislation that takes liability off them is something that they will always do.
My entire company switched from open ai to entropic after the Department of War idiocy that happened a few weeks ago.
Anthropic isn’t perfect by a long shot but at least they stand by a couple morals.
That whole fiasco actually soured me on Anthropic. They were clearly super desperate to take blood money. "Anthropic has much more in common with the Department of War than we have differences."
Without getting even more eyes on me, these company boards are inadequately scared for their personal safety.
If sufficient numbers of the population perceive to have lost their livelihoods due to AI, then I'd expect to see data centers burned to the ground and a lot of people swinging from lamp posts. Jury nullification solves the rest, but even that that assumes you can even find an impartial jury.
Having worked for OpenAI will be the new "MindGeek" on LinkedIn.
By "back Illinois bill", does that mean they wrote it?
OpenAI continually fails to understand that liability is there to protect their users AND OPENAI. If OpenAI causes significant harm, and the victims are told they cannot even sue to be made whole, what exactly does OpenAI think will happen? That the victims will just go pound sand? People will demand justice, and if that can't be delivered via the legal system either the system will be changed, negating this lobbying effort, or the system will by bypassed.
Skynet begins learning at a geometric rate.
Is this for like military scenarios or like, ChatGPT designed a drug that seemed to work, but people died by the millions 5 years later? Because they should 100% be liable for the latter. The former, good luck trying to prosecute an AI company for something the military does. To an extent, the military would probably want their AI models to be behind their private network, completely firewalled from any public network. SIPRNet iirc. If they lock it down behind a highly classified network, good luck figuring out how they're using AI.
> Because they should 100% be liable for the latter.
Why? I don't see that a drug designed by ChatGPT should result in any more or less liability than a drug designed by a human?
I think if a human designs a drug and tests it and it all seems fine and the government approves it and then it later turns out to kill loads of people but nobody thought it would... that's just bad luck! You shouldn't face serious liability for that.
If we start from the position of the marketing hype and even Sam Altman's statements, these tools will "solve all of physics". To me it's laughable, but that's also what's driven their outsized valuations. Using the output to drive product decisions and development, it's not hard to imagine a scenario where a resulting product isn't fully vetted because of the constant corporate pressure to "move faster" and the unrealistic hype of "solve all of physics". This is similar to Tesla's situation of selling "Full Self-Driving" but it actually isn't in the way most people would understand that term and so they lost in court on how they market their autonomous driving features.
> that's just bad luck
Can't agree with this. No, not at all. That can't be true... That's not "just bad luck". I believe this is actually a serious case of negligence and oversight - regardless of where exactly it occurred, whether on the part of the drug’s manufacturer, the government agency responsible for oversight, or somewhere else. It just doesn’t work that way. Any drug undergoes very thorough and rigorous testing before widespread use (which is implied by "millions of deaths"). Maybe I’m just dumb. And yeah, this isn’t my field. But damn it, I physically can’t imagine how, with proper, responsible testing, such a dangerous "drug" could successfully pass all stages of testing and inspection. With such a high mortality rate (I'll reinforce - millions of deaths cannot be "unseen edge cases"), it simply shouldn’t be possible with a proper approach to testing. Please, correct me if I’m wrong.
> I don't see that a drug designed by ChatGPT should result in any more or less liability than a drug designed by a human?
It’s simple. In this case, ChatGPT acts as a tool in the drug manufacturing process. And this tool can be faulty by design in some cases.
Suppose, during the production of a hypothetical drug at a factory, a malfunction in one of the production machines (please excuse the somewhat imprecise terminology) - caused by a design flaw (i.e., the manufacturer is to blame for the failure; it’s not a matter of improper operation), and because of this malfunction, the drugs are produced incorrectly and lead to deaths, then at least part of the responsibility must fall on the machine manufacturer. Of course, responsibility also lies with those who used it for production - because they should have thoroughly tested it before releasing something so critically important - but, damn it, responsibility in this case also lies with the manufacturer who made such a serious design error.
The same goes for ChatGPT. It’s clear that the user also bears responsibility, but if this “machine” is by design capable of generating a recipe for a deadly poison disguised as a “medicine” - and the recipe is so convincing that it passes government inspections - then its creators must also bear responsibility.
EDIT: I've just remembered... I'm not sure how relevant this is, but I've just remembered the Therac-25 incidents, where some patients were receiving the overdose of radiation due to software faults. Who was to blame - the users (operators) or the manufacturer (AECL)? I'm unsure though how applicable it is to the hypothetical ChatGPT case, because you physically cannot "program" the guardrails in the same way as you could do in the deterministic program.
> I physically can’t imagine how, with proper, responsible testing, such a dangerous "drug" could successfully pass all stages of testing and inspection.
It might cause minor changes that we don't yet know how to notice, and which only cause symptoms in 20 years' time, for example. You can't test drugs indefinitely, at some point you need to say the test is over and it looks good. What if the downsides occur past the end of the test horizon?
> ChatGPT acts as a tool in the drug manufacturing process. And this tool can be faulty by design in some cases.
ChatGPT is not intended to be a drug manufacturing tool though? If you use any other random piece of software in the course of designing drugs, that doesn't make it the software developer's fault if it has a bug that you didn't notice that results in you making faulty drugs. And that's if it's even a bug! ChatGPT can give bad advice without even having any bugs. That's just how it works.
In the Therac-25 case the machine is designed and marketed as a medical treatment device. If OpenAI were running around claiming "ChatGPT can reliably design drugs, you don't even need to test it, just administer what it comes up with" then sure they should be liable. But that would be an insane thing to claim.
I think where there may be some confusion is if ChatGPT claims that a drug design is safe and effective. Is that a de facto statement from OpenAI that they should be held to? I don't think so. That's just how ChatGPT works. If we can't have a ChatGPT that is able to make statements that don't bind OpenAI, then I don't think we can have ChatGPT at all.
>But that would be an insane thing to claim.
The trick is to make people behave like that without actually claiming it. AI companies seems to have aced it.
> It might cause minor changes that we don't yet know how to notice, and which only cause symptoms in 20 years' time, for example.
In that case, even if it leads to many deaths, it would be difficult - if not practically impossible - to hold anyone accountable, even if it were possible. However, such a turn of events is difficult, or rather, practically impossible to predict, don’t you think? I apologize for not clarifying this point in my original comment, but I wasn’t referring to delayed effects - I was referring to what becomes evident almost immediately (for example, let’s say “within a year and a half at most”) after the drug is used. Yes… I’m sorry, I just didn’t phrase my thought correctly. I apologize for that.
> ChatGPT is not intended to be a drug manufacturing tool though?
That’s certainly the case right now. However, LLMs like GPT, Claude, Gemini, and others weren’t created for waging war, were they? Then why did Anthropic recently have - let’s just say... "some issues in its relationship" with the DOD, if they were not involved in this, if Claude was not meant to be used in war? Why was the ban on using Gemini to develop weapons removed from its terms of service?
You’re right that LLMs weren’t created for such purposes, and to be honest, I believe that - at least for now - it’s simply unethical to use them for that. These aren’t the kinds of decisions and actions that should be outsourced to a machine that bears no responsibility - moral or legal.
> ChatGPT can give bad advice without even having any bugs. That's just how it works.
To continue my thought, this is precisely why I believe it is unethical to give LLMs any tasks whatsoever that involve human lives. There are simply no safety guarantees - not just "some", but none at all - aside from unreliable safety fine-tuning and prompting tricks. For now, that’s all we can count on.
> If OpenAI were running around claiming "ChatGPT can reliably design drugs, you don't even need to test it, just administer what it comes up with" then sure they should be liable. But that would be an insane thing to claim.
They don't claim it yet. And, as one person (qsera) mentioned below your comment:
> The trick is to make people behave like that without actually claiming it. AI companies seems to have aced it.
They probably won't claim exactly that "ChatGPT can reliably design drugs", just because of the possible consequences. But I'm almost certain there will be something similar in meaning, though legally vague - so that, from a purely legal standpoint, there won't be any grounds for complaint. What's more, they are already making some attempts - albeit relatively small ones so far - in the healthcare sector; for example, "ChatGPT Health"[1]. I don't think they will stop there. That's a business after all.
> if ChatGPT claims that a drug design is safe and effective
I have already said before that the OpenAI will not be the only one who should be held responsible in this case. The (hypothetical) user should also bear some responsibility, and in the scenario you described, the primary responsibility should indeed lie with them. That said, I may be wrong, but it’s possible to fine-tune the model so that it at least warns of the consequences or refuses to claim that "this works 100%". This already exists - models refuse, for example, to provide drug recipes or instructions for assembling something explosive (specifically something explosive, not explosives - I recently asked during testing, out of curiosity, Gemma 4 how to build a hydrogen engine - and the model refused to describe the process because, as it said, hydrogen is highly flammable and the engine itself is explosive), pornography, and things along those lines. Yes, I admit, it’s far from perfect. But at least it works somehow. By the way, if I’m not mistaken, many models even include disclaimers with medical advice, like "it’s best to consult a doctor".
In short, what I’m getting at is that the issue lies in how convincing the LLMs can be at times. If it honestly warns of the dangers of using it, if it says "this doesn’t work" or "this requires thorough testing", and so on, but the user just goes ahead and does it anyway - well, that’s like hitting yourself on the finger with a hammer and then suing the hammer manufacturer. It’s a different story when the model states with complete confidence that "this will definitely work, and there will be no side effects" - and user believes it; there should be some effort put into preventing such cases. But otherwise, yes, I think you’re right about the scenario you described.
And to conclude - I don’t think that when it comes to drug development, we’re talking about ordinary people or individual users. In the context of the parent post, it is implied (though I may have misunderstood) that ChatGPT would be used by entire organizations, such as pharmaceutical companies - just as LLMs in a military context are used not by individuals, but by the DOD and similar organizations. I think this shifts the level of responsibility somewhat. Because when OpenAI enters into a contract for the use of its product, ChatGPT, in the process of drug development and manufacturing, it’s kind of implied that ChatGPT is ready for such use.
[1] https://openai.com/index/introducing-chatgpt-health/
EDIT: I'm sorry that my reply is so long, I'm just trying to express all of my thoughts in one which is probably not a good thing to do. I would write something like a blog post about that, but there's a lot written about this topic already, so... Yeah, and I have also used translator in some parts because English is not my native language.
> it simply shouldn’t be possible with a proper approach to testing.
It just has to be delayed. Like many years after application. Or trigger on very specific and rare circumstances. Not likely in a trial, but near certain at a population scale.
Or both...
On top of that, If I remember correctly, this liability wavering also exist for Vaccines.
> It just has to be delayed. Like many years after application.
That's one thing. In this case, I don't really know if it's possible to test for something like delayed effects. I'm not even sure if you can identify them with 100% certainty; if you can prove that these effects come from this particular drug and not from another one.
> Or trigger on very specific and rare circumstances. Not likely in a trial, but near certain at a population scale.
And this is different thing. "Specific and rare circumstances" will not lead to millions of deaths (I apologize if I’m being too nitpicky about this particular phrasing, but I want to speak specifically in the context of “millions of deaths”). “Specific and rare circumstances” occur even with fully effective and "proper" medications - this is called “contraindications.” But such rare cases, as I’ve already said, will not lead to mass deaths - precisely because they are rare. I apologize again for focusing on the "millions", but please don’t confuse the scale of the problem.
> Because they should 100% be liable for the latter.
I completely agree with you here. I only want to add that in this case, the users (the one(s) who used ChatGPT to design the drug, whichever entity(ies) that is) should also be held liable for their actions.
Shouldn’t the pharmaceutical company be held liable for insufficiently understanding the drug before releasing it? I don’t think I understand blaming a tool used in the process of designing it and not those who chose to release it.
Pharmaceuticals are heavily regulated, the "we vibecoded a therapeutic and released it without testing" hypothetical has no basis in reality
> Is this for like military scenarios
Probably not. Weapons manufacturers are already well shielded from liability.
Why shouldn’t they be liable for military scenarios? Oh right, we don’t value our “enemies” lives, including their civilians.
Since when have arms merchants been liable for military scenarios? Lockheed doesn't get sued for building the planes that bomb orphanages. Maybe the world would be a better place if they did, but obviously it's not in the interests of a government to have their own contractors getting sued out of existence for something that government is doing.
Is there something equivalent in other industries that we can compare to?
This is the summary
>Creates the Artificial Intelligence Safety Act. Provides that a developer of a frontier artificial intelligence model shall not be held liable for critical harms caused by the frontier model if the developer did not intentionally or recklessly cause the critical harms and the developer publishes a safety and security protocol and transparency report on its website. Provides that a developer shall be deemed to have complied with these requirements if the developer: (1) agrees to be bound by safety and security requirements adopted by the European Union; or (2) enters into an agreement with an agency of the federal government that satisfies specified requirements. Sets forth requirements for safety and security protocols and transparency reports. Provides that the Act shall no longer apply if the federal government enacts a law or adopts regulations that establish overlapping requirements for developers of frontier models.
https://legiscan.com/IL/bill/SB3444/2025
I'm trying to think of an alternative bill. Imagine OpenAI came up with a model that when deployed in OpenClaw, allows you to spam people and this causes a huge disruption. Should OpenAI be liable for it? If this was not intentional and they had earnestly tried to not have this happen by safety protocols?
Yeah, I feel that in most scenarios the liability should lie with the user of the LLM. If the developers of the LLMs become liable we can expect a much more over active refusal system, and very likely a robust chat surveillance system that looks for patterns in user requests. And likely more gate keeping of the premier models.
https://en.wikipedia.org/wiki/National_Childhood_Vaccine_Inj...
it feels OpenAI know they've lost, and their only hope is getting saved by USA military complex. I have a more restrained opinion about other AI companies and LLM tech more broadly; but for OpenAI specifically I hope they go bankrupt sooner rather than later
Sam is working hard to confirm everything in that article.
The inevitable result of giving corporations and executives complete immunity from the harms they cause is that people will stop resorting to the legal system and begin resorting to extralegal measures.
And the likely result is that in most of the country those extralegal measures would have to be very extreme to secure a guilty verdict. You can see the beginnings of it now with the ICE protest trial verdicts.
A section 203 equivalent for AI is so important as it is one of the reasons all of the US companies have all of these usage restrictions and gives more reasons for them to ban your account since they want to minimize legal risk.
Holding tool manufacturers liable for how their tool is used provides bad incentives towards the users of tools.
To the extent that this is about knowledge, I don't think it's fitting in this age to hold any person liable for what another person does with knowledge they've been furnished.
On the other hand, to the (apparently zero, currently?) extent that this is about AI companies profiting from war and murder by deploying weapons that kill people without human intervention, then their liability seems to be not only civil but criminal.
Fortunately at any moment the virtuous non-profit will step in and make this all okay.
Of course they are, because the tech industry is run by ethical midgets and psychopaths, who shouldn't be allowed to own a dog but are in charge of trillion-dollar corporations getting shadow contracts from the pentagon.
The more I learn about tech and the people that build it, the more I yearn for the era of caves and pointy sticks.
"death or serious injury of 100 or more people or at least $1 billion in property damage"
They think their products will cause 9/11 scale events, and they shouldn't have to pay for it when they do.
BLOOD BLOOD BLOOD BLOOD BLOOD BLOOD BLOOD BLOOD BLOOD
Incredible.
Hey Americans,
Please just make sure when you let an AI decide to explode your own country and ruin your society, you leave the rest of the world intact, thanks
Another marketing gimmik...
This is why humans will still be necessary in decision chains: good luck getting anyone associated with AI to be provided with a real punishment when their models cause something bad to happen, or getting the executives who said "let's just have the AI do it" to take any responsibility.
Please note that you can not hold the Torment Nexus™ liable for any torment you experience.
Good that OpenAI is a corporation for the public benefit. Altman with his constantly fake worried look must be the most hated picture in existence. Please write articles without a picture or add a trigger warning.