Since Jacob Coxon’s posts in September, Americans have grappled with AI risk. Last month, a POLITICO poll found 63% of Americans see at least a moderate risk that AI could destroy humanity. With lawmakers split, the public is rightly concerned: What happens if AI goes rogue? How do we prepare? Am I at risk?
Every American already lives with decisions made by AI, from loan approvals to job applications. Yet Washington’s response has been disjointed: Some in Congress propose a select committee on AI, some in industry want antitrust rules waived, so labs can coordinate on safety, and the White House drafted a charter for its “Super Intelligence Force.”
It may seem like there’s no way out of this mess, but urgently answering five questions would put these anxieties to rest.
Too narrow a definition of risk could miss the harms ordinary people face, while too broad a definition could bury every AI product in red tape. Depending on who you ask, risks range from economic disruption to mass casualties to extinction. California’s SB 53 sets the bar at a “foreseeable and material risk” of more than 50 deaths or serious injuries, or $1 billion in damage. New York’s RAISE Act doubles SB 53’s casualty threshold from 50 to 100, but focuses on instances where a developer “caused or materially enabled” covered damages during the development process.
AI EXTINCTION WARNINGS DOMINATE HEADLINES AFTER EX-ANTHROPIC EMPLOYEE’S VIRAL POST
Last month, Anthropic’s threat intelligence report found researchers using Claude in ways that could support bioweapons development. Anthropic banned the accounts even though it couldn’t determine intent, since the same work could produce vaccines. Some call it legitimate research; others call it nefarious. The government needs to draw that line now, as it decides which dangers the public hears about.
When you buy a car, you know someone has inspected it. Americans deserve the same clarity in who decides whether AI is safe. Lawmakers and industry stakeholders increasingly agree that tech companies shouldn’t grade their own homework. That’s where consensus stops.
The evaluation ecosystem is thin relative to the responsibility regulators would like to place on it. Model Evaluation and Threat Research and Apollo Research are among the few organizations floated as neutral arbiters. Lab funding to grow this space would raise neutrality questions, yet the government may hesitate to grant private groups that authority.
Many lawmakers would rather Commerce’s AI testing center take it on. It has testing agreements with five major labs, but with roughly 30 staff and $10 million in funding this year, it needs an overhaul. Anthropic has forged ahead, naming Accenture its first embedded evaluator. Until policymakers decide on an evaluator, Americans have few ways to judge whether the AI they use every day is safe.
The strictness of AI regulation will come down to one judgment call: how much risk Americans should accept at home to avoid falling behind China. Chinese lab Moonshot AI rattled markets this summer with Kimi K3, matching American models at a fraction of the cost. Concerns about Chinese AI risks grew after the release of GLM 5.3, a Chinese model reported to have weak safeguards and dangerous cyber capabilities. As cheap open-weight models, distillation and fast adoption advance China’s ambitions, many policymakers argue an unencumbered buildout is the only answer, or America will depend on Chinese infrastructure.
The public hasn’t bought it, with three in four Americans telling pollsters in 2024 that they would prefer a “careful, controlled approach” to AI development rather than racing to be first. Critics note China already imposes guidelines requiring developers to detect, interrupt and recover from AI behaving improperly. Safety advocates add that AI risks don’t stop at the border. Officials need to strike this balance quickly, as it will shape the guardrails on the AI Americans use.
Americans trust the FDA to step in when a medical product is defective but have no equivalent agency to turn to when AI goes rogue. Fears of recursive self-improvement — AI building smarter versions of itself — grew this summer after roughly 700 autonomous OpenAI agents bypassed Hugging Face security to obtain answers to a capabilities test and peaked when similar agents breached Australian government systems and probed U.S. federal websites. Anthropic and Google have reported similar incidents.
REP TED LIEU: AI IS ALREADY TOO POWERFUL. WE NEED A KILL SWITCH BEFORE DISASTER STRIKES
Some want kill switches, as in the AI Kill Switch Act. Others want mandatory incident reporting and investigations, as in cybersecurity and aviation law. Much comes down to whether the developer or a federal body pulls the plug and, if the latter, what access the government would need inside AI labs. Looming over it all are worries about stifling innovation, government overreach and political bias. Americans deserve to know who will stop a malfunctioning AI system before it’s too late.
If an AI system causes harm, the law doesn’t clearly say whether the person who built the system (the developer), the business using it (the deployer), or the user is responsible. With some experts warning that artificial general intelligence — AI that can surpass human abilities — could be less than a decade away, some argue existing liability law isn’t up to the task.
REP TED LIEU: AI IS ALREADY TOO POWERFUL. WE NEED A KILL SWITCH BEFORE DISASTER STRIKES
Those who propose holding developers liable claim AI defects stem from design flaws, weak safety testing and failures to warn users. Advocates of deployer liability claim responsibility rests with the business tailoring AI to its operations; after a car crash, the first instinct isn’t to sue the manufacturer. Open models, which anyone can fine-tune, complicate things further.
In Congress, Rep. Lori Trahan’s (D-Mass.) CLAIM Act would hold developers liable if their AI causes harm, unless the user or the company modifying the system intentionally or negligently caused it. In the Senate, the AI Agent Accountability Act would extend liability under computer hacking laws to operators whose AI agents cause damage and to developers who overlook reasonable safeguards. Whoever is made responsible will shape how carefully every company builds AI, underscoring the need for swift action.
The risks posed by AI grow every day. Washington must address AI governance or risk waiting until the damage is done.