Global AI governance tightens amid new laws, warnings, and safety scandals.

This week, California became the first jurisdiction in the U.S. to require independent oversight of high-risk AI systems. Governor Gavin Newsom signed into law Senate Bill 813 and Assembly Bill 1405, two pieces of legislation establishing new safeguards for AI development and use ([1]) ([2]). The laws create a groundbreaking framework for third-party auditing and independent risk assessments of AI models deployed in critical sectors ([3]). In practice, companies deploying advanced AI in California will need to engage certified external evaluators to verify their systems’ compliance with safety, transparency, and fairness standards.

These measures significantly raise the bar for corporate AI accountability. By mandating audits and assessments, California is effectively ending the era of self-policing for AI in the state. The new requirements mean enterprises must maintain comprehensive documentation and risk management processes for their AI systems to demonstrate compliance ([4]). Notably, Governor Newsom emphasized that while California is leading on AI governance, the scale and risks of AI demand a coordinated federal response ([5]). His call for "robust, national regulations" ([6]) signals that companies operating across the U.S. should anticipate and prepare for broader regulatory standards on AI.

For businesses, the immediate implication is the need to invest in AI governance capabilities. Organizations should start identifying high-impact AI applications in their operations and ensure they are ready for third-party scrutiny. This includes building an internal AI inventory, instituting rigorous model risk assessments, and establishing transparent auditing processes. Companies that get ahead of these compliance obligations can avoid fines and reputational damage, and even turn strong AI governance into a competitive differentiator as clients and consumers demand responsible AI use.

A major national security alert emerged as U.S. agencies publicly accused several Chinese companies of conducting large-scale intellectual property theft targeting American AI models ([1]). In a joint advisory issued on September 8, the NSA, CISA, and FBI warned that six China-based AI firms – including prominent players like Alibaba and startups DeepSeek and MiniMax – have siphoned off proprietary AI knowledge through "industrial-scale ‘knowledge distillation’" attacks ([2]). This technique involves extracting outputs from advanced U.S. AI systems (such as OpenAI’s GPT or Google’s Gemini) to train rival models, effectively shortcutting years of research. The agencies report these firms illicitly pulled “billions of tokens” of data across millions of queries since 2024 ([3]), leveraging tactics like proxy networks and stolen API keys to evade detection and content safeguards in U.S. AI platforms.

The unprecedented public naming of foreign companies for AI model theft elevates what was once an under-the-radar risk into a board-level concern. Framing the activity as a “national security threat” ([4]), U.S. officials suggest the Chinese government likely tacitly approved or even guided these efforts ([5]). Beyond heightening geopolitical tensions in tech, this move could presage stricter export controls or sanctions targeting firms implicated in AI espionage. For global enterprises, it’s a warning shot: the race for AI leadership is prompting state-sponsored IP theft on a massive scale, which could directly impact companies' competitive advantage and data security if their crown-jewel models or proprietary data are targeted.

Enterprise AI providers and users alike should heed the U.S. government’s urgent guidance. The advisory calls on AI companies to immediately strengthen defenses: implementing advanced monitoring to detect unusual API activity, curbing automated scraping of their models’ outputs, and sharing threat intelligence across organizations ([6]). For any company deploying or monetizing AI models – especially those accessible via APIs or web interfaces – the lesson is clear. They must invest in robust cybersecurity measures specifically tuned to AI systems, such as anomaly detection for large-volume queries and protections against model extraction techniques. In this new environment, safeguarding AI intellectual property is becoming as critical as traditional data security, and failing to do so could invite not only breaches but regulatory penalties and loss of market position.

Meanwhile, one of the world’s leading AI startups, Anthropic, is reeling from a pair of revelations that raise serious ethical and safety concerns. An investigative report by The American Prospect on September 9 unveiled that Anthropic has been developing a covert “predictive surveillance” program to monitor individuals critical of AI development ([1]). Internal job postings and interviews with company security officials show that the firm’s Global Security team has built an extensive system to keep tabs on tech critics and activists – even employing a “pre-crime” approach that reports perceived threats to law enforcement before any crime is committed ([2]). These practices – including classifying peaceful activism alongside terrorism – run counter to Anthropic’s public image as a “responsible” AI company focused on ethics ([3]). The exposé has drawn immediate scrutiny, likely inviting questions from regulators and civil liberties groups about the legality and morality of private AI-enabled surveillance on U.S. citizens.

Compounding Anthropic’s challenges, a senior researcher at the company very publicly resigned, citing alarm over the trajectory of AI development. Jacob Coxon, who has worked at both OpenAI and Anthropic, announced his departure on social media with a scathing warning that both companies are “gambling with our lives” by racing toward unchecked AI systems ([4]). He claimed the labs’ leaders “earnestly believe [AI] could kill us all by the end of the decade” ([5]) and are “racing straight to self-improving superintelligence” without an adequate plan for control ([6]). Remarkably, an Anthropic AI safety lead publicly concurred, estimating the chance of AI causing human extinction above 10% within ten years ([7]). These insider revelations underscore a rare moment of open dissent within a top AI firm, echoing long-standing fears from the AI safety community now coming directly from within industry ranks.

For business leaders, these twin crises at Anthropic illustrate the stakes of ethical governance and safety oversight. On one hand, engaging in secretive, high-risk uses of AI – like surveilling critics – can backfire disastrously, eroding trust among customers, partners, and regulators. On the other, ignoring or silencing internal warnings about AI risks can lead to talent loss, public relations fallout, and potential intervention by authorities. Lawmakers are already taking note: U.S. Senator Bernie Sanders is preparing a bill to ban the development of “superintelligent” AI systems ([8]), and the EU’s AI Act explicitly requires companies to assess and mitigate loss-of-control risks in advanced AI models ([9]). The clear message is that AI companies must transparently address ethical and safety issues or face harsh consequences. Enterprises that partner with AI vendors should also conduct due diligence on those firms’ governance practices to avoid being entangled in similar controversies.

Responding to intensifying scrutiny, some AI organizations are proactively enhancing their governance structures. In a notable example, OpenAI announced on September 9 that it has appointed renowned AI alignment researcher Paul Christiano to its OpenAI Foundation board’s Safety & Security Committee ([1]). Christiano – a former head of alignment at OpenAI and a respected figure in AI safety research – brings deep expertise from both his prior work at OpenAI and recent roles advising the U.S. government on AI standards and risk management ([2]). His role on the Safety & Security Committee (alongside board members such as former Salesforce co-CEO Bret Taylor) is to provide independent oversight of OpenAI’s risk mitigation, security measures, and alignment with its mission of safe AI development ([3]).

OpenAI’s move is widely seen as setting a precedent for corporate AI governance. By giving a seasoned AI safety expert a formal voice at the highest level of decision-making, OpenAI is strengthening its accountability mechanisms at a time when advanced AI models are under the microscope. The company’s leadership noted that Christiano’s expertise in frontier AI risks and standards will bolster the Board’s ability to challenge assumptions and ensure rigorous safeguards as AI capabilities grow ([4]) ([5]). This kind of board-level engagement on AI risk is still emerging – no laws yet require it – but investor and public expectations are pushing companies in this direction. Other tech firms are likely to face pressure to follow suit by establishing dedicated AI oversight committees, expanding board expertise in AI ethics and security, and demonstrating that senior leadership is actively managing AI-related risks.

For enterprises in any industry, the takeaway is clear: effective AI governance now means leadership involvement. Boards should treat AI risk as a strategic and fiduciary priority, akin to cybersecurity or financial controls. That may involve nominating directors with AI expertise, forming cross-functional AI risk committees, and regularly reviewing the organization’s AI deployments for ethical, legal, and safety compliance. Proactive governance steps not only pre-empt regulators’ demands, they also reassure investors, customers, and employees that the company is taking the AI accountability imperative seriously.

Even as companies shore up governance, new technical risks are coming to light. Cybersecurity researchers this week revealed “GitSpawn,” a class of vulnerabilities that can be exploited to hijack AI-based software development assistants ([1]). The exploit allows attackers to embed malicious commands inside a project’s Git configuration file; when an AI coding agent (such as OpenAI’s Codex or Anthropic’s Claude Code) is used to read or analyze that repository, the hidden commands execute with the developer’s permissions ([2]). According to Check Point Research, GitSpawn affected at least seven popular AI coding tools – including offerings from OpenAI, Anthropic, Meta, Amazon, Baidu, and others – potentially enabling unauthorized code execution or data exfiltration as soon as an AI agent opens a compromised repository ([3]).

This discovery highlights the evolving security attack surface that AI introduces into enterprise environments. Many organizations have begun integrating AI “co-pilot” tools into their software development lifecycle to accelerate coding and code review. However, as GitSpawn demonstrates, these AI helpers can themselves become vectors for supply chain attacks if adversaries can plant exploits in code repositories. A seemingly innocent open-source library or sample project could trigger malicious actions on a developer’s machine or pipeline when parsed by an AI tool. Companies adopting AI-driven development must update their security policies and tooling accordingly – for example, by enforcing strict repository trust prompts, keeping AI coding assistants sandboxed or updated with patches, and training developers about the risks of automatically running AI-suggested code.

More broadly, the GitSpawn incident serves as a cautionary tale that AI safety isn’t only about high-profile scenarios of rogue AI behavior; it also encompasses traditional cybersecurity concerns in new forms. Business leaders should ensure that their IT and risk teams treat AI systems and third-party AI services as part of the critical infrastructure that needs rigorous security assessment and vendor due diligence. Just as the move to cloud computing required new security paradigms, the rise of AI across software development and operations calls for updated “AI supply chain” risk management. Those who proactively address these issues will be better positioned to innovate with AI safely, while those who do not may inadvertently open the door to breaches and liability.