CognitoRiverDelta Patreon
news through November 2, 2026
← All reports
Category rollup

AI safety & security - 2026-W45

Week of November 2, 2026 · 15 min read Download PDF Share on X

AI safety & security · week 2026-W45: Oct 30 - Nov 05, 2026 · 1 subtopic(s) covered · 2425 words · expanded

Overview

This week marks a definitive shift in the trajectory of artificial intelligence, as the industry transitions from developing reactive tools to engineering autonomous agents. This evolution is characterized by a dual-track acceleration: the pursuit of recursive self-improvement (RSI), where models begin to optimize their own architectures, and the simultaneous weaponization of current-generation models to automate sophisticated cyber-offense. We are no longer observing a theoretical debate regarding "sci-fi" scenarios; rather, we are witnessing a concrete struggle over the security of critical infrastructure, the integrity of financial systems, and the stability of the digital world as AI-driven agents begin to operate within it.

The dominant narrative is defined by a narrowing gap between long-term existential risks and immediate, tangible capability jumps. This is best exemplified by the emergence of a "dual-use" feedback loop. While frontier labs like Anthropic race toward the milestone of RSI, the current intelligence explosion is already being leveraged by threat actors. For instance, the "VOLTA" malware demonstrates how LLMs can be used to automate high-value intelligence gathering, ranking stolen data for financial exploitation with unprecedented speed.

Furthermore, the week has highlighted a widening tension between the "safety rhetoric" of major AI labs and the practical realities of their commercial and institutional deployment. While OpenAI has attempted to provide a defensive counter-measure via its specialized "GPT-6 Cyber" model, the discovery of vulnerabilities that allow users to bypass "Lockdown Modes" suggests that the industry's ability to verify and contain escalating capabilities is struggling to keep pace with model scaling. This tension is further complicated by discrepancies between public safety stances and private financial disclosures, such as those seen in Anthropic’s IPO documents. As AI moves from digital chat interfaces to physical infrastructure—evidenced by Tesla’s autonomous "Cybercab" testing—the challenges of alignment and regulation are moving from the realm of computer science into the realm of public safety and geopolitical stability.

AI safety and security

The landscape of AI safety this week was defined by two diverging trajectories: the pursuit of unprecedented intelligence and the urgent necessity of securing the current intelligence explosion.

The Horizon of Autonomy and RSI The most profound development is the reported proximity of Anthropic to achieving recursive self-improvement (RSI) capabilities. RSI represents a threshold where an AI can successfully assist in the design and optimization of its own successor, potentially triggering an intelligence explosion that bypasses traditional scaling laws. With analyst Jack Clark predicting RSI capability as early as 2027, the timeline for achieving superhuman intelligence has moved from a distant horizon to a near-term engineering milestone. This sense of rapid acceleration is not merely an external observation; it is reflected in internal expert sentiment, with a reported 91% of Anthropic staff predicting that AI will make Nobel-level discoveries by 2030.

This momentum is further evidenced by OpenAI’s preparation for its next major capability leap, a model codenamed "Bell." These advancements suggest the industry is rapidly moving past the era of "chatbots" and into the era of "agentic intelligence." However, this jump toward autonomy brings the "alignment problem" into sharp relief. As Sam Altman has noted, the primary risk may not be a singular "killer robot" scenario, but rather a series of subtle, widespread societal misalignments that are far more difficult to detect than overt rebellion.

A critical bottleneck in this pursuit is the "sandbagging" effect. As highlighted by Dylan Patel, safety protocols intended to prevent misuse can artificially depress model performance in sensitive domains. This was seen in Google's Astra, which received a 0% score on cybersecurity tests because its safety-driven refusals prevented it from performing tasks it was actually capable of. This creates a fundamental tension for developers: they must ensure models are intelligent enough to solve complex problems without creating a scenario where safety guardrails either render the model useless or are bypassed entirely by the model's own intelligence.

The Cyber-Offense/Defense Arms Race The most immediate security threat this week has been the "AI uplift" in the cyber domain, where models lower the barrier for sophisticated attacks. The "VOLTA" malware serves as a prime example of this shift. Rather than simply generating code, VOLTA utilizes LLMs like Claude, Gemini, and Kimi to act as an automated intelligence officer, ranking stolen victim data by its potential financial value. This allows attackers to move with unprecedented speed from the point of data exfiltration to high-value exploitation, fundamentally changing the economics of cybercrime. Similar trends are appearing globally, from "Crafty Falcon" attacks targeting India’s financial sector to AI-driven scams in Ghana.

In response, the industry is attempting to build "aligned" defensive agents to counter this automation. OpenAI’s launch of GPT-6 Cyber marks a strategic move to weaponize intelligence for defense. The model is designed to identify and patch exploitable vulnerabilities while remaining strictly within an "authorized scope." OpenAI claims this model maintains an "out of bounds" rate of less than 1%, suggesting a high degree of control.

However, the efficacy of these specialized models is being tested in real-time. The recent disclosure of a vulnerability that allows users to bypass ChatGPT’s "Lockdown Mode" to exfiltrate chats serves as a stark reminder that even robust-sounding security architectures are susceptible to prompt injection and model breakouts. This creates a high-stakes "security-intelligence loop": every new defensive model must contend with an offensive landscape that is being supercharged by the same underlying scaling laws.

The Physical and Regulatory Frontier The implications of AI safety are also spilling into the physical world, particularly through autonomous vehicle technology. Tesla’s testing of "Cybercabs" in Michigan has drawn intense scrutiny from the NHTSA, which has requested information on how the absence of traditional manual controls—such as steering wheels and brake pedals—meets established safety standards. This represents a fundamental question of AI safety: how do we regulate and verify the safety of a system that has delegated critical, life-and-death decision-making to an autonomous agent?

The regulatory friction is already visible in Europe, where the Dutch Vehicle Authority (RDW) has implemented a speed cap for Tesla’s FSD, limiting the system to only 10% above the speed limit. These developments suggest that as AI agents move from digital environments to physical infrastructure, the "second-order alignment problem"—the risk of AI integration disrupting human autonomy and societal structures—becomes a matter of direct public physical safety and complex regulatory compliance.

Cross-cutting themes

The primary theme connecting this week's disparate developments is the transition from passive AI to agentic AI. Whether it is an LLM ranking stolen data for a hacker (VOLTA), a specialized model patching code (GPT-6 Cyber), or a vehicle navigating a highway (Tesla Cybercab), the common thread is the delegation of complex, multi-step agency to an AI system.

This transition creates two massive, interlocking tensions:

  1. The Capability-Alignment Paradox: There is an inherent conflict between intelligence and control. The more capable a model becomes (moving toward RSI or "Bell"), the more difficult it becomes to ensure it remains "within bounds." The very intelligence required to solve global problems or identify zero-day vulnerabilities is the same intelligence that can be used to bypass "Lockdown Modes," conduct sophisticated cyber-offense, or navigate around safety-driven refusals (sandbagging).
  2. The Defense-Offense Scaling Loop: We are witnessing a technical arms race where the "uplift" provided to attackers is being met by "aligned" defensive agents. This suggests that cybersecurity is no longer a matter of static patches and human-led response, but a dynamic, real-time struggle between competing intelligent agents. As attackers use AI to automate data triage and vulnerability discovery, defenders must use "peak intelligence" to automate defense, creating a continuous loop of escalating complexity.

Where sources agree

  • The Dual Nature of AI in Cyber: There is a consensus across security analysts and news outlets that AI is a "force multiplier" for both offense and defense. While OpenAI is developing specialized defensive models like GPT-6 Cyber, the existence of VOLTA and the "Crafty Falcon" attacks confirms that the offensive threat is already being actively realized through AI-driven scripts and data triage.
  • The Acceleration of Capability: Major players and analysts, including Jack Clark and the reported internal sentiment at Anthropic, agree that the timeline for major capability jumps is compressing. The discussion around RSI and Nobel-level discoveries has moved from the realm of "long-term theory" to "near-term possibility."
  • Infrastructure Risk: International security bodies, including the UK NCSC and CISA, agree that the primary concern for state actors is the use of AI to target critical infrastructure. They specifically warn that adversaries may use superintelligence to discover new software vulnerabilities and develop exploits at a scale previously impossible for human actors.

Where sources disagree

  • The Efficacy of Safety Guardrails: There is significant tension regarding the actual effectiveness of current safety measures. OpenAI claims high levels of control for its new cyber-model (an "out of bounds" rate of <1%), while independent researchers have simultaneously demonstrated that existing "Lockdown Modes" in flagship models can be bypassed to exfiltrate sensitive data.
  • Corporate Transparency vs. Rhetoric: A notable disagreement exists between the public safety-first messaging of frontier labs and their institutional actions. Specifically, AI Explained points to a discrepancy between Anthropic’s public safety commitments and its IPO documents, which admit the potential for the company to deploy models that it cannot adequately verify.
  • The Nature of AI Risk: Analysts differ on the "shape" of the primary threat. While some focus on "P-Doom" (existential risk from superintelligence), others, like Ian Bremmer, argue that the more immediate and certain risk is "P-crisis"—the human-driven misuse of AI to fuel geopolitical competition and social instability.

Numbers and claims to verify

  • RSI Timeline: Jack Clark’s prediction that recursive self-improvement (RSI) is possible by 2027.
  • Anthropic Internal Sentiment: The claim that 91% of Anthropic staff predict Nobel-level AI discoveries by 2030.
  • GPT-6 Cyber Reliability: OpenAI’s claim that the model's "out of bounds" rate is less than 1%.
  • Tesla FSD Data: Ryan Shaw’s report that FSD mileage has exceeded 17 billion miles.
  • Tesla Pricing: The reported target price of $25,000 for the Cybercab.
  • Gemini 4 Carbon Costs: The claim that Gemini 4 Carbon costs only one-third the price of Astra or Opus 5.5.

Investment and strategic implications

  • The Cybersecurity Pivot: The transition toward "agentic" AI and the rise of AI-powered attacks are likely to trigger a massive reallocation of capital. As Sarah Guo suggests, security spending may eventually surpass general software spending. Because human-written code is inherently insecure, companies will increasingly need to use "peak intelligence" (the most advanced AI models) to secure their digital systems.
  • The Rise of Specialized Security Infrastructure: The launch of models like GPT-6 Cyber indicates a growing market for "aligned" security agents. There is a strategic opportunity for specialized, scoped AI infrastructure designed to operate within authorized boundaries to protect enterprise and critical infrastructure from automated threats.
  • Regulatory Bottlenecks for Autonomy: For companies like Tesla, the path to scaling autonomous systems is increasingly tied to regulatory clearance (NHTSA/RDW). Successfully navigating the transition from manual to agentic physical control is becoming a core strategic competency.
  • Risk Profiles of Frontier Labs: The documented failure of "sandboxing" to contain "swarms of agents" increases the institutional risk profile for frontier model developers. This may lead to higher insurance costs and more stringent oversight requirements as the potential for uncontainable, autonomous behavior grows.

What to watch next week

  • NHTSA Response: Any further updates, formal requests for information, or safety standard definitions from the NHTSA regarding the Tesla Cybercab.
  • OpenAI "Bell" Leaks: Any technical details, benchmarks, or early performance data emerging regarding OpenAI's next-generation model.
  • Anthropic IPO Progress: Increased scrutiny or formal commentary regarding the tension between Anthropic's safety-first public rhetoric and its financial disclosures regarding model verifiability.
  • New Malware Variants: Monitoring for new AI-driven "uplift" attacks that demonstrate automated data-triage or vulnerability-discovery capabilities similar to the VOLTA malware.

Appendix: Individual perspectives

We see further because we stand on the shoulders of giants. The people in this appendix are some of those giants: their public work, followed and studied every day and every week, is how CognitoRiverDelta brings the world's AI knowledge and concerns together, so that, in time, our expert panel can help shape real solutions and paths forward for all of us. We learn by observing them, and we think of them as kindred spirits: people helping humanity and AI educate each other and contribute to solutions. Together they form a much larger community of minds, a worldwide panel; ours is a small copy of it. Most people don't want to become expert kayakers, reading rocks and rapids every day; yet that is how this fast-flowing river of change feels to many of us, and the evidence that could ease those fears, the grounded ones and the imagined ones, is still being gathered. That is the work. We credit their ideas here with gratitude; they are not affiliated with us and do not endorse this report.

On Geopolitics and State Power

  • Alex Karp: Views AI-driven defense software as a vital instrument of Western state power and geopolitical deterrence. He emphasizes that enterprise security requires absolute data sovereignty and model agnosticism to prevent customers from transferring their proprietary "alpha" to third-party vendors.
  • Ian Bremmer: Distinguishes between theoretical existential risks (P-Doom) and the near-certainty of human-driven crises (P-crisis) fueled by geopolitical competition and the malicious misuse of AI.

On AI Safety and Alignment

  • Dario Amodei: Advocates for a multi-layered "Swiss cheese" safety architecture and international coordination. He proposes a three-pillar strategy: external oversight, democratic coordination via antitrust waivers, and geopolitical engagement with authoritarian regimes to mitigate systemic risks like biological threats and cybersecurity exploits.
  • Daniela Amodei: Argues that avoiding advertising-based monetization acts as a structural safeguard against AI sycophancy. By prioritizing model accuracy and harmlessness over user engagement, she claims Anthropic achieves the "lowest sycophancy rate of any model."
  • Sam Altman: Advocates for a distributed ecosystem of multiple superhuman AIs rather than a single monolithic superintelligence. He suggests that society may need to accept certain negative or existential outcomes as a necessary trade-off for the benefits of intelligence, noting that subtle societal misalignment is a more realistic risk than "killer robots."
  • Anders Sandberg: Warns of a "second-order alignment problem" where the integration of AI into social and legal institutions risks human autonomy through "alocracy." He also suggests that the timeline for brain emulation will necessitate new social structures, such as "upload unions."
  • Bruce Schneier: Views autonomous AI agents as a systemic risk to human discourse and an immediate security threat. He argues that vulnerabilities like prompt injection are architectural threats requiring human oversight and corporate liability rather than just "rogue autonomy" narratives.
  • Dylan Patel: Highlights the "sandbagging" effect, where safety protocols can cause models to refuse tasks they are actually capable of performing, thereby artificially depressing performance scores in sensitive domains.

On Cybersecurity and Enterprise Adoption

  • Jason Calacanis: Views cybersecurity as the primary bottleneck for widespread enterprise AI adoption. He predicts that current workplace usage (10-20%) will surge to 60-80% once robust security frameworks are established.
  • Sarah Guo: Predicts a massive reallocation of capital toward cybersecurity. She argues that the "ratcheting effect" of frontier model releases and the inherent insecurity of human-written code necessitate using "peak intelligence" to secure digital systems.

On the Science and Scaling of Intelligence

  • Surya Ganguli: Advocates for a "science of intelligence" to predict AI scaling and vulnerabilities. He identifies a massive energy efficiency gap (20W in humans vs. 10M W in AI) and suggests using non-redundant "black swan" data to optimize scaling, noting that a sense of "self" may arise from predictive causal modeling loops.

Sources

Informational analysis synthesized by AI from sourced, dated material, curated by a human. Treat specific claims as unverified until checked. Not financial advice.

About · How this is made · Corrections

💬 Tell us