Final analysis:

Without mitigations like continuous monitoring, adaptive retraining, or diverse data sourcing, the likelihood approaches 90%+ for most models within months to years. However, well-maintained systems (e.g., via MLOps practices) can reduce this to near zero, as degradation is largely a solvable engineering problem rather than an inescapable fate. In critical applications like healthcare or finance, unchecked degradation could lead to biased decisions or systemic failures, underscoring the need for robust oversight.

Two case histories:

The first case is one of many documented examples of Internet hacking of American infrastructure:

Chinese hackers recently spent nearly a year deep inside a public power utility in Littleton, Massachusetts. U.S. officials said it was part of a large-scale plan to destabilize American infrastructure in the event of a future conflict. Hackers linked to Iran previously breached several U.S. municipal water utilities as well. Now over 50,000 public water treatment plants across the United States are hoping to stop attacks like that from happening to them.

More…

The second case involves using AI agents to reverse engineer websites:

There are several documented cases where AI agents have been used to reverse engineer corporate or corporate-like websites, often for purposes like understanding functionality, identifying vulnerabilities, or analyzing strategies. These typically involve AI-driven tools that interact with the site’s UI, APIs, network traffic, or content without direct access to the source code. Below are some real-world examples based on reported experiments and applications. Note that these are often conducted in controlled or ethical contexts, such as research or security testing, and reverse engineering can raise legal and ethical considerations depending on the intent and permissions. (Grok)

We are used to the term “ransomware” where a group of people break into a site to blackmail an institution or government agency to regain control of their information. Hacking includes breaking into systems with sleeper agents that launch coordinated attacks on the infrastructure of a country at very little cost.

What would I do to attack an enemy? I would use AI agents to identify infrastructure that is vital for local resources such as water, sewer and electrical utilities as shown in the second case outlined above. Transportation (trucking, railroad and airlines) would also be identified. I would then direct AI agents to find backdoors or administrative passwords to give me control whenever the time is “appropriate” to launch a coordinated shutdown.

I am not writing this as a suggestion for a course of action: I am writing this to warn people that this is already done. We have seen too many examples over the past twenty years with hackers.

Over 11 billion records exposed since 2005, with costs in the trillions (e.g., $4.1B for healthcare alone in 2012). (Grok)

Why would we assume that they did not make their task easier with AI agents? Then again, one EMP could still take down our electrical grid since Congress did not authorize any hardening of our system.

The actual story is the reverse reverse engineering since Iran’s drones were developed by spoofing an American drone. Let’s compare that to the cost of one F-35:

Approximately 2,929 LUCAS drones could be produced with the budget for a single F-35 (rounding up from the fractional result, though exactly it covers 2,928 full units with some leftover funds). Note that this is a direct cost comparison and doesn’t account for factors like development overhead, bulk production discounts, or operational expenses. (Grok)

For a quick reference:

The F-35 is the DoD’s most expensive weapons program, with total lifecycle costs now exceeding $2 trillion—400% over initial estimates adjusted for inflation. (Grok)

The future of war is asymmetric warfare:

In an era of advanced technology, asymmetric warfare favors innovation and adaptability. Weaker actors can use commercial off-the-shelf tech (e.g., drones, AI) to challenge superpowers, leading to prolonged conflicts and high costs for the stronger side.

This has influenced U.S. military doctrine, emphasizing irregular warfare as a core activity alongside conventional ops.

However, it’s not always successful for the underdog—superior forces can adapt with counterinsurgency strategies, intelligence, and alliances. (Grok)