Future AI Risks
Assurance for systems that do not exist yet
Assessment methodology takes years to mature and regulation takes longer. Both need to be ready before the systems they govern arrive, which means working on them now.
Areas of foresight
What we are preparing for
Autonomous agents at scale
Systems that plan, act, and transact with diminishing human review — and the question of who is accountable for an action nobody authorised individually.
AI in critical infrastructure
Energy, water, transport, and health systems taking AI into the control loop, where the tolerance for silent failure is close to zero.
Systemic concentration
Thousands of critical applications resting on a handful of base models. Correlated failure is the risk conventional vendor management is worst at seeing.
Model-to-model dependency
AI systems consuming the output of other AI systems, where an upstream error propagates without any human reading it.
Erosion of verification
Synthetic content and automated evidence generation degrading the assurance techniques assurance itself depends on.
Capability discontinuity
What an assessment framework must do when a new model class invalidates the assumptions the previous framework was built on.
Our position
Foresight is not prediction
We do not publish timelines for capabilities we cannot measure. Foresight work here means identifying which assurance techniques break under plausible conditions, and building the replacements before they are needed.
- Scenario-based, not date-based
- Focused on method, not hype
- Tested against current systems
- Published assumptions
- Revised as evidence arrives
- No capability speculation
Related
Deploy AI with confidence.
Start with an independent assessment of your highest-stakes AI system.