In AI governance, predicting future AI behavior in advance is not sufficient to ensure safety. Expected Behaviors are uses and behaviors that can be anticipated during AI development and evaluation. Developers can prepare test cases, evaluate models, and address known problems. With traditional software, extensive testing of expected inputs, processes, and outputs can provide relatively high confidence about behavior during operation.

AI, however, may exhibit Emergent Behaviors after deployment that were not sufficiently anticipated during development. These behaviors do not arise from the AI model alone. After deployment, New use cases, New contexts, and New interactions emerge. Unexpected uses, connections with other systems, and deployment in different social or institutional environments can produce new behaviors and potentially create Unpredictable risks.

This is not simply a consequence of insufficient AI performance. Even with greater model accuracy and more extensive pre-deployment evaluation, it is difficult to enumerate every future use case, context, and interaction in advance. As AI is deployed more widely, the number of possible combinations increases. AI governance therefore has a structural limitation if it relies on Prediction alone.

Pre-deployment risk assessment and safety testing remain essential. However, testing before deployment cannot address every new situation that emerges during operation. What is also required is continuous observation and governance during operation. New patterns, deviations, anomalies, repetitions, and shifts must be detected, evaluated, and fed back into governance.

I2EA therefore adopts not “Predict everything before deployment,” but “Evaluate before deployment, observe after deployment, and govern continuously.”

This concept leads to AI Vigilance. It does not attempt to predict the future perfectly. Instead, it assumes that unanticipated phenomena will emerge and provides a mechanism to detect them, evaluate them, and feed them back into governance.