Pre-Execution Governance of Compositional Enterprise AI: Causal Prediction and Minimal Architectural Intervention for Emergent Capabilities
Keywords:
Autonomous AI, Risk Management, AI Governance, Emergent Capabilities, Minimal Architectural Intervention, Causal Risk Prediction, Pre-Execution Governance, Compositional Enterprise AIAbstract
This study introduced a pre-execution governance framework for compositional enterprise AI to mitigate risks stemming from the interactions among AI agents, models, tools, databases, memory, permissions, and external services. The four metrics - policy compliance, risk interception, unnecessary intervention, and governance decision outcomes - were used to compare baseline AI governance, traditional governance, and the proposed governance model. The proposed framework integrated causal risk prediction and limited architectural changes prior to executing AI actions. The evaluation demonstrated that the proposed approach led to a policy compliance of 94.2%, risk interception of 93.6% and legitimate autonomy of 91.8%, with only 11.6% of unnecessary interventions. The overall accuracy of causal risk prediction was 92.4% and multi-agent interactions were still relatively challenging to predict. This study has shown that pre-execution governance, through targeted architectural interventions, can enhance the control of risks while maintaining legitimate autonomy of the AI.
References
AI, N. (2024). Artificial intelligence risk management framework: Generative artificial intelligence profile. NIST Trustworthy and Responsible AI Gaithersburg, MD, USA.
Anderljung, M., Barnhart, J., Korinek, A., Leung, J., O'Keefe, C., Whittlestone, J., ... & Wolf, K. (2023). Frontier AI regulation: Managing emerging risks to public safety. arXiv preprint arXiv:2307.03718.
Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., ... & Kaplan, J. (2022). Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073.
Debenedetti, E., Zhang, J., Balunovic, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024). Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents. Advances in neural information processing systems, 37, 82895-82920.
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Iii, H. D., & Crawford, K. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86-92.
Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., & Narasimhan, K. (2024, May). Swe-bench: Can language models resolve real-world github issues?. In International Conference on Learning Representations (Vol. 2024, pp. 54107-54157).
Kapoor, S., Stroebl, B., Siegel, Z. S., Nadgir, N., & Narayanan, A. (2024). Ai agents that matter. arXiv preprint arXiv:2407.01502.
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., ... & Koreeda, Y. (2022). Holistic evaluation of language models. arXiv preprint arXiv:2211.09110.
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., ... & Gebru, T. (2019, January). Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency (pp. 220-229).
Ruan, Y., Dong, H., Wang, A., Pitis, S., Zhou, Y., Ba, J., ... & Hashimoto, T. (2024, May). Identifying the risks of LM agents with an LM-emulated sandbox. In International Conference on Learning Representations (Vol. 2024, pp. 27031-27098).
Schaeffer, R., Miranda, B., & Koyejo, S. (2023). Are emergent abilities of large language models a mirage?. Advances in neural information processing systems, 36, 55565-55581.
Shevlane, T., Farquhar, S., Garfinkel, B., Phuong, M., Whittlestone, J., Leung, J., ... & Dafoe, A. (2023). Model evaluation for extreme risks. arXiv preprint arXiv:2305.15324.
Wei, J. (2022). Emergent abilities of large language models. arXiv preprint arXiv:2206.07682.
Zaharia, M., Khattab, O., Chen, L., Davis, J. Q., Miller, H., Potts, C., ... & Ghodsi, A. (2024). The shift from models to compound AI systems. The Berkeley Artificial Intelligence Research Blog.
Zhou, S., Xu, F. F., Zhu, H., Zhou, X., Lo, R., Sridhar, A., ... & Neubig, G. (2024, May). Webarena: A realistic web environment for building autonomous agents. In International Conference on Learning Representations (Vol. 2024, pp. 15585-15606).
Downloads
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
