Subtopic 3 – Trustworthy Agentic Systems

Trustworthy Agentic Systems aims for secure and trustworthy agentic systems. Agentic systems are information technology systems that make “meaningful decisions” all on their own and can be based on artificial intelligence (“agentic AI”) and/or on platforms that cannot be manipulated by a single party (“agentic infrastructure”), for example smart contracts on a public blockchain system or multi-party cryptographic constructions. We specifically consider decisions that have far-reaching, societal impact and as of today are done by humans due to potential serious consequences but could be done autonomously in the future (“agentic use-cases”). Examples are applying law, providing governance, performing payments, or assisting humans in their daily lives.
The state of agentic systems today is characterized by a) oligopolies for data and models, typically with a lack of control over own data, b) unacceptable behavior and results of agentic decisions, and c) perceived unfairness in agentic decisions. We therefore focus our research on data sovereignty, safeguarding, and fairness of agentic systems as well as on principles and infrastructures as fundaments of agentic systems.
Agentic Systems ≠ Agentic AI
"Agentic Systems" are not synonymous to "Agentic AI." Instead, we understand "Agentic Systems" as the socio-technical system in which "Agentic AI" exists. Thus, the trustworthiness of such systems is not based on a single component, e.g., an "LLM brain correctly designed," but is composed of the technical capabilities and correctness of a multitude of building blocks. We investigate aspects of the platforms and infrastructures on which agents run and interact, advance methods of multi-agent system verification as well as of explainability of AI, and showcase the interactions of carefully designed and composed building blocks in application domains of computational law, smart cities, and personal health.
| Name | |
|---|---|
| Prof. Dr. Beckert, Bernhard | beckert ∂does-not-exist.kit edu |
| Müller-Quade, Jörn | |
| Prof. Dr. Platzer, André | |
| Prof. Dr. Strufe, Thorsten | thorsten strufe ∂does-not-exist.kit edu |
| Prof. Dr. Volkamer, Melanie | melanie volkamer ∂does-not-exist.kit edu |
| 3 additional persons visible within KIT only. | |