Theme 3 – Trustworthy Agentic Systems

“Trustworthy Agentic Systems” investigates security and privacy aspects of systems that are meant to act fully on their own, according to European democratic standards. Furthermore, the use of those systems has far-reaching consequences. Theme 3 is contributing to the societal challenges of geopolitical crisis and AI revolution.

The objective of Theme 3 aims for secure and trustworthy systems that make meaningful decisions on their own and can be based on artificial intelligence (“agentic AI”) and/or on platforms that cannot be manipulated by a single party (“agentic infrastructure”).

The scientific challenge of Theme 3 is to provide guarantees and comprehensibility of agentic systems at scale, regarding their fairness security, and privacy aspects. We investigate a spectrum of trade-offs for tunable protection of agentic systems, in particular grand challenges like protection of Explainable AI as well as identification and optimization of the real-world “price-to-pay” in terms of utility for privacy and robustness in distributed or decentralized learning.
 

What Are Agentic Systems?

"Agentic Systems" are not synonymous to "Agentic AI." Instead, we understand "Agentic Systems" as the socio-technical system in which "Agentic AI" exists. Thus, the trustworthiness of such systems is not based on a single component, e.g., an "LLM brain correctly designed," but is composed of the technical capabilities and correctness of a multitude of building blocks. We investigate aspects of the platforms and infrastructures on which agents run and interact, advance methods of multi-agent system verification as well as of explainability of AI, and showcase the interactions of carefully designed and composed building blocks in application domains of computational law, smart cities, and personal health.