Safe & Secure Agentic AI: Governance, Identity, and Red-Teaming Training Course

Foundations: Threat Models for Agentic AI

• Categories of agentic threats: misuse, privilege escalation, data leakage, and supply-chain risks
• Adversary profiles and attacker capabilities relevant to autonomous agents
• Mapping assets, trust boundaries, and critical control points for agents

Governance, Policy, and Risk Management

• Governance frameworks for agentic systems (roles, responsibilities, approval gates)
• Policy design: acceptable use, escalation rules, data handling, and auditability
• Compliance considerations and gathering evidence for audits

Non-Human Identity & Authentication for Agents

• Creating identities for agents: service accounts, JWTs, and short-lived credentials
• Least-privilege access patterns and just-in-time credentialing
• Identity lifecycle management: rotation, delegation, and revocation strategies

Access Controls, Secrets, and Data Protection

• Fine-grained access control models and capability-based patterns for agents
• Secrets management, encryption in transit and at rest, and data minimization
• Safeguarding sensitive knowledge sources and PII from unauthorized agent access

Observability, Auditing, and Incident Response

• Designing telemetry for agent behaviour: intent tracing, command logs, and provenance
• SIEM integration, setting alerting thresholds, and ensuring forensic readiness
• Runbooks and playbooks for managing agent-related incidents and containment

Red-Teaming Agentic Systems

• Planning red-team exercises: scope, rules of engagement, and safe failover procedures
• Adversarial techniques: prompt injection, tool misuse, chain-of-thought manipulation, and API abuse
• Executing controlled attacks to measure exposure and impact

Hardening and Mitigations

• Engineering controls: response throttles, capability gating, and sandboxing
• Policy and orchestration controls: approval flows, human-in-the-loop, and governance hooks
• Model and prompt-level defenses: input validation, canonicalization, and output filters

Operationalizing Safe Agent Deployments

• Deployment patterns: staging, canary, and progressive rollout for agents
• Change control, testing pipelines, and pre-deployment safety checks
• Cross-functional governance: security, legal, product, and ops playbooks

Capstone: Red-Team / Blue-Team Exercise

• Perform a simulated red-team attack against a sandboxed agent environment
• Defend, detect, and remediate as the blue team using established controls and telemetry
• Present findings, remediation plans, and proposed policy updates

Summary and Next Steps