# AI Red Teaming and Assurance Course (Self-Paced)

Canonical URL: <https://www.graduateschool.edu/courses/ai-red-teaming-and-assurance-self-paced>

## Overview

The capstone of the applied AI security path, this course closes the loop between testing an AI system and being able to defend the decision to run it. It separates two things agencies routinely conflate: red teaming is deliberately trying to break, mislead or misuse a system, while assurance is the documented evidence that the system is trustworthy enough for its intended purpose. It runs the engagement lifecycle — scoping testing to the system's risk classification, setting objectives, writing rules of engagement, and handling team composition and deconfliction — then technique, matching method to attack surface and weighing manual testing against automated approaches.

It covers agentic AI, where the attack surface expands from what a system says to what a system does, and the assurance case as claim, argument and evidence, in the form an oversight body can audit, alongside Test, Evaluation, Verification and Validation records, model and system cards, and how documentation feeds a risk-acceptance decision. It teaches difficult judgment calls: telling a red-team finding apart from an ordinary software bug, prioritizing by impact rather than technical complexity, and recognizing when a finding becomes an incident. There is no lab; scenario-based practice includes a prompt-assessment exercise using real example prompts from an engagement, so acquisition and oversight staff can take it alongside testers. It sits atop a four-course ladder running from AI Security Fundamentals for the Cyber Workforce Course (Self-Paced) and AI Security & Governance for Government Course (Self-Paced) through Securing AI/ML Systems Course (Self-Paced) and AI Security Monitoring and Incident Response Course (Self-Paced).

## What you'll learn

- Distinguish red teaming from adjacent assurance activities, and place it correctly in the AI assurance lifecycle; Run a red-team engagement lifecycle from planning through reporting
- Scope testing to a system's risk classification, and set objectives that produce usable results; Write rules of engagement, and handle team composition and deconfliction
- Match adversarial testing techniques to the attack surface they actually address; Weigh manual expert-driven testing against automated and tool-assisted approaches
- Apply probing techniques to generative AI systems; Tell a genuine red-team finding apart from an ordinary software bug
- Build Test, Evaluation, Verification and Validation records that support a risk decision; Construct an assurance case as claim, argument and evidence
- Interpret model cards and system cards, and judge independent evaluation results; Structure a findings report and prioritize by impact rather than technical complexity
- Manage remediation and retesting, and apply human oversight as a mitigation; Recognize when a finding becomes an incident
- Extend testing and assurance to foundation models and to agentic, tool-using systems

## Prerequisites

General agency cybersecurity awareness training, plus familiarity with AI governance and AI/ML threat terminology, as covered in [AI Security & Governance for Government Course (Self-Paced)](/courses/ai-security-and-governance-for-government-self-paced). [AI Security Monitoring and Incident Response Course (Self-Paced)](/courses/ai-security-monitoring-and-incident-response-self-paced) is helpful but not required.

## Curriculum

#### Module 1

- Foundations: What Is AI Red Teaming, and Where Does It Fit in AI Assurance?
- What red teaming is and is not
- Where it sits in the AI assurance lifecycle
- The range of assurance activities
- The guidance the course draws on
- Who is involved

#### Module 2

- Planning and Scoping a Red-Team Engagement
- The engagement lifecycle
- Scoping to risk classification
- Setting red-team objectives
- Rules of engagement
- Team composition and deconfliction

#### Module 3

- Adversarial Testing Techniques for Machine Learning and Generative AI
- Matching technique to attack surface
- Manual versus automated and tool-assisted red teaming
- Generative AI probing techniques
- A preview of testing agentic and tool-using systems
- Telling red-team findings apart from ordinary bugs

#### Module 4

- AI Assurance Frameworks: TEVV, Assurance Cases, and Documentation
- Test, Evaluation, Verification and Validation
- The assurance case as claim, argument and evidence
- Model cards and system cards
- Independent evaluation and measurement science
- From assurance documentation to a risk-acceptance decision

#### Module 5

- From Findings to Fixes: Reporting, Remediation, and Human Oversight
- Structuring a findings report
- Prioritizing by impact rather than technical complexity
- Remediation and retesting
- Human oversight as a mitigation category
- When a finding becomes an incident

#### Module 6

- Red Teaming and Assurance for Generative and Agentic AI; Capstone Scenario
- Foundation models and misuse-risk testing
- Agentic AI and the expansion of the attack surface from output to action
- Why red teaming and assurance are recurring rather than one-time
- A capstone scenario

## Pricing

**Tuition:** $1049