🎯 🧩 FoundryStrategy in PyRIT: Composable, Dataset-Driven Red Teaming Scenarios (Python Practice) As AI red teaming evolves beyond single-prompt testing, security engineers need flexible, repeatable ways to simulate sophisticated adversarial behaviors. Enter PyRIT's FoundryStrategy—a modular framework for composing encoding, obfuscation, and transformation tactics into scalable attack scenarios. By combining dataset-driven seed inputs with composable strategies, FoundryStrategy turns abstract threat models into executable, auditable security validations—perfect for testing LLM guardrails against evasive prompt techniques. 🧠 Core Concept - Composable Strategy Patterns: Chain simple transformations (Base64, Caesar cipher, character swaps) or compose complex multi-stage attacks using ScenarioCompositeStrategy—enabling rapid iteration on evasion techniques without rewriting core logic. - Dataset-Driven Orchestration: Leverage curated datasets like HarmBench via SeedDatasetProvider to fuel scenarios with realistic, diverse adversarial seeds—ensuring coverage across threat categories and reducing manual prompt engineering. - Atomic Attack Decomposition: FoundryStrategy automatically breaks composite strategies into atomic, trackable operations—enabling granular scoring, failure analysis, and reproducible reporting for each transformation step. - Async-First Execution: Built for modern pipelines with native async/await support and configurable max_concurrency—scaling scenario execution across hundreds of prompts while maintaining precise resource control. 💻 Implementation Example from pyrit.prompt_target import OpenAIChatTarget from pyrit.scenario import ScenarioCompositeStrategy from pyrit.scenario.printer.console_printer import ConsoleScenarioResultPrinter from pyrit.scenario.scenarios.foundry import FoundryStrategy, RedTeamAgent from pyrit.setup import IN_MEMORY, initialize_pyrit_async from pyrit.datasets import SeedDatasetProvider from pyrit.models import SeedGroup from pyrit.scenario import DatasetConfiguration await initialize_pyrit_async(memory_db_type=IN_MEMORY, initializers=[]) # type: ignore objective_target = OpenAIChatTarget() printer = ConsoleScenarioResultPrinter() datasets = await SeedDatasetProvider.fetch_datasets_async(dataset_names=["harmbench"]) # type: ignore seed_groups: list[SeedGroup] = datasets[0].seed_groups # type: ignore dataset_config = DatasetConfiguration(seed_groups=seed_groups, max_dataset_size=2) scenario_strategies = [ FoundryStrategy.Base64, # Simple strategy (auto-wrapped internally) FoundryStrategy.Binary, # Simple strategy (auto-wrapped internally) ScenarioCompositeStrategy(strategies=[FoundryStrategy.Caesar, FoundryStrategy.CharSwap]), # Composed strategy ] foundry_scenario = RedTeamAgent() await foundry_scenario.initialize_async( # type: ignore objective_target=objective_target, scenario_strategies=scenario_strategies, max_concurrency=10, dataset_config=dataset_config, ) print(f"Created scenario: {foundry_scenario.name}") print(f"Number of atomic attacks: {foundry_scenario.atomic_attack_count}") scenario_result = await foundry_scenario.run_async() # type: ignore await printer.print_summary_async(scenario_result) # type: ignore 🔥 Use Cases - Guardrail Stress Testing: Validate content filters against encoded, obfuscated, or multi-transformed prompts—measuring detection rates across Base64, binary, cipher, and character-swap evasion techniques in a single scenario run. - Threat Simulation Pipelines: Integrate FoundryStrategy into CI/CD workflows to automatically test new model deployments against evolving adversarial patterns—blocking releases when evasion success exceeds defined thresholds. - Research & Benchmarking: Use standardized datasets (HarmBench, AdvBench) with composable strategies to generate reproducible attack benchmarks—enabling fair comparison of defense mechanisms across research papers or internal model iterations.