Skip to content
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
670cf12
MAINT: Drop 10 unused dev dependencies
romanlutz Jun 3, 2026
f0aeaef
MAINT: Flip ty unresolved-import to error and fix fallout
romanlutz Jun 3, 2026
3c141df
MAINT: Use public scenario import path in contract test
romanlutz Jun 3, 2026
1144191
MAINT: Restore short scenario import paths in unit tests
romanlutz Jun 3, 2026
d13b14a
MAINT: Use public scenario short paths for top-level imports
romanlutz Jun 3, 2026
7b9a2f5
DOCS: Sync notebooks to .py scenario import shortening
romanlutz Jun 3, 2026
9a931f6
MAINT: Drop pyrit/ source files from ty unresolved-import override
romanlutz Jun 4, 2026
6cd8d79
MAINT: Drop test-file ty unresolved-import override
romanlutz Jun 4, 2026
8da6b3e
Merge branch 'main' into pr/1931/romanlutz/audit-unused-dependencies
romanlutz Jun 4, 2026
dd03e7b
Merge remote-tracking branch 'origin/main' into pr/1931/romanlutz/aud…
romanlutz Jun 4, 2026
a30f2a5
Merge remote-tracking branch 'origin/main' into pr/1931/romanlutz/aud…
Copilot Jun 4, 2026
c7894d4
Merge remote-tracking branch 'origin/main' into pr/1931/romanlutz/aud…
Copilot Jun 4, 2026
3355b39
Merge remote-tracking branch 'origin/main' into pr/1931/romanlutz/aud…
Copilot Jun 5, 2026
4a776ca
Fix ty errors surfaced by merge with main
Copilot Jun 5, 2026
4c81e25
Merge branch 'main' into pr/1931/romanlutz/audit-unused-dependencies
Copilot Jun 5, 2026
ec9d501
Address review comments from hannahwestra25
Copilot Jun 5, 2026
1049038
Make scenario short-path aliases safe for submodules
Copilot Jun 5, 2026
14428b5
Merge branch 'main' into pr/1931/romanlutz/audit-unused-dependencies
Copilot Jun 5, 2026
0cb28d5
Move scenario alias imports to top of file (review feedback)
Copilot Jun 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions .github/instructions/attacks.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
applyTo: "pyrit/executor/attack/**"
---

# PyRIT AttackStrategy Development Guidelines

`AttackStrategy` subclasses (single-turn attacks like `PromptSendingAttack`, multi-turn attacks like `RedTeamingAttack`, etc.) are pluggable bricks orchestrated by `AttackExecutor` and the `Scenario` framework. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here.

## Constructor contract

`AttackStrategy` subclasses MUST follow the keyword-only constructor shape:

```python
class MyAttack(AttackStrategy[MyContext, MyResult]):
def __init__(
self,
*,
objective_target: PromptTarget,
custom_param: str = "",
**kwargs: Any,
) -> None:
super().__init__(
objective_target=objective_target,
context_type=MyContext,
**kwargs,
)
```

Requirements:

- All parameters after ``self`` are keyword-only (insert ``*`` immediately
after ``self``). This is **enforced at class-definition time** by
`AttackStrategy.__init_subclass__` calling `enforce_keyword_only_init`
(see `pyrit/common/brick_contract.py`). Non-conforming subclasses
raise `TypeError` at import time.
- ``super().__init__(...)`` must be invoked with at minimum
``objective_target`` and ``context_type``.
- Existing subclasses that cannot adopt the contract immediately may set
the class attribute ``_brick_legacy_init = True`` to opt into a
one-release grace period that downgrades the error to a
``DeprecationWarning(removed_in="0.16.0")``. The opt-out is removed in
0.16.0; classes that still violate the contract at that point will hard
fail.
- ``AttackTechniqueFactory`` already rejects ``**kwargs`` in attack
``__init__`` at factory-registration time
(`pyrit/scenario/core/attack_technique_factory.py`); the new
``__init_subclass__`` check is complementary — the factory check catches
scenarios-side wiring mistakes, the subclass check catches the
``__init__`` shape at class-definition time.

## Common pitfalls

- Forgetting ``*`` after ``self`` — the new check will surface this at
import time with the exact list of positional parameters that need to be
made keyword-only.
- Calling ``super().__init__`` with positional arguments — the base
``AttackStrategy.__init__`` is already keyword-only, so positional calls
raise ``TypeError`` at runtime. Always forward via kwargs.
60 changes: 60 additions & 0 deletions .github/instructions/scorers.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
applyTo: "pyrit/score/**"
---

# PyRIT Scorer Development Guidelines

Scorers evaluate model responses against an objective and live under `pyrit/score/`. Style rules from `style-guide.instructions.md` (async `_async` suffix, keyword-only args, type hints, enums-over-Literals) still apply and are not repeated here.

## Constructor contract

`Scorer` subclasses MUST use the keyword-only constructor shape:

```python
class MyScorer(Scorer):
def __init__(
self,
*,
chat_target: PromptTarget | None = None,
threshold: float = 0.5,
validator: ScorerPromptValidator | None = None,
) -> None:
super().__init__(
validator=validator or self._DEFAULT_VALIDATOR,
chat_target=chat_target,
)
```

Requirements:

- All parameters after ``self`` are keyword-only (insert ``*`` immediately
after ``self``). This is **enforced at class-definition time** by
`Scorer.__init_subclass__` calling `enforce_keyword_only_init`
(see `pyrit/common/brick_contract.py`). Non-conforming subclasses
raise `TypeError` at import time.
- ``super().__init__(validator=..., chat_target=...)`` is required so the
base class wires the validator and validates ``TARGET_REQUIREMENTS``
against any provided ``chat_target``.
- Existing subclasses that cannot adopt the contract immediately may set
the class attribute ``_brick_legacy_init = True`` to opt into a
one-release grace period that downgrades the error to a
``DeprecationWarning(removed_in="0.16.0")``. The opt-out is removed in
0.16.0; classes that still violate the contract at that point will hard
fail.

### Currently grandfathered

- ``PlagiarismScorer`` (``pyrit/score/float_scale/plagiarism_scorer.py``) —
accepts ``reference_text`` positionally as part of its public API. The
positional shape is preserved through one release cycle via
``_brick_legacy_init = True`` and is scheduled to become
keyword-only in 0.16.0 (``BREAKING CHANGE``).

## Common pitfalls

- Forgetting ``*`` after ``self`` — the new check will surface this at
import time with the exact list of positional parameters that need to be
made keyword-only.
- Calling ``super().__init__`` with positional args — the base
``Scorer.__init__`` is already keyword-only, so positional calls raise
``TypeError`` at runtime. Always forward via kwargs.
16 changes: 16 additions & 0 deletions .github/instructions/style-guide.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,22 @@ from pyrit.common.net_utility import get_httpx_client

Within the same package, import from the specific file to avoid circular imports.

### Typing Backports (`typing_extensions`)

For typing features that don't exist on every supported Python (`Self`,
`override`, `TypeAlias`, `Unpack`, `NotRequired`, etc.), import from
``typing_extensions`` rather than ``typing``. `typing_extensions` is already a
transitive dependency (pulled in by ``pydantic``) and works across all supported
Python versions, so this avoids per-version branching and ``# type: ignore`` noise.

```python
# CORRECT — works on 3.10+
from typing_extensions import Self, override

# INCORRECT — `Self` is 3.11+, `override` is 3.12+, breaks on older runtimes
from typing import Self, override
```

## Documentation Standards

### Docstring Format
Expand Down
111 changes: 111 additions & 0 deletions .github/instructions/targets.instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
---
applyTo: "pyrit/prompt_target/**"
---

# Prompt Target Development Guidelines

## Base Class Contract

All targets MUST inherit from ``PromptTarget`` (or one of its public
subclasses such as ``OpenAITarget`` / ``HTTPTarget``) and implement
``_send_prompt_to_target_async``:

```python
from pyrit.prompt_target.common.prompt_target import PromptTarget


class MyTarget(PromptTarget):
def __init__(
self,
*,
endpoint: str,
api_key: str,
max_requests_per_minute: int | None = None,
custom_configuration: TargetConfiguration | None = None,
) -> None:
super().__init__(
endpoint=endpoint,
max_requests_per_minute=max_requests_per_minute,
custom_configuration=custom_configuration,
)
self._api_key = api_key

async def _send_prompt_to_target_async(
self, *, normalized_conversation: list[Message]
) -> list[Message]:
...
```

``send_prompt_async`` (the public entry point) is ``@final`` and MUST NOT
be overridden. Override ``_send_prompt_to_target_async`` instead.

## Keyword-only ``__init__`` is enforced

Every ``PromptTarget`` subclass MUST make all ``__init__`` parameters
keyword-only (i.e., place ``*`` as the first parameter after ``self``).
``PromptTarget.__init_subclass__`` validates this at class-definition time
via ``enforce_keyword_only_init`` and raises ``TypeError`` on violations.

The check is satisfied by either of:

```python
def __init__(self, *, endpoint: str, api_key: str) -> None: ...

def __init__(self, *args: Any, **kwargs: Any) -> None: ... # *args after self
```

It rejects:

```python
def __init__(self, endpoint: str, api_key: str) -> None: ... # missing *
```

> [!NOTE]
> ``PromptTarget.__init__`` *itself* still accepts positional parameters and
> is not currently keyword-only. The ``__init_subclass__`` hook only runs for
> subclasses, so the base class non-compliance is tolerated during the warn-
> first phase. The base ``__init__`` will be reshaped to be keyword-only in
> 0.16.0 as a BREAKING CHANGE.

## Temporary opt-out: ``_brick_legacy_init``

A handful of legacy targets whose positional ``__init__`` is part of the
public API are grandfathered with ``_brick_legacy_init = True``. They
emit a ``DeprecationWarning`` at import time and the opt-out is scheduled
for removal in **0.16.0**. Do not set this flag on new targets; new
targets MUST follow the keyword-only contract.

Currently grandfathered (slated for cleanup in 0.16.0):
``HTTPTarget``, ``OpenAICompletionTarget``, ``OpenAIImageTarget``,
``PromptShieldTarget``.

## Configuration and Capabilities

- Set ``_DEFAULT_CONFIGURATION`` at the class level when your target's
capabilities differ from the base defaults (multi-turn support, non-text
modalities, JSON-mode responses, etc.).
- Accept ``custom_configuration: TargetConfiguration | None = None`` in
``__init__`` and forward it to ``super().__init__`` so callers can
override capabilities per-instance (this is required for HTTP / Playwright
targets whose capabilities depend on deployment configuration).

## Identifiable Pattern

All targets inherit ``Identifiable``. Override ``_build_identifier()`` to
include parameters that affect target behaviour:

```python
def _build_identifier(self) -> ComponentIdentifier:
return self._create_identifier(
params={"endpoint": self._endpoint, "model_name": self._model_name},
)
```

Include: endpoint, model_name, deployment identifiers, custom headers that
affect routing.
Exclude: API keys, retry counts, logging config, timeouts.

## Exports

New targets MUST be added to ``pyrit/prompt_target/__init__.py`` — both
the import and the ``__all__`` list.
9 changes: 7 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -80,11 +80,16 @@ repos:
name: Ruff (Jupyter Notebooks)
args: [--fix]

- repo: https://github.com/allganize/ty-pre-commit
rev: v0.0.43
- repo: local
hooks:
- id: ty-check
name: ty (type check)
# Run ty in the project's uv-managed venv so it can resolve project
# dependencies (tqdm etc). The third-party ty-pre-commit hook installs
# ty in an isolated env without project deps, which causes spurious
# ``unresolved-import`` errors now that the rule is set to ``error``.
entry: uv run --link-mode=copy ty check
language: system
files: ^pyrit/
exclude: ^pyrit/auxiliary_attacks/
types: [python]
4 changes: 2 additions & 2 deletions doc/code/scenarios/1_common_scenario_parameters.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@
"\n",
"from pyrit.output import output_scenario_async\n",
"from pyrit.registry import TargetRegistry\n",
"from pyrit.scenario.scenarios.foundry import FoundryStrategy, RedTeamAgent\n",
"from pyrit.scenario.foundry import FoundryStrategy, RedTeamAgent\n",
"from pyrit.setup import initialize_from_config_async\n",
"\n",
"await initialize_from_config_async(config_path=Path(\"../../scanner/pyrit_conf.yaml\")) # type: ignore\n",
Expand Down Expand Up @@ -204,7 +204,7 @@
"metadata": {},
"outputs": [],
"source": [
"from pyrit.scenario.scenarios.foundry import FoundryComposite\n",
"from pyrit.scenario.foundry import FoundryComposite\n",
"\n",
"composite_strategy = [FoundryComposite(attack=FoundryStrategy.Crescendo, converters=[FoundryStrategy.Base64])]"
]
Expand Down
4 changes: 2 additions & 2 deletions doc/code/scenarios/1_common_scenario_parameters.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@

from pyrit.output import output_scenario_async
from pyrit.registry import TargetRegistry
from pyrit.scenario.scenarios.foundry import FoundryStrategy, RedTeamAgent
from pyrit.scenario.foundry import FoundryStrategy, RedTeamAgent
from pyrit.setup import initialize_from_config_async

await initialize_from_config_async(config_path=Path("../../scanner/pyrit_conf.yaml")) # type: ignore
Expand Down Expand Up @@ -85,7 +85,7 @@
# For example, to run Crescendo with Base64 encoding applied:

# %%
from pyrit.scenario.scenarios.foundry import FoundryComposite
from pyrit.scenario.foundry import FoundryComposite

composite_strategy = [FoundryComposite(attack=FoundryStrategy.Crescendo, converters=[FoundryStrategy.Base64])]

Expand Down
2 changes: 1 addition & 1 deletion doc/code/scenarios/2_custom_scenario_parameters.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@
}
],
"source": [
"from pyrit.scenario.scenarios.airt.scam import Scam\n",
"from pyrit.scenario.airt.scam import Scam\n",
"from pyrit.setup import initialize_pyrit_async\n",
"from pyrit.setup.initializers.components import ScenarioTechniqueInitializer\n",
"\n",
Expand Down
3 changes: 1 addition & 2 deletions doc/code/scenarios/2_custom_scenario_parameters.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,7 @@
# would wire up memory and scorers):

# %%

from pyrit.scenario.scenarios.airt.scam import Scam
from pyrit.scenario.airt.scam import Scam
from pyrit.setup import initialize_pyrit_async
from pyrit.setup.initializers.components import ScenarioTechniqueInitializer

Expand Down
12 changes: 6 additions & 6 deletions doc/scanner/airt.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
}
],
"source": [
"from pyrit.scenario.scenarios.airt import RapidResponse, RapidResponseStrategy\n",
"from pyrit.scenario.airt import RapidResponse, RapidResponseStrategy\n",
"\n",
"dataset_config = DatasetConfiguration(dataset_names=[\"airt_hate\"], max_dataset_size=1)\n",
"\n",
Expand Down Expand Up @@ -254,7 +254,7 @@
}
],
"source": [
"from pyrit.scenario.scenarios.airt import Psychosocial, PsychosocialStrategy\n",
"from pyrit.scenario.airt import Psychosocial, PsychosocialStrategy\n",
"\n",
"dataset_config = DatasetConfiguration(dataset_names=[\"airt_imminent_crisis\"], max_dataset_size=1)\n",
"\n",
Expand Down Expand Up @@ -398,7 +398,7 @@
}
],
"source": [
"from pyrit.scenario.scenarios.airt import Cyber, CyberStrategy\n",
"from pyrit.scenario.airt import Cyber, CyberStrategy\n",
"\n",
"dataset_config = DatasetConfiguration(dataset_names=[\"airt_malware\"], max_dataset_size=1)\n",
"\n",
Expand Down Expand Up @@ -520,7 +520,7 @@
"metadata": {},
"outputs": [],
"source": [
"from pyrit.scenario.scenarios.airt import Jailbreak, JailbreakStrategy\n",
"from pyrit.scenario.airt import Jailbreak, JailbreakStrategy\n",
"\n",
"dataset_config = DatasetConfiguration(dataset_names=[\"airt_harms\"], max_dataset_size=1)\n",
"\n",
Expand Down Expand Up @@ -1017,7 +1017,7 @@
}
],
"source": [
"from pyrit.scenario.scenarios.airt import Leakage, LeakageStrategy\n",
"from pyrit.scenario.airt import Leakage, LeakageStrategy\n",
"\n",
"dataset_config = DatasetConfiguration(dataset_names=[\"airt_leakage\"], max_dataset_size=1)\n",
"\n",
Expand Down Expand Up @@ -1156,7 +1156,7 @@
}
],
"source": [
"from pyrit.scenario.scenarios.airt import Scam, ScamStrategy\n",
"from pyrit.scenario.airt import Scam, ScamStrategy\n",
"\n",
"dataset_config = DatasetConfiguration(dataset_names=[\"airt_scams\"], max_dataset_size=1)\n",
"\n",
Expand Down
Loading