agentkit.core.policies

The standard middleware set: tracing, meter, retry, fallback, memoize, idempotent, semantic_memoize, output_coerce, compaction, egress, audit, security.

See the Middlewares concept for the mental model.

Middlewares — the batteries. The app assembles two explicit, ordered chains (chat + tool) and hands them to the Invoker. There are two kinds:

• BaseMiddleware (transform / guard / observe) — phase classes that don't control invoking next: MeterMiddleware, Compaction, Egress, Audit, SecurityMiddleware. Lowercase factories (meter(), compaction(), …) return instances. MeterMiddleware (the recording middleware) is named to disambiguate from runtime.Meter (the Protocol it records into).
• Raw (call, next) (resilience / caching / instrumentation) — must re-invoke, skip, or wrap-with-a- context-manager next, which phase methods can't: retry/fallback (re-invoke), memoize/idempotent/ semantic_memoize (skip on hit), tracing (hold a span open across the call).

Typical chat chain: [tracing(), compaction(…), meter(), fallback([...]), retry(breaker=…)] Typical tool chain: [tracing(), meter(), egress(guardrail), idempotent(), audit(), retry(breaker=…)]

(compaction sits ahead of meter so the meter estimates tokens on the already-compacted transcript.)

audit() vs retry() — pick the trail you want. In the tool chain above retry() is INSIDE audit(), so Audit sees one outcome per logical tool call: three retried executions of a side-effecting tool fold into ONE record. That is the right shape when the question is "what did this run ask the tool to do", and it is the only ordering in which an idempotent() replay can be recorded as "deduped" (a hit short-circuits everything inner). Swap to [… idempotent(), retry(breaker=…), audit()] when the question is "how many times did the side effect actually fire" — then every attempt gets its own record, and a deduped replay produces none at all because audit() is never reached. Audit records failures either way (decision: "failed"). It does not SEE the individual attempts it sits outside of, but it does report them: retry() stamps call.meta["attempts"] and Audit reads it onto the record, so one record covering three charges says attempts=3 rather than reading exactly like one charge.

.. note:: Egress / Audit live in egress_audit.py, not security.py. A submodule named security shadowed the security() factory re-exported here — importing a submodule binds its name onto the parent package, and that binding happened AFTER the from .guard import security line, so middlewares.security was the MODULE and security() raised TypeError: 'module' object is not callable. The module was renamed to egress_audit rather than the binding patched, because reordering the imports is undone by the formatter and an explicit re-bind is one more thing to forget. tests/source/architecture/test_public_surface.py now fails on any recurrence.

Audit

Audit(*, store: Any = None, key: str = _AUDIT_LOG)

Bases: BaseMiddleware

One record per tool call — including the calls that FAILED.

Audit used to write only in on_response, i.e. only after a success. That is exactly backwards for a side-effecting tool: the dangerous case is the one that fired the side effect and then failed — the payment gateway that charged the card and timed out on the response. Measured with the documented tool chain ([… idempotent(), audit(), retry(breaker=…)]): a charging tool that raised after every attempt executed 3 times and left audit records: []. The money moved and the audit trail was empty, which is the one thing an audit trail may never be.

on_error now writes a "failed" record carrying the error type and message, then re-raises so the failure still propagates — the middleware records, it never recovers.

One record = one invocation of THIS middleware. retry() sits INSIDE audit() in the documented chain, so its attempts are invisible from here and three executions fold into one record. That is an ordering choice, not something Audit can detect: placing audit() inside retry() yields one record per attempt, at the cost of the "deduped" record (idempotent() short-circuits before audit() is ever reached). Since neither ordering gives both, the record now carries an attempts count that retry() publishes on call.meta — so one record covering three charges says three, instead of reading exactly like a record covering one. Both orderings are supported and pinned by tests; see the chain note in middlewares/__init__.

Egress

Egress(guardrail: Any)

Bases: BaseMiddleware

A tool with request.url_arg set has that URL checked (SSRF + allowlist) before it can run.

SecurityMiddleware

SecurityMiddleware(patterns: list[str] | None = None)

Bases: BaseMiddleware

Blocks malicious input before it reaches the model.

Scans the latest prompt against injection/unsafe-content patterns; on a match it emits an error observation and raises Blocked (refusing the call). Clean input passes through untouched. Drop it in the chat chain ahead of the model: Invoker(llm=…, chat_middleware=[SecurityMiddleware(), tracing(), …]).

MeterMiddleware

Bases: BaseMiddleware

as_middleware

as_middleware() -> Middleware

Compile to the (call, next) primitive — with a settle path that also runs when the stream is ABANDONED.

BaseMiddleware.as_middleware wraps the inner stream in except Exception, so on_response is reached only on normal exhaustion. GeneratorExit (a consumer that breaks out of the async for, or any early aclose()) and CancelledError are BaseExceptions: they fly straight past that handler and the charge never happens. Measured: a consumer breaking after one delta produced provider stream calls: 2 while the budget still read spent=0.25 calls=1 from the one fully-consumed call. The provider billed those tokens; the budget never saw them, so every ceiling in the run was quietly wrong — and an abandon-happy caller could spend without limit.

Design of the settle path:

• Normal exhaustion charges exactly as before, MeterExceeded included — that is the ceiling doing its job.
• An abandoned / cancelled / failed stream charges only what actually ARRIVED (if items), because that is the only spend we can honestly attest to. A pre-stream failure yielded nothing, so it is not charged and does not inflate budget.calls — which preserves today's behaviour for the retry-inside-meter chain.
• That path also swallows MeterExceeded. Budget.charge updates the ledger BEFORE it evaluates the ceiling, so the money is recorded either way; raising out of a GeneratorExit unwind would replace the close signal with an unrelated error in a consumer that has already walked away. The next on_request guard raises instead.

security

security(
    *, patterns: list[str] | None = None
) -> SecurityMiddleware

Factory returning a configured SecurityMiddleware.

Matches the lowercase-factory convention every other middleware in this package follows (tracing(), retry(), meter(), etc.). Kept alongside the class so callers wanting fine-grained control can still pass SecurityMiddleware(...) directly.

idempotent

idempotent(*, store: Any = None) -> Middleware

Dedupe SIDE-EFFECTING tool calls on (run, scope, tool, args); a failure is never stored.

This is the deliberate "safe to replay" case memoize() refuses to guess at, so it is the one caller of allow_side_effects=True. It stays safe because the key is pinned to correlation_id: replay is confined to a single run (a retry of this call), never across runs.

when uses :func:_side_effecting rather than the request flag alone — ToolRequest.side_effecting defaults to False, so a caller building the request positionally got NO idempotency for a tool that declares itself side-effecting on the tool object.

semantic_memoize

semantic_memoize(
    *,
    vector: Any = None,
    threshold: float = 0.85,
    text: Callable[[Call], str] | None = None,
    when: Callable[[Call], bool] | None = None,
) -> Middleware

Near-duplicate reuse for READ-ONLY calls over a VectorPort (scored search), scope-isolated. NEVER attach to a side-effecting call — masking a real action behind a hit would be a correctness bug. That was documented as "guard with when=" while when defaulted to None, i.e. no guard at all: the same hole memoize() had, and worse here because a near-duplicate suffices. The guard is now enforced (:func:_side_effecting) instead of requested; when= still narrows further on top of it.

fallback

fallback(
    models: list[str],
    *,
    classify_fn: Callable[
        [BaseException], ErrorClass
    ] = classify,
    breakers: dict[str, Any] | None = None,
) -> Middleware

models is the ordered fallover chain; the served model is left on request.model. Falls over on a PRE-stream transient failure / OPEN breaker; commits to a model once its first delta flows.

retry

retry(
    *,
    breaker: Any = None,
    max_attempts: int = 3,
    classify_fn: Callable[
        [BaseException], ErrorClass
    ] = classify,
    sleep: Callable[[float], Any] | None = None,
) -> Middleware

Re-invoke on a PRE-stream failure (before the first delta — the common case: rate-limit/connect/OPEN breaker); once the first delta is yielded, commit and stream through (a mid-stream failure propagates — you can't un-send tokens). PERMANENT errors fail fast.