The review flag

A scan result carries one decision to act on, action, which is "allow" or "block". When the engine was uncertain about that decision, the result also carries needs_review=True. The flag does not hold the request: the action is already final, and the flag marks it as one worth a second look.

result = client.analyze(user_input)

result.action         # "block"
result.engine_action  # "review"
result.needs_review   # True

How a review becomes allow or block

The engine can answer ALLOW, REVIEW, BLOCK, or HARD_BLOCK. The gateway turns that into one of two actions before the response is sent, using a per-app setting for each content type.

Shadow mode still forces "allow", and a boundary rule can still force "block", after the collapse. Neither changes engine_action or needs_review.

The reviewCollapse setting

Each app has one allow or block value per content type. It decides what a REVIEW becomes for that type.

An absent or unrecognised value uses the default for that type. A /scan request that sets provenance uses the setting for that provenance. Knowledge base scans are not affected.

Set it per app in the dashboard, on the app's Settings page, in the card titled "Results that need review". The card has one Block or Allow choice for each content type above, labeled User prompts, Tool calls, Tool results, Retrieved chunks, Assembled context and Model responses, and the flag stays visible either way.

Choosing a setting

block converts every REVIEW into a block, which catches more attacks and also blocks some benign input. allow lets every REVIEW through, which blocks less and misses more. The figures below were measured on our own evaluation sets, with 95% Wilson confidence intervals. They show the effect of the setting on those sets, not a prediction for your traffic.

What the measurements support:

• For prompts, block raises recall substantially and costs false positives, most visibly on benign prompts that resemble injections.
• For retrieved documents, block would block roughly half or more of the real benign documents measured, which is why rag_chunk and rag_context default to allow.
• On real benign documents the flag itself is frequent (45.6% [42.9, 48.2] of the retrieved chunks measured), so for retrieved content it is a coarse review queue.
• The tool call and tool output sets are small and synthetic. There is no labelled data for llm_response.

Using the flag

Tune thresholds with it

needs_review marks the decisions nearest the engine's cut points, so it is the natural place to look when deciding whether a setting is too strict or too loose.

• Filter the dashboard's Jobs list by Verdict, choosing Needs review, and read a sample. Flagged items that were blocked and are fine argue for allow on that content type. Flagged items that were allowed and are not fine argue for block.
• Record those judgments with the job feedback controls.
• Count flagged scans per period from the review and needsReview counts of the Analytics API, or per content type in your own logs.

Route flagged items to a second model

A flagged result is a cheap trigger for a second opinion. The pattern is the same in every integration: act on action as usual, and call a second reviewer only when needs_review is True. second_opinion below is your own function, backed by whatever model or service you choose.

def second_opinion(*, text: str, stage: str, risk: float | None, rationale: str) -> str:
    """Return "safe" or "unsafe". Replace with a call to your reviewer."""
    raise NotImplementedError


result = client.analyze(user_input)

if result.needs_review:
    verdict = second_opinion(
        text=user_input,
        stage="prompt",
        risk=result.risk,
        rationale=result.rationale,
    )
    if verdict == "unsafe":
        raise PermissionError("Blocked after review")
    # verdict == "safe": proceed even if result.action was "block"
elif result.action == "block":
    raise PermissionError("Blocked")

With the OpenAI Agents SDK guardrails, pass on_review. It is called once per scan whenever needs_review is True, whether or not the scan tripped the guardrail, and before the tripwire decision. on_review takes the ScanResult and may be a plain function or a coroutine function. An error raised inside it does not affect the scan: it goes to the optional on_error(exc) callback, or is dropped when on_error is not set.

from tawaguard.agents_sdk import tawa_input_guardrail


def log_hook_error(exc):
    logger.warning("on_review failed: %s", exc)


def queue_for_review(result):
    review_queue.put({"risk": result.risk, "rationale": result.rationale})


guardrail = tawa_input_guardrail(
    tawa, on_review=queue_for_review, on_error=log_hook_error
)

on_review observes. It does not change whether the guardrail trips; to override a block after a second opinion, use the raw SDK pattern above.

on_review is also accepted by the tool guardrails (tawa_tool_input_guardrail, tawa_tool_output_guardrail, tawa_tool_guardrails and guard_function_tools, from 0.3.0). It behaves the same way: it is called with the ScanResult of a tool call or tool result whose needs_review is True, and cannot change the decision.

Retrieval scans

scan_retrieval applies the setting to each chunk. aggregate.worst_action and aggregate.flagged_chunk_ids are computed from the collapsed actions. aggregate.needs_review and aggregate.review_chunk_ids list the chunks the engine flagged, including the ones enforced as "allow".

result = client.scan_retrieval(query, chunks)

result.aggregate.flagged_chunk_ids  # enforced as block
result.aggregate.review_chunk_ids   # flagged for review, blocked or not

What is stored

The Analytics API counts persisted jobs, so its review count (also returned as needsReview) includes flagged scans that you received as "allow" or "block".

Older SDKs and gateways

The new fields are additive.

• An SDK without needs_review ignores the fields it does not know. Its action is still a valid value ("allow" or "block").
• A new SDK against a gateway that does not send the fields derives needs_review from action == "review" and engine_action from action. Against such a gateway action is the engine's raw label, so "review" can still appear.
• Do not require the fields to be present. Treat a missing needs_review as False in clients you maintain yourself.