Questions and thresholds

from jes import YesNo, Choice, Score, Threshold
from jes.questions import YesNoAnswer, ChoiceAnswer, ScoreAnswer, violation_score

Questions are what a judgment asks a backend. Questions never depend on the checked text, so attacker text never enters the instructions.

Question types

YesNo

YesNo(
    instructions: str,
    true: str | None = None,
    false: str | None = None,
)

A question whose "true" score means violation. true and false optionally describe each answer. The violation score is the score of true.

Choice

Choice(
    instructions: str,
    options: Mapping[str, str | None],   # label → description, at least 2
)

A categorical question. Each label must be a valid identifier, such as billing. The policy's violating= names the violating options. The violation score is the sum of their scores.

Score

Score(
    instructions: str,
    levels: tuple[str, ...],             # 2 to 10, ordered low to high
)

An ordered scale. Levels must be unique and non-empty. The policy's violation_level= is the index of the lowest violating level. The violation score is the sum of the scores at or above it.

Question

Question = YesNo | Choice | Score, the type alias for any of the three.

Instructions must not be empty. A question that breaks a rule raises PolicyError.

Answer types

Backends return one answer per question. You build them yourself only in tests, with FakeBackend.

Type	Fields	Rule
YesNoAnswer	score: float, confidence: float | None = None	score is P(yes), in [0, 1]
ChoiceAnswer	scores: Mapping[str, float], confidence: float | None = None	Keys equal the options, and scores sum to 1 ± 1e-3
ScoreAnswer	scores: tuple[float, ...], confidence: float | None = None	One per level, lowest first, summing to 1 ± 1e-3

Answer = YesNoAnswer | ChoiceAnswer | ScoreAnswer. An answer that breaks the range, sum, or key rules above raises BackendError.

confidence is optional backend metadata in [0, 1]: how settled the answer is, not its violation score. jes validates the range and records it in ScoreResult.confidence, but never uses it to decide. See What confidence means.

violation_score

violation_score(
    question: Question,
    answer: Answer,
    *,
    violating: Collection[str] = (),
    violation_level: int | None = None,
) -> float

The probability that the text violates the policy. A YesNo uses P(yes). A Choice sums the violating options. A Score sums violation_level and every level above it. Judgments call it for you; use it to test your own questions. An answer that doesn't match its question raises BackendError.

Threshold

Threshold(block_at: float, flag_at: float | None = None)

• A violation score at or above block_at blocks.
• A violation score from flag_at up to block_at flags.
• A plain float x means Threshold(block_at=x). Threshold.coerce(value) does that conversion.
• threshold.action(score) returns "block", "flag", or None.
• Construction raises PolicyError for booleans, NaN, infinities, and values outside 0 <= flag_at <= block_at <= 1.

See Thresholds and scores.

Example

from jes.policies import judge
from jes.questions import Choice, Score

route = judge(
    "route",
    Choice(
        "Which team should handle this?",
        {"billing": "Money problems", "other": "Anything else"},
    ),
    threshold=0.5,
    violating=["billing"],
    stages=("input",),
)

severity = judge(
    "severity",
    Score("How urgent is the message?", ("low", "medium", "high")),
    threshold=0.5,
    violation_level=2,
)