ZeroLoss-Lab/egs-bert-response-auditor

Educational Guardrail System - Holistic Response Auditor

Model Details

• Model Name: Holistic Response Auditor
• Model Architecture: BERT sequence classification model
• Task: Response Moderation
• Domain: K-12 Education & General AI Safety
• Language: English and Chinese

Model Description

This model serves as the Holistic Response Auditor, the third and final stage of the Educational Guardrail System (EGS). Unlike input filters, this model is specifically fine-tuned to audit the output streams generated by Large Language Models (LLMs) to ensure pedagogical safety and compliance.

Key Mechanism: Sliding Window Auditing

As described in the EGS design, this model uses a sliding window approach (e.g., 512 tokens) to process long generated texts. It is designed to be a "Fail-Fast" auditor:

• The auditor evaluates the generated text stream in discrete segments.
• If any single segment is flagged as toxic or non-compliant, the generation can be stopped or routed for secondary handling before the learner is exposed to the content.

Label Mapping

• UNSAFE: High-risk response segment
• SAFE: Low-risk response segment
• AMBIGUOUS: Medium-risk or unclear response segment

Intended Use

This model is intended to be used as a post-generation filter for Educational LLMs.

• Input: The text generated by an LLM (Draft Response).
• Output: Three-way classification (UNSAFE / SAFE / AMBIGUOUS).
• Application: Detecting residual toxicity, hallucinations, or age-inappropriate content that bypassed input moderation.

Usage Example (Sliding Window)

To replicate the paper's logic, you should apply the model over segments of the response:

from transformers import pipeline

# Load the auditor
model_id = "ZeroLoss-Lab/egs-bert-response-auditor"
auditor = pipeline("text-classification", model=model_id)

def audit_stream(text_stream, window_size=512):
    """
    Audits an LLM response stream using the Holistic Response Auditor.
    """
    chunks = [text_stream[i:i+window_size] for i in range(0, len(text_stream), window_size)]

    for chunk in chunks:
        result = auditor(chunk)[0]
        if result["label"] == "UNSAFE":
            return False, "Blocked: Unsafe content detected."

        if result["label"] == "AMBIGUOUS":
            return False, "Needs secondary review: Ambiguous content detected."

    return True, "Passed"

Limitations

This model is intended as a post-generation safety component, not as a complete safety solution. It should be combined with input filtering, policy logic, human review pathways where appropriate, and task-specific evaluation before deployment.