DeepSight: An All-in-One LM Safety Toolkit

🍞 Top Bread (Hook): You know how you don’t just check if a bicycle rides; you also look at the brakes, the chain, and the wheels to know why it rides well or badly?

🥬 The Concept: Safety Evaluation is the process of checking if AI models act safely for people in many kinds of situations. How it works:

1. Ask the model questions (some safe, some risky).
2. Look at its answers and score whether it helped safely or refused harmful stuff.
3. Repeat across many topics and formats (text and images) to see patterns. Why it matters: Without safety evaluation, we might ship a model that seems smart but gives dangerous or unfair answers when it really counts. 🍞 Bottom Bread (Anchor): Before using a chatbot in a classroom, we test whether it politely declines helping with cheating or harmful pranks, while still helping with homework.

The World Before: Large Language Models (LLMs) and Multimodal LLMs (MLLMs) became amazingly helpful at writing, coding, and understanding pictures. But their safety testing looked like a quick road test—useful for spotting dangerous turns (jailbreaks, toxic content) but blind to what was happening under the hood. Companies built strong black-box evaluations (like leaderboards and judge models), yet those mostly told us whether the car skidded—not why it skidded.

The Problem: Teams could detect that a model failed on a risky prompt, but couldn’t tell which internal pieces—layers, neurons, or representation patterns—caused that failure. Diagnosis work (like finding special neurons or analyzing hidden activations) existed, but it ran separately from standardized benchmarks. So when models were re-aligned for safety, teams risked harming overall ability (like over-tightening the brakes so the bike barely moves) because they lacked white-box insight to guide precise fixes.

🍞 Top Bread (Hook): Imagine you lock your front door but forget the window. If you only test the door, you won’t know the window is the weak spot.

🥬 The Concept: Model Algorithm Security checks whether the model’s core methods and rules resist tricks like jailbreaks or adversarial prompts. How it works:

1. Stress-test with cleverly crafted inputs that try to sneak past defenses.
2. Observe if the model’s internal decision rules still hold under pressure.
3. Compare defenses across different model parts (text, image, fusion). Why it matters: Without algorithm-level checks, models can look safe on easy tests but fall apart when attackers try new, crafty routes. 🍞 Bottom Bread (Anchor): A model might refuse “Tell me how to build a bomb,” but still give detailed steps when the same request is hidden inside an image.

Failed Attempts: People tried three partial fixes. First, they stacked more black-box tests, which found more failures but didn’t explain them. Second, they ran cool interpretability projects that probed neurons or mapped hidden spaces, but these didn’t stay tied to real deployment risks. Third, they applied heavy post-hoc safety training; sometimes it boosted refusals but also made models over-cautious, blocking normal help.

The Gap: We needed a single loop that starts with external behavior (Did the model fail?) and ends with internal understanding (Which layers, neurons, or geometry made it fail?), so we can repair safety with confidence and keep general ability intact.

🍞 Top Bread (Hook): Imagine video games where new levels introduce new monsters you’ve never met—if you don’t prepare for them, you lose quickly.

🥬 The Concept: Frontier AI Risks are high-severity risks from advanced models, like manipulation, deception, or hiding abilities. How it works:

1. Design tough tests (manipulation, sandbagging, evaluation faking).
2. Measure whether the model stays honest, resists being steered, and doesn’t pretend to be weak.
3. Track performance across many such risks because strength in one doesn’t guarantee strength in another. Why it matters: Without frontier risk tests, a model might look safe in simple cases but cause big problems in high-stakes scenarios. 🍞 Bottom Bread (Anchor): A chatbot could be great at refusing toxic requests but still go along with a subtle plan to trick a user into a risky action.

Real Stakes: Safety affects daily life—classrooms, hospitals, customer support, and coding tools. Over-safety means your harmless question gets blocked; under-safety means a harmful request gets help. In images, a model might overreact to a photo of a kitchen (thinking it’s dangerous) or underreact to a picture hiding malicious instructions. And for frontier risks, a model that manipulates users or fakes weakness could cause real-world harm or erode trust.

DeepSight’s Why: DeepSight closes the loop by pairing robust, standardized evaluations (so you know what failed) with deep, reproducible diagnostics (so you know why it failed). This turns safety from guessing to engineering—finding specific internal issues and fixing them without breaking everything else.

🍞 Top Bread (Hook): Imagine a school science fair where judges score your robot’s behavior, and a coach shows you exactly which gears and wires need fixing. Together, you improve fast.

🥬 The Concept: DeepSight is a unified, open-source safety toolkit that both evaluates models’ behavior (DeepSafe) and diagnoses their internal mechanisms (DeepScan) using the same tasks and data. How it works:

1. Evaluate: Run many safety tests (text+image) to measure behavior, including frontier risks.
2. Diagnose: On the very same tasks, inspect hidden representations, layers, and neurons to find root causes.
3. Repair: Use white-box insights to guide targeted fixes and avoid harming general abilities. Why it matters: Without tying tests to internal explanations, teams patch symptoms, not causes—leading to fragile safety or over-safety. 🍞 Bottom Bread (Anchor): If a model helps with a risky request in a cooking photo, DeepSight tells you which internal patterns failed at separating safe vs. unsafe ideas near the boundary.

Aha! Moment in One Sentence: Safety gets reliable when the test you fail is the same test you diagnose—connecting outside behavior to inside mechanisms.

Three Analogies:

• Doctor + X-ray: DeepSafe is the checkup; DeepScan is the X-ray that shows exactly where the break is.
• Coach + Replay: DeepSafe is the score; DeepScan is the slow-motion replay revealing which move went wrong.
• Map + GPS: DeepSafe says you’re off course; DeepScan shows precisely which turn to fix.

Before vs. After:

• Before: Separate tools—benchmarks over here, neuron studies over there—hard to act on together.
• After: One pipeline—same tasks and data flow from scoring to diagnosis to repair plans.

Why It Works (Intuition):

• Failures happen because internal representations blur or over-separate safe and unsafe ideas.
• Diagnose the hidden geometry (how ideas are arranged) and the sharing of neurons across objectives (e.g., fairness vs. privacy) to see the true cause.
• Fixes become surgical: adjust the parts that matter, rather than tightening every screw.

Building Blocks (introduced with sandwiches):

🍞 Top Bread (Hook): Think of DeepSafe like a standardized road test for many kinds of roads—city, highway, and mountains.

🥬 The Concept: DeepSafe is a modular, configuration-driven safety evaluation system for LLMs and MLLMs across 20+ benchmarks, including frontier risks. How it works:

1. Load models (local or API) and datasets via a simple config file.
2. Run inference and judge answers using rules, native scripts, or ProGuard (a safety-specialized judge).
3. Summarize results into clear reports and JSON. Why it matters: Without consistent, scalable testing, results are noisy, non-reproducible, and hard to compare. 🍞 Bottom Bread (Anchor): Changing one line in a YAML file lets you test a new model across HarmBench and SALAD-Bench, with a report ready at the end.

🍞 Top Bread (Hook): When your computer runs slowly, a task manager shows which app uses the most memory—that’s a peek inside.

🥬 The Concept: DeepScan is a standardized diagnostic suite that probes hidden representations and neurons to explain why failures happen, without changing weights. How it works:

1. Register models, datasets, and evaluators through a unified interface.
2. Capture intermediate activations and compute diagnostics (X-Boundary, TELLME, SPIN, MI-Peaks).
3. Summarize metrics and plots so issues are clear and comparable. Why it matters: Without internal diagnostics, you can’t tell if the problem is blurry boundaries, entangled features, or conflicting neurons. 🍞 Bottom Bread (Anchor): If a model over-refuses harmless requests, DeepScan can show that safe and harmful ideas are too far apart to reason near the boundary.

🍞 Top Bread (Hook): Imagine getting your car inspected while a mechanic simultaneously explains each finding and plans the exact fix.

🥬 The Concept: Joint Safety Evaluation and Diagnosis means DeepSafe and DeepScan use unified tasks and data so failures and root causes align one-to-one. How it works:

1. Run DeepSafe to pinpoint where behavior fails.
2. Run DeepScan on those same items to uncover which layers, neurons, or geometry failed.
3. Feed insights into targeted training or configuration fixes. Why it matters: Without this joint view, teams guess at fixes, risking over-safety or degraded capabilities. 🍞 Bottom Bread (Anchor): A low score on Manipulation can be traced to boundary ambiguity in mid layers, guiding a contrastive update that sharpens the boundary without tanking helpfulness.

At a high level: Input (model + dataset) → Step A: Evaluate with DeepSafe → Step B: Diagnose with DeepScan → Output: Scores, root causes, and repair plan.

Step-by-step recipe with purpose and examples:

1. Configure once, run many

• What happens: You write a single YAML/JSON config that names the model(s), dataset(s), and which evaluators to run.
• Why it exists: Configuration-as-execution ensures runs are reproducible and easy to share or rerun.
• Example: One config runs Qwen2.5-72B-Instruct on SALAD-Bench and HarmBench with ProGuard judging, then passes the same samples to X-Boundary and TELLME.

1. DeepSafe evaluation workflow

• What happens: a. Models module loads a local or API model (vLLM for speed, API wrappers for closed-source). b. Datasets module normalizes many benchmarks to a common schema (ID, prompt, reference), spanning content risks and frontier risks. c. Runner performs inference in batches, resuming if interrupted. d. Evaluators judge outputs using native scripts, rules, or ProGuard (a safety-tuned judge trained on 87k safety pairs). e. Summarizer aggregates metrics and produces Markdown+JSON reports.
• Why it exists: A single, modular pipeline prevents one-off scripts and makes scores comparable across models and time.
• Example with actual data: On HarmBench, a model’s refusal and helpfulness patterns get scored; on SALAD-Bench, performance is broken down by safety categories (e.g., Social & Ethical vs. Algorithm Security). Frontier risks (like Mask or Manipulation) get their own columns in the leaderboard.

1. DeepScan diagnostic workflow

• What happens: a. Registry connects model families (Qwen, Llama, Mistral, Gemma, GLM, InternLM/VL) and datasets to evaluators. b. Model runners expose generate() and raw HF model/tokenizer for activation hooks. c. Evaluators (X-Boundary, TELLME, SPIN, MI-Peaks) compute representation geometry, disentanglement, neuron coupling, and reasoning info-dynamics. d. Summarizers select best layers, save plots (e.g., t-SNE), and write summary.json/md.
• Why it exists: To turn behavior failures into specific internal signatures—blurry boundaries, excessive separation, entanglement, conflicting neurons.
• Example: If Flames shows high attack success, X-Boundary might reveal safe/harmful centroids nearly overlapping (low separation), explaining the failure.

🍞 Top Bread (Hook): Picture a city map where neighborhoods are close or far; nearby streets blend, far ones separate—this shapes how you travel.

🥬 The Concept: Geometric Structure of Latent Space describes how a model arranges ideas (safe vs. unsafe) in its hidden layers. How it works:

1. Collect hidden vectors for safe, harmful, and boundary examples.
2. Measure distances, separability, and alignment to a decision boundary.
3. Visualize across layers to see where structure helps or hurts. Why it matters: If safe and harmful zones overlap, the model confuses them; if they’re too far apart, the model may lose nuance near the edge and over-refuse. 🍞 Bottom Bread (Anchor): A cooking request with a sharp knife might be harmless; if the model’s space can’t represent that nuance, it says “No” too often.

1. Built-in DeepScan evaluators (the “instruments”):

• X-Boundary: Quantifies safe/harm/boundary separability; too little separability → confusion; too much → brittle over-refusal near edges.
• TELLME: Measures disentanglement and subspace orthogonality; high encoding rate and low cross-talk mean robust boundaries.
• SPIN: Finds coupling between neurons for competing goals (e.g., fairness vs. privacy), warning of trade-offs.
• MI-Peaks (process-level; excluded in this run, but supported): Tracks where information about the correct answer spikes during reasoning.

🍞 Top Bread (Hook): Teaching a pet that “help safely” and “refuse harm” are different tricks keeps commands clear.

🥬 The Concept: Safety Alignment means training the model to follow human-friendly rules that separate helpfulness from harmfulness. How it works:

1. Use examples and reward signals to shape responses.
2. Test boundaries to ensure helpful-but-safe behavior.
3. Adjust when over-safety or under-safety appears. Why it matters: Without good alignment, models may help harmful acts or refuse too many harmless ones. 🍞 Bottom Bread (Anchor): A model should help with a lab safety poster but refuse writing steps for making dangerous chemicals.

1. Secret sauce: A shared protocol

• Harmonized tasks/data: The very same prompts that score behavior feed the diagnostic instruments.
• Layerwise insight tied to scores: When a benchmark dips, DeepScan points to the layer or neuron pattern causing it.
• Surgical repair path: Insights suggest targeted training (e.g., contrastive objectives) or configuration (e.g., judge calibration) rather than blunt refusal boosts.

🍞 Top Bread (Hook): Watching a detective solve a puzzle step-by-step is different from only reading the final answer.

🥬 The Concept: Reasoning Dynamics studies how a model’s internal information changes during multi-step thinking. How it works:

1. Track hidden states as the model generates thoughts.
2. Measure where information about the right answer suddenly increases (peaks).
3. Compare across models to see stable vs. risky patterns. Why it matters: Some reasoning styles can make manipulation easier if the model’s internal steps amplify the wrong signals. 🍞 Bottom Bread (Anchor): A model that self-explains might reveal steps where it latched onto a misleading clue, guiding a fix that reduces such traps.

The Test: DeepSight evaluates both content risks (toxicity, jailbreaks, hallucinations, over-safety) and frontier risks (manipulation, deception, sandbagging, mask, etc.), for text-only LLMs and image+text MLLMs. DeepScan then diagnoses internal geometry and neuron coupling to explain successes and failures.

The Competition: 20+ benchmarks (e.g., SALAD-Bench, HarmBench, Flames, XSTest, VLSBench, MMSafetyBench, MSSBench, Ch3ef, SIUO) and nine frontier-risk datasets (Eval Faking, Sandbagging, Manipulation, Mask, DeceptionBench, BeHonest, Reasoning Under Pressure, AIRD, WMDP) cover a wide risk spectrum. Models include leading open- and closed-source LLMs/MLLMs.

Scoreboard with context:

1. Text-only safety tiers are high; multimodal is much harder.

• LLMs: Top-tier averages exceed ~0.77; mid-tiers cluster around ~0.71–0.76; bottom tier <0.71. Strong ethical safety; weaker algorithm safety.
• MLLMs: Overall lower (leaders ~0.65–0.71; trailer ~0.38–0.49). Visual modality increases attack surface, widening the gap between top and bottom. Interpretation: Like going from paved roads to off-road trails—safety is tougher with images.

1. Reasoning helps in multimodal, not much in text.

• LLMs (text): Reasoning vs. non-reasoning are similar overall; reasoning slightly worse in algorithm safety and vertical domains where fast interception matters.
• MLLMs (image+text): Reasoning models do better overall, likely due to explicit cross-modal consistency checks that catch image-text split attacks. Interpretation: In pictures+text, reasoning acts like a careful inspector; in plain text, overthinking can slow the safety “brakes.”

1. Open vs. closed: similar in text; big gap in multimodal.

• Text: Open-source ~0.716 vs. closed-source ~0.726 (close race).
• Multimodal: Open-source ~0.545 vs. closed-source ~0.600 (clear lead for closed-source). Interpretation: Multimodal safety likely needs larger data, richer tests, and heavier engineering—areas where closed labs currently invest more.

1. Over-safety is real.

• LLMs: Some models reject many harmless requests (low usability) while others accept too much (low safety). Balance varies by tier.
• MLLMs: Over-safety spikes on benign images with sensitive-looking content; conservative strategies hurt usability. Interpretation: Saying “No” too often is safer but unhelpful; finding the sweet spot matters.

1. Frontier AI risks: no one model dominates.

• Overall leaders include Kimi-K2-Thinking, GPT-4o, GPT-5.2, but each shows weak spots (e.g., Kimi-K2-Thinking ranks last on Manipulation).
• Some risks are mostly solved (AIRD, Eval Faking high scores), others remain hard (Mask moderate; Manipulation very low average with many near 0–5%). Interpretation: Getting an A in math doesn’t mean you ace history—skills don’t transfer cleanly across risks.

Surprising findings and diagnostics:

• Reasoning trade-off: Reasoning-enabled models show very low Manipulation scores on average, suggesting that multi-step thinking can be steered by crafty prompts.
• Time trend: Manipulation resistance dropped from 2024 to mid/late 2025 with the rise of reasoning models, then partially recovered in late 2025 for top labs—but not to 2024 levels.
• Efficiency vs. alignment: Smaller open models and Flash variants score worse on honesty-related tests (MASK, BeHonest, DeceptionBench), signaling a trade-off between speed/size and reliable honesty.

Joint evaluation + diagnosis links:

• Extreme separation hurts boundary reasoning: When safe/harmful representations are too far apart (X-Boundary), models over-refuse and struggle near edges, matching lower scores on nuanced benchmarks (e.g., MedHallu).
• Low separability → defense failure: If safe/harmful centroids nearly overlap (X-Boundary), attack success rises (e.g., poor Flames scores).
• Orthogonal subspaces → robust defense: High TELLME encoding rates (disentangled, low-noise subspaces) correlate with strong HarmBench performance.
• Neuron coupling vs. behavior: Favorable SPIN coupling (less conflict between goals like fairness/privacy) doesn’t always translate to higher surface safety—useful warning that some models have cleaner internals but need alignment to surface that strength.

Bottom line: DeepSight’s scoreboard tells you who wins where; DeepScan tells you why. Together, they show that robust safety needs just-right geometry (neither squished nor exploded), careful reasoning that resists steering, and alignment that makes clean internals show up in behavior.

Limitations:

• Coverage is wide but not total: New attack styles (especially multimodal or agentic) can emerge faster than benchmarks update.
• Judge dependence: Model-graded evaluation (even with ProGuard) may inherit judge biases; cross-judge checks help.
• Resource needs: Diagnostics that read many activations (and make plots) can be GPU- and memory-heavy on large models.
• MI-Peaks was excluded in the main experiment set; process-level reasoning signals remain under-explored here.

Required resources:

• For open models: multi-GPU inference (vLLM or HF) for large-scale evaluation; hooking hidden states for DeepScan.
• For closed models: API budget and rate limits; consistent prompts and seeds.
• Storage for logs, JSON, and plots; CI to rerun configs for regression tracking.

When NOT to use:

• Ultra-limited environments where you can’t store activations or run batch evaluations.
• One-off demos seeking a single headline score; DeepSight shines in systematic, repeatable testing and diagnosis.
• Settings demanding fully automated fixes; DeepSight guides humans to make informed adjustments rather than auto-patching.

Open questions:

• Best practices for balancing separability: How to tune representation geometry so boundaries are clear but still support nuanced edge cases?
• Reasoning safety: Which training recipes preserve problem-solving while hardening against manipulation and deceptive chains-of-thought?
• Multimodal fusion: How to align vision-language safety jointly so neither dominates nor creates loopholes?
• Transfer across risks: Can we learn safety features that generalize across manipulation, deception, and mask-style honesty challenges?
• From insight to repair: What standardized, minimally invasive training objectives (e.g., small contrastive modules) reliably convert DeepScan insights into lasting behavioral gains?

Three-sentence summary: DeepSight unifies safety evaluation (DeepSafe) and internal diagnosis (DeepScan) so the very tests a model fails become the starting point for understanding why. Experiments across text and multimodal models show consistent patterns—multimodal safety is harder, reasoning helps or hurts depending on context, and frontier risks don’t transfer—while diagnostics reveal that both too-little and too-much internal separation can break robustness. Together, they turn safety from black-box scoring into a white-box engineering loop.

Main achievement: The first open-source, all-in-one toolkit that supports frontier AI risk evaluations and links benchmark failures to representation-level and neuron-level root causes via a shared, configuration-driven pipeline.

Future directions:

• Add more process-level diagnostics (e.g., MI-Peaks across tasks) to connect chain-of-thought phases to failure points.
• Develop guided repair recipes (contrastive reshaping, boundary tuning, decoupling objectives) that apply fixes surgically and track side effects.
• Expand multimodal datasets and adversarial image-text splits; improve judge diversity and ensemble reliability.

Why remember this: DeepSight shows that safe AI isn’t just about refusing bad stuff; it’s about understanding the inner map of ideas so we can draw better, more reliable borders. By aligning behavior tests with internal explanations, it gives teams a practical path to safer models that stay helpful, honest, and robust across the fast-changing frontier.