SmartSnap: Proactive Evidence Seeking for Self-Verifying Agents

🍞 Hook: Imagine you ask a friend to set an alarm on your phone, and they say, “Done!” but don’t show you the alarm screen. Do you trust them? Probably not—you want proof.

🥬 The World Before: Many AI agents can tap, type, and swipe through apps to follow our instructions, like adding a calendar event or turning on dark mode. But one giant missing piece makes training them hard: knowing for sure if they actually finished the task. People used to write long, app-specific rules to check success (like reading hidden app states), or they sent the agent’s entire step-by-step history to a big AI judge to decide later. Both ways were clunky—either lots of manual work or super expensive and unreliable.

🍞 Anchor: It’s like grading a homework assignment. You can create a special answer key for each problem (time-consuming), or read every scribble a student made (slow and noisy). Neither feels great.

🍞 Hook: You know how teachers don’t just care if you got the right answer, but also if you can show your work clearly?

🥬 Problem: Existing verification is passive and post-hoc—agents do stuff, then a separate judge tries to figure out what happened from a huge, messy transcript. This causes two big problems: (1) High cost and delay, because the judge must read everything; (2) More hallucinations and wrong decisions, because long, noisy contexts confuse judges.

🍞 Anchor: It’s like checking if a cake is baked by reading a 10-page diary of the baker’s day, instead of simply looking at a toothpick test.

🍞 Hook: Think of a coach who watches the play and asks players to record key moments to review later.

🥬 Failed Attempts:

• Rule-based scripts: Engineers wrote per-app checkers using hidden states. They worked in narrow settings but were hard to build and didn’t scale to new apps.
• Full-trajectory VLM judges: A single vision-language model looked at every screenshot and message. It was more general but very costly and often got lost in the noise.

🍞 Anchor: Like trying to find one clue in a whole season of game footage—expensive and easy to miss the key part.

🍞 Hook: You know how it’s easier to trust someone who shows you just the right photos that prove their story?

🥬 The Gap: The agent wasn’t helping with verification. It acted first and hoped a judge could make sense of everything later. What was missing was an agent that plans not only to do the task but also to collect just-enough evidence while doing it.

🍞 Anchor: A delivery driver who snaps a photo of the package at your door makes proof simple and reliable.

🍞 Hook: Imagine a treasure hunt where you must bring back the exact items as proof, not just tell a story about what you did.

🥬 Why This Matters: Training agents at scale needs automatic, trustworthy success signals. Without that, we waste time and money and risk teaching bad habits. If agents can prove success themselves with minimal, strong evidence, we get cheaper training, fewer mistakes, and faster progress toward useful assistants on phones and computers.

🍞 Anchor: If your phone agent can both set an alarm and show the alarm turned on in one crisp snapshot, you’ll trust it—and we can train it more easily.

— New Concepts —

🍞 Hook: Imagine teaching a pet a trick by giving treats when it does the right steps and ignoring the rest.

🥬 Reinforcement Learning (RL): RL is a way to train an agent by rewarding good behavior and not rewarding bad behavior.

• How it works: (1) The agent tries actions; (2) It gets feedback (rewards); (3) It learns which actions lead to good outcomes.
• Why it matters: Without RL, agents only copy examples and don’t improve from their own mistakes.

🍞 Anchor: Like a game where you learn by getting points for scoring and none for misses.

🍞 Hook: Think of a board game where the next move depends only on the current position, not the whole past.

🥬 Markov Decision Process (MDP): An MDP is a recipe for decision-making where the agent looks at the current state, picks an action, and moves to a new state, aiming for long-term rewards.

• How it works: (1) See state; (2) Choose action; (3) Environment changes; (4) Repeat until done.
• Why it matters: Without MDPs, it’s hard to structure how agents learn from steps and outcomes.

🍞 Anchor: Like navigating a maze: you see where you are now, pick a turn, and keep going.

🍞 Hook: Picture a smart helper that can read instructions and operate your phone like a human.

🥬 Agentic Reinforcement Learning: Agentic RL means training such helpers (agents) to use tools and interfaces through RL.

• How it works: (1) Read a task; (2) Interact with the app; (3) Get rewards for success; (4) Improve.
• Why it matters: Without Agentic RL, we can’t scale learning for real apps and devices.

🍞 Anchor: It’s like teaching a robot friend to use your tablet by trying, failing, and getting better with feedback.

🍞 Hook: You know how a referee decides if a goal counts by looking at clear evidence?

🥬 Task Verification: Task verification is deciding if an agent actually completed the task.

• How it works: Check proof against the instruction and rules.
• Why it matters: Without verification, we can’t tell success from failure and can’t reward correctly.

🍞 Anchor: Like confirming homework by checking the final answers and key steps.

🍞 Hook: Think of a very smart reader who can grade essays.

🥬 LLM-as-a-Judge: A Large Language Model (LLM) reads the agent’s output and decides if the task was done right.

• How it works: Feed the instruction and agent’s evidence to the LLM; it returns a verdict and feedback.
• Why it matters: Without an automatic judge, large-scale training isn’t practical.

🍞 Anchor: Like a fair teacher who scores your work based on what’s written, not guesses.

🍞 Hook: Imagine someone who can look at pictures and read text together.

🥬 Vision-Language Models (VLMs): VLMs understand both images and text to make decisions.

• How it works: Process screenshots plus words; reason about them together.
• Why it matters: Without VLMs, we can’t judge tasks that need seeing the screen.

🍞 Anchor: Like solving a picture riddle by reading the caption and looking at the photo.

🍞 Hook: You know how great students don’t just finish their project—they also include the perfect photos and captions that prove they did it?

🥬 The “Aha!” Moment: Make the agent itself collect and submit a tiny set of decisive, grounded snapshots as proof while it works, so the judge can verify quickly and safely.

• What it is: SmartSnap trains a Self-Verifying Agent that both completes the task and builds its own proof package.
• How it works: (1) The agent does the task; (2) It picks key action→result pairs as exhibits; (3) If proof is missing, it takes extra, evidence-focused actions; (4) It submits 1–3 curated exhibits to a judge; (5) The judge returns structured feedback and rewards.
• Why it matters: Without self-verification, judging is expensive and error-prone; with it, judging becomes cheap, focused, and reliable.

🍞 Anchor: Like turning in a lab report with just the right photos of your experiment that make your results undeniable.

— Three Analogies —

1. Detective: The agent isn’t just doing the mission; it also gathers the most telling clues (exhibits) so the case is easy to close.
2. Sports Replay: Instead of rewatching the whole game, the agent hands the referee the 2–3 replay angles that decide the call.
3. Science Fair: Don’t bring your whole messy notebook—bring the final chart and the one photo that shows the reaction worked.

— Before vs. After —

• Before: Agents acted blindly about proof; judges slogged through long, noisy histories, costing time and money and risking mistakes.
• After: Agents plan for proof as they act, create missing evidence on purpose, and submit a crisp micro-dossier. Judges work less and judge better.

— Why It Works (Intuition) —

• Shrinks the context: A few strong exhibits beat a giant, noisy transcript. Less noise → fewer hallucinations.
• Grounds truth in actions: Each exhibit is a tool call plus its direct result, so proof is factual, not fluffy.
• Couples doing with proving: Requiring proof nudges better plans, cleaner steps, and accurate checks of results.
• Reward shaping: The judge gives specific signals (validity, completeness, formatting, conciseness), turning fuzzy success into learnable hints.

— Building Blocks —

🍞 Hook: Imagine you’re the student and also your own editor.

🥬 Self-Verifying Agent: An agent with two missions—Execute and Verify.

• How it works: (1) Finish the task; (2) Review interactions; (3) Pick (or create) the few exhibits that prove success; (4) Submit them.
• Why it matters: Without the verify mission, the agent leaves all the burden to the judge.

🍞 Anchor: A chef who cooks the dish and selects the exact photo that shows it’s perfectly baked.

🍞 Hook: You know how a great photo album has only the best shots that tell the whole story?

🥬 Evidence Curation (Exhibits): An exhibit is an atomic action→observation pair (e.g., Tap → New screen XML) pulled from the history.

• How it works: (1) Save tool call ID and its response; (2) Choose only the decisive ones; (3) Present them in a standard format.
• Why it matters: Without exhibits, proof can be vague or easy to fake.

🍞 Anchor: Like showing the exact receipt and the item photo to prove you bought the right thing.

🍞 Hook: Think of packing a carry-on: take everything necessary, nothing extra, and be clever if something is missing.

🥬 3C Principles: Completeness (include all pivotal proof), Conciseness (avoid redundant noise), Creativity (generate missing proof with extra actions).

• How it works: (1) Review steps; (2) Ensure all critical moments are covered; (3) Trim extras; (4) If a key view is missing, go capture it.
• Why it matters: Without 3C, you get either flimsy or bloated proof.

🍞 Anchor: If the task is “record income of 8000 CNY as salary,” a perfect evidence set shows the entry created with amount and category visible—no fluff, no gaps.

🍞 Hook: Picture a fair judge with a checklist.

🥬 LLM-as-a-Judge with Reward Shaping: The judge sees only the curated evidence, applies strict rules (no assumptions), and returns multi-part rewards (format, validity, success, brevity).

• How it works: (1) Check evidence relevance; (2) If success is claimed, demand unequivocal proof; (3) Penalize formatting errors or too many exhibits.
• Why it matters: Without structured feedback, learning to curate evidence is slow and fuzzy.

🍞 Anchor: Like a rubric that says “2 points for relevant images, 5 for clear proof, -1 for bad format,” making improvement obvious.

🍞 Hook: Imagine a class where your score depends on how your project compares to classmates, not a separate critic.

🥬 Group Relative Policy Optimization (GRPO): A learning method that skips a separate critic by comparing rewards across a group of trajectories from the same policy.

• How it works: (1) Sample several attempts; (2) Normalize each reward by the group’s mean and spread; (3) Update policy to favor above-average attempts.
• Why it matters: Without GRPO, training would be heavier and slower.

🍞 Anchor: Like grading on a curve—do better than the group, and you move up.

At a high level: Instruction → Agent executes and watches results → Curates exhibits using 3C (and creates missing proof if needed) → Submits 1–3 exhibits → LLM judge scores with structured rewards → GRPO updates the agent.

Step-by-step recipe:

1. Inputs

• Natural language task instruction (e.g., “Turn on dark theme”).
• Live environment with tool APIs (tap, type, swipe, back, home, get_current_xml, submit).
• Agent system prompt that encodes the 3C Principles and formatting rules.
• Verifier (LLM) prompt that enforces strict, traceable judging with no assumptions.

1. Augmented Actions and Trajectory

🍞 Hook: Imagine your toolbox adds not only tools to build but also a “camera” to prove what you built.

🥬 Augmented MDP: The action space mixes task actions (A_exec: tap, type, etc.) and curation actions (A_curate: submit with evidence IDs), so the policy plans doing and proving together.

• How it works: (1) Observe screen XML; (2) Choose an action; (3) Record action and next observation; (4) When ready, call submit(message, evidences=[IDs]).
• Why it matters: Without augmented actions, proof isn’t a first-class decision and arrives too late.

🍞 Anchor: Like a to-do list that ends with “attach the 2 photos that prove it’s done.”

1. Grounded Exhibits

🍞 Hook: Think of a diary where each entry is “I did this → here’s exactly what happened next.”

🥬 Exhibit Definition: Each exhibit is (action_t, observation_{t+1})—a tool call and its immediate result.

• How it works: (1) Keep IDs for every tool call; (2) Later select the decisive IDs; (3) Auto-format them into a standard chat template for the judge.
• Why it matters: Without this pairing, claims can be uncheckable.

🍞 Anchor: “Tap(Add) → New ‘Create Entry’ screen shows Amount=8000 and Category=Salary.” That pair is solid proof.

1. 3C Evidence Curation

🍞 Hook: Pack your backpack smartly: nothing missing, nothing extra, and add what you forgot before leaving.

🥬 3C in practice: Completeness (cover all must-have steps), Conciseness (smallest convincing set), Creativity (take extra actions to reveal proof if absent).

• How it works: (1) After task steps, quickly audit: Do I visibly show the final state and key fields?; (2) If not, navigate to the screen that proves it; (3) Keep total exhibits to 1–3.
• Why it matters: Without 3C, judges either can’t confirm or get overwhelmed.

🍞 Anchor: To prove “income 8000 CNY marked as salary,” include the entry list showing that item; if not visible, open the list view before submitting.

1. Verifier and Reward Shaping

🍞 Hook: Picture a fair checklist that rewards doing it right and keeping it tidy.

🥬 Evidence Validity Check: Judge marks evidence relevant or irrelevant; relevant (even proof of failure) gets a small positive reward.

• How it works: (1) If evidence relates to the task, +small; (2) If it’s off-topic, penalty.
• Why it matters: Without this, the agent might wander or submit junk.

🍞 Anchor: For “turn off Wi-Fi,” a screen showing Wi-Fi still on is valid (it proves failure), but a music app screen is invalid.

🍞 Hook: Think of a science judge who says, “No guessing—show me the measurement.”

🥬 Strict Grounding (Zero Assumptions, Traceable Reasoning): If the agent claims success, the judge demands explicit, unambiguous proof tied to specific exhibits.

• How it works: (1) Judge cites exact exhibit IDs; (2) No filling gaps from memory; (3) Outcome reward is 1 only if proof is airtight.
• Why it matters: Without strict grounding, hallucinations slip in.

🍞 Anchor: The judge must see the exact toggle switch Off, not just trust the agent’s summary.

🍞 Hook: Imagine getting docked points for messy formatting or talking too much.

🥬 Formatting and Conciseness: A format reward penalizes wrong output schema; a conciseness penalty scales with the number of exhibits.

• How it works: (1) Follow the exact submit(message, evidences=[...]) signature; (2) Keep exhibits ≤3.
• Why it matters: Without this, downstream tools break and judges get overloaded.

🍞 Anchor: If you submit five nearly identical screens, you lose points.

1. Policy Optimization with GRPO

🍞 Hook: Like a race where you learn by beating your average lap time.

🥬 Group Relative Policy Optimization (GRPO): Sample a group of attempts, compute each attempt’s relative advantage vs. the group, and update to favor above-average ones.

• How it works: (1) Roll out G trajectories for a task; (2) Compute rewards (format + validity + completion + conciseness); (3) Normalize by group stats; (4) Update policy to increase the probability of better trajectories.
• Why it matters: Without GRPO, you’d need a separate critic model, slowing training.

🍞 Anchor: If your attempt has tighter proof and cleaner format than your peers, your policy moves toward that style.

1. Training and Inference Details (Concrete)

• Environment: AndroidLab virtual devices; observation via compressed XML tree; action tools include tap/swipe/type and submit.
• Agent backbones: LLaMA3.1-8B (act-only), Qwen2.5-7B, Qwen3-8B, Qwen3-32B.
• Verifier: DeepSeek-R1 (text-only, strong reasoning) evaluates the curated exhibits; runs 3 times and uses 2/3 majority.
• Rewards: Example weights—validity small positive (e.g., +0.2/0.5), completion +0.8/1.0 if airtight success, format penalty -1 for schema errors, conciseness penalty proportional to exhibit count.
• Limits: Max turns per episode (e.g., 30); context length (e.g., 32k tokens); train on many GPUs for stability.

Secret Sauce (What’s clever?)

• Turning proof into an action: By adding curation to the action space, the agent “plays to the judge,” planning both solution and evidence.
• Exhibits as atomic facts: Action→result pairs are hard to fake and easy to check.
• Dense, structured rewards: The agent gets precise nudges to be correct, relevant, well-formatted, and brief.
• Creativity principle: When proof is missing, create it—this flips verification from passive to proactive.

🍞 Hook: Think of a school tournament where teams must both solve the challenge and submit the 2–3 photos that prove it.

🥬 The Test: Evaluate if SmartSnap-trained agents can finish Android tasks and self-prove success using minimal, decisive evidence.

• What they measured: Success Rate (SR), Sub-Goal Success Rate (Sub-SR), Reversed Redundancy Ratio (RRR), Reasonable Operation Ratio (ROR).
• Why it matters: We want not only more wins (SR) but smarter steps (ROR), fewer wasted moves (RRR high), and steady progress on sub-steps (Sub-SR).

🍞 Anchor: It’s like grading both the final answer and how cleanly you worked.

🍞 Hook: Imagine racing against bigger, fancier bikes and still keeping up because you pick smarter lines.

🥬 The Competition: Strong baselines like DeepSeek V3.1 and Qwen3-235B-A22B, and prompting/finetuning-only versions of the same backbones.

• Why compare this way: Shows if smaller or equal-sized models with SmartSnap can challenge larger models by being smarter about proof, not just bigger.

🍞 Anchor: David vs. Goliath—but with better highlights, not heavier armor.

🍞 Hook: Report cards make sense when you know the class average.

🥬 The Scoreboard (Context-rich):

• Qwen3-8B-Instruct: SR jumped to about 36.23% with RL (+26.08% vs. prompt-only baseline)—that’s like going from a shaky C to a solid B+ while others still hover at C.
• Qwen3-32B-Instruct: SR rose to about 34.78% (+16.66%)—like moving from mid-B to strong B+, rivaling much larger models.
• LLaMA3.1-8B (act-only): Despite no interleaved reasoning, SmartSnap RL lifted SR to ~31.15% (+26.08%), showing the self-verification habit improves planning even with limited tool modes.
• Overall: Gains >16% across families/scales. Evidence counts converged around ~1–3, turns decreased, and responses shortened—agents got cleaner and faster.

🍞 Anchor: The agents learned to hand the judge just the right snapshots and finish in fewer moves—quality over quantity.

🍞 Hook: Sometimes the quiz has a trick question where everyone stumbles.

🥬 Surprising Findings:

• Domain gaps: Maps.me and some Calendar/Zoom tasks stayed tough across models—likely due to missing world/app knowledge that RL alone can’t fill.
• Overfitting risk: Training rewards climbed with shrinking variance, hinting at overfitting the 726 training tasks; bigger, balanced datasets would help.
• Self-verifying SFT > vanilla SFT: Even without RL, SFT that includes the self-verification pattern outperformed plain imitation. Teaching “how to prove” boosts “how to plan.”

🍞 Anchor: Learning to show your work made students better at solving the problems, not just copying answers.

— New Concepts Introduced Here —

🍞 Hook: Think of a playground with many different games.

🥬 AndroidLab Benchmark: A set of 138 tasks across 9 Android apps on virtual devices, used to train and evaluate agents.

• How it works: Agents interact via tools; states are XML trees; success is judged by an LLM.
• Why it matters: Without a standard arena, we can’t fairly compare methods.

🍞 Anchor: It’s the tournament field where everyone plays the same matches.

🍞 Hook: Scores mean more when you know what they measure.

🥬 Success Rate (SR): Percent of tasks fully completed.

• How it works: Judge decides success per task; SR is successes divided by total tasks.
• Why it matters: Without SR, we can’t tell overall effectiveness.

🍞 Anchor: Like the fraction of homework problems you got right.

🍞 Hook: Sometimes finishing parts still shows progress.

🥬 Sub-Goal Success Rate (Sub-SR): How often important intermediate steps are achieved.

• How it works: Judge checks if milestones (like correct amount or date) are met.
• Why it matters: Without Sub-SR, we miss partial competence growth.

🍞 Anchor: It’s credit for showing correct steps, even if the final box wasn’t checked.

🍞 Hook: Fewer extra steps means you’re getting sharper.

🥬 Reversed Redundancy Ratio (RRR): Higher means fewer unnecessary actions.

• How it works: Compare useful vs. redundant moves; invert so bigger is better.
• Why it matters: Without RRR, agents might flail and still sometimes succeed.

🍞 Anchor: Like finishing a maze without wandering in circles.

🍞 Hook: Right moves in the right places.

🥬 Reasonable Operation Ratio (ROR): How often actions make sense given the screen.

• How it works: Judge scores whether actions fit the visible UI.
• Why it matters: Without ROR, success might come from lucky clicks, not skill.

🍞 Anchor: Like choosing the correct button because you understood the menu, not by guessing.

🍞 Hook: Even the best backpack can’t carry what you don’t own.

🥬 Limitations:

• Knowledge gaps: Some domains (e.g., mapping/navigation) need specific background that RL alone won’t teach; continual pretraining (CPT) and curated corpora are needed.
• Compute heavy: Running many sandboxed devices for RL needs strong engineering and lots of GPUs.
• Act-only constraints: Models without interleaved reasoning (e.g., certain tool modes) benefit less and may plateau sooner.
• Overfitting risk: With small, skewed task sets, agents improve on training but generalize less.

🍞 Anchor: If your study guide misses a topic, you’ll likely stumble on that section of the test.

🍞 Hook: A nice kitchen still needs electricity and groceries.

🥬 Required Resources:

• High-end GPU clusters for long-context RL.
• Stable Android emulators and fast tool APIs.
• A capable judge LLM and carefully crafted prompts.
• Balanced, diverse tasks to avoid overfitting.

🍞 Anchor: Think of needing both a good oven and quality ingredients to bake well.

🍞 Hook: Not every lock opens with the same key.

🥬 When NOT to Use:

• Tasks that can’t visibly prove success (no screen evidence) or need hidden/internal states you can’t show.
• Domains with severe knowledge gaps and no data to pretrain on yet.
• Ultra-tiny devices or budgets where long-context RL is infeasible.

🍞 Anchor: If you can’t take a photo of the result, a photo-based proof system won’t help.

🍞 Hook: Every good method opens new questions to explore.

🥬 Open Questions:

• Best reward mix: What weights and components drive the fastest, most robust learning?
• Adaptive curation: Can the agent predict the judge’s uncertainty and tailor exhibits on the fly?
• Cross-domain transfer: How well do evidence-curation skills move from mobile to web or desktop?
• Verifier design: Could multimodal judges or ensembles cut errors further without big cost?
• Data curriculum: What task sequences and variety best prevent overfitting while keeping training efficient?

🍞 Anchor: It’s like tuning a recipe—how much salt, which oven temp, and does it still taste great in a different kitchen?

🍞 Hook: Great work is convincing when the proof speaks for itself.

🥬 Three-Sentence Summary: SmartSnap turns agents into self-verifiers that both do tasks and proactively gather the 1–3 exhibits that prove success. By grounding evidence in action→result pairs and following the 3C Principles (Completeness, Conciseness, Creativity), the agent hands a lean, decisive dossier to a strict LLM judge. This makes verification cheaper and safer and boosts success rates across model sizes on Android tasks.

🍞 Anchor: It’s the difference between saying “I did it” and showing the two photos that make doubt impossible.

Main Achievement: Elevating proof to a first-class action—so the agent plans to show, not just to do—enabling scalable, reliable training without brittle, hand-crafted checkers or costly full-trajectory reviews.

Future Directions: Enrich domain knowledge with continual pretraining; expand to web and desktop arenas; refine reward shaping and exhibit selection; and explore stronger, possibly multimodal judges. Larger, more balanced task pools and better tool scaffolds should further stabilize and accelerate learning.

Why Remember This: SmartSnap reframes verification from a burden after the fact to a built-in habit of the agent. That shift—from passive judging to proactive proving—marks a practical path to trustworthy, efficient digital assistants we can confidently deploy.