Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning

🍞 Hook: You know how a smart poster can say “Coffee is a battery” without any words—maybe by drawing a coffee cup like a power pack? That little twist makes you smile because it feels true in a new way.

🥬 The Concept (Visual Metaphor): A visual metaphor is a picture that borrows features from one world (like machines) to explain another (like drinks) so a new meaning pops out. How it works: 1) pick a subject (coffee), 2) mix it with a carrier from another world (battery), 3) show a small but meaningful mismatch (a drink shaped like a battery), 4) let the viewer feel the new idea (coffee = energy). Why it matters: Without metaphors, pictures often say only what they literally show, losing punch and persuasion.

🍞 Anchor: A sneaker ad might show shoe soles gripping a cliff like rock‑climbing gear to mean “amazing traction.”

The World Before: Generative AI was great at painting pretty scenes and copying styles. It could make cats in Van Gogh style or change colors and textures. But most systems focused on pixels and appearances. They could not reliably capture the hidden message behind a clever ad or poster, like “This cream is as fresh as a rose” shown by blending jar and flower in a smart way.

🥪 Visual Metaphor Transfer (VMT) 🍞 Hook: Imagine remixing a magic trick: you learn the secret and then perform the same trick with different props. 🥬 The Concept: VMT means taking the underlying idea of one picture’s metaphor and applying that idea to a new subject. How it works: 1) understand the original picture’s logic, 2) keep its key relationship, 3) find a new carrier that fits a new subject, 4) rebuild the image so the same message appears in a new way. Why it matters: Without VMT, AI either copies the surface or guesses randomly, missing the smart logic. 🍞 Anchor: From “pillow = pill (natural sleep aid)” to “coffee = battery (energy boost).”

The Problem: AI didn’t know how to separate the creative essence (the why) from the visible stuff (the what). It needed a way to bottle up the idea so it could pour that same idea into a different bottle.

Failed Attempts: Text‑only systems asked users to write long prompts to force the metaphor, but most people don’t want to write essays to get one good image. Pixel editing swapped shapes or textures but often broke the meaning. MLLMs could describe pictures, yet they still struggled to decode non‑literal logic without lots of hints.

🥪 Conceptual Blending Theory (CBT) 🍞 Hook: Think of mixing two board games—like chess and checkers—to invent a new game that uses shared rules in a surprising way. 🥬 The Concept: CBT says creative meaning comes from blending two input spaces using a shared Generic Space (common relationships), making a blended space with new meaning. How it works: 1) choose two worlds, 2) find shared roles/relations (Generic Space), 3) project parts into a blend, 4) let new meaning emerge. Why it matters: Without CBT, we don’t know what stays the same when we change worlds. 🍞 Anchor: “Coffee = battery” shares the relation “provides energy,” even though one is a drink and the other a device.

The Gap: We needed a way to turn CBT’s idea into a step‑by‑step recipe that an AI can follow, not just a human theory.

🥪 Schema Grammar (G) 🍞 Hook: Imagine a LEGO instruction booklet—it shows how pieces connect, no matter their colors. 🥬 The Concept: Schema Grammar is a structured blueprint that separates the idea logic from the specific objects in an image. How it works: 1) list the subject and carrier with their attributes, 2) write down the Generic Space (the shared relation), 3) design the violation points (the clever mismatch), 4) capture the emergent meaning (the message). Why it matters: Without a schema, the AI can’t reliably move the idea to a new subject. 🍞 Anchor: For “pillow = pill” the schema notes: subject pillow, carrier medicine, shared relation “helps sleep,” violation “pillow shaped/behaving like a pill,” meaning “natural sleep aid.”

Real Stakes: Ads that click in one second, memes that spread fast, educational posters that simplify tough ideas—all need strong metaphors. This work helps AI build them on purpose, not by accident.

🍞 Hook: You know how a good recipe can be used with different ingredients and still taste right? Keep the cooking steps, swap the foods.

🥬 The Aha! Moment: If we preserve the same relationship (the Generic Space) and rebuild the picture around it, we can move a metaphor from one image to another subject on demand.

Multiple Analogies:

1. Map analogy: Keep the route (logic), change the vehicle (objects). You still get to the same destination (meaning).
2. Music analogy: Keep the melody (relation), play it with new instruments (subject/carrier). The tune is recognizable.
3. Sports analogy: Keep the rule (logic), change the players (visual parts). The game still makes sense.

Before vs After:

• Before: AI juggled styles and shapes but often lost the hidden message.
• After: AI extracts a Schema Grammar, protects the Generic Space, selects a new carrier, and rebuilds the scene so the same idea shines through in a new form.

🥪 Generic Space (the shared relation) 🍞 Hook: Imagine two stories with different characters but the same moral. 🥬 The Concept: The Generic Space is the common relation that exists in both worlds (e.g., “provides energy,” “smooths roughness,” “protects from harm”). How it works: 1) find the shared role, 2) keep it fixed, 3) project subjects/carriers around it, 4) let meaning emerge. Why it matters: Without this anchor, transfers drift and become random mash‑ups. 🍞 Anchor: Juice refuels a car vs. cream recharges dry skin—both “restore power.”

Building Blocks (the Schema Grammar pieces): 🥪 Subject (S) 🍞 Hook: Think of the hero of the story. 🥬 The Concept: The subject is the main thing you’re talking about (like coffee). How it works: identify core attributes, keep them recognizable. Why it matters: If the hero is unrecognizable, the message gets lost. 🍞 Anchor: Coffee stays clearly coffee: cup, color, context.

🥪 Carrier (C) 🍞 Hook: Picture the costume your hero wears to send a signal. 🥬 The Concept: The carrier is the borrowed world (like a battery) that frames the subject. How it works: choose a carrier whose role matches the Generic Space. Why it matters: The wrong carrier confuses the message. 🍞 Anchor: Energy message → battery is a natural carrier.

🥪 Violation Points (V) 🍞 Hook: Like a playful rule break in a game that makes everyone look twice. 🥬 The Concept: Violation points are the clever mismatches (shape/scale/context) that create the “aha!” moment. How it works: compare subject attributes to carrier norms and design a targeted clash. Why it matters: No violation, no spark—just a normal picture. 🍞 Anchor: Coffee cup shaped with battery cells/brackets.

🥪 Emergent Meaning (I) 🍞 Hook: Think of the moment a joke “lands.” 🥬 The Concept: Emergent meaning is the new idea you feel after seeing the blend. How it works: the brain resolves the mismatch and forms a takeaway. Why it matters: If meaning doesn’t land, the metaphor fails. 🍞 Anchor: “Coffee = power boost.”

🥪 Agentic Reasoning (multi‑agent teamwork) 🍞 Hook: Like a movie crew—writer, casting director, cinematographer, and critic. 🥬 The Concept: Different specialized agents handle understanding, transferring, drawing, and critiquing the metaphor. How it works: 1) Perception agent extracts schema, 2) Transfer agent keeps the Generic Space and finds carriers, 3) Generation agent writes prompts to render, 4) Diagnostic agent finds and fixes errors. Why it matters: If one person does everything, mistakes pile up; specialists keep quality high. 🍞 Anchor: The critic catches “battery geometry not clear” and sends it back to fix the prompt.

Why It Works (intuition): Humans read metaphors by spotting a shared relation and then enjoying a neat mismatch. The framework encodes that same habit: freeze the shared relation, design a tidy mismatch, and guide a generator to paint it. The critic loops back when something feels off, just like a human art director.

High‑Level Recipe: Input (reference image + target subject) → Perception Agent (extract schema) → Transfer Agent (preserve Generic Space, find new carrier) → Generation Agent (structured prompt) → Diagnostic Agent (multi‑level fixes) → Output image.

🥪 Perception Agent 🍞 Hook: Think of an art detective who explains how a poster’s trick works. 🥬 The Concept: The perception agent turns the reference image into a Schema Grammar (who’s the subject, what’s the carrier, what’s shared, where’s the violation, what’s the meaning). How it works: 1) identify subject/carrier and attributes, 2) infer the Generic Space (shared relation), 3) spot violation points, 4) write the emergent meaning. Why it matters: Without this step, we can’t copy the idea—only the pixels. 🍞 Anchor: From “pillow as pill”: subject pillow, carrier medicine, shared relation “sleep aid,” violation “pillow acting like a pill,” meaning “natural sleep help.”

Details: A vision‑language model is prompted to think step‑by‑step (S/C → attributes → Generic Space → violations → meaning). Example: For “rose cream,” it extracts: subject=cream, carrier=rose/flower, shared relation=“comes from fresh natural source,” violation=jar formed from petals, meaning=“born straight from the bloom.”

🥪 Transfer Agent 🍞 Hook: Like a great translator who keeps the joke funny in a new language. 🥬 The Concept: The transfer agent rebuilds the schema for a new subject but keeps the Generic Space unchanged. How it works: 1) profile the target subject’s attributes, 2) choose a new carrier from a different domain that shares the same relation, 3) design new violation points that are clear and drawable, 4) restate the meaning in the new context. Why it matters: If the relation drifts, the new image stops meaning the same thing. 🍞 Anchor: From “pillow as pill (sleep aid)” to “coffee as battery (energy).”

Details: The agent checks that carrier choice truly embodies the same relation. For “hair conditioner smoothing,” rope fibers may stand in for hair texture, with violations showing rope transitioning to silky strands—keeping the shared relation “makes rough → smooth.”

🥪 Generation Agent 🍞 Hook: Imagine turning a blueprint into a beautiful house. 🥬 The Concept: The generation agent converts the schema into a detailed prompt so a T2I model can paint the idea faithfully. How it works: 1) anchor layout with the carrier’s iconic features, 2) describe the violation clearly (shape/scale/context), 3) encode the mood so the meaning feels right (lighting, color, atmosphere). Why it matters: A vague prompt blurs the logic; the picture looks nice but says nothing new. 🍞 Anchor: “A coffee cup with visible battery cells, metallic connectors, studio lighting, bold contrast to suggest energy.”

Details: The LLM writes prompts that mention spatial relations, key geometry, and negative prompts (avoid literal batteries without coffee cues) so the metaphor stays centered.

🥪 Diagnostic Agent (with backtracking) 🍞 Hook: Think of a careful teacher who pinpoints exactly why a math answer went wrong. 🥬 The Concept: The diagnostic agent checks if the subject stands out, the violation is visible, the relation is clear, and the meaning lands—then sends targeted fixes. How it works: 1) prompt‑level fixes (clarify geometry, add constraints), 2) component‑level fixes (pick a better carrier, redesign the violation), 3) abstraction‑level fixes (rethink the Generic Space if it was too vague). Why it matters: Without pinpointed feedback, you keep redrawing the same mistake. 🍞 Anchor: If “battery” isn’t obvious, it adds “rectangular cells, voltage labels” to the prompt; if still unclear, it may switch to a “power bank” carrier.

Concrete Walk‑Through: Input a reference ad “glass lenses turn blurry → clear,” target subject “skin cream.”

• Perception: Extract shared relation “improves clarity/quality,” violation “glasses over a blurry view become sharp,” meaning “see better.”
• Transfer: Target relation becomes “improves skin texture,” choose carrier “sanding/polishing tool” or “refining filter,” design violation “cream jar with polishing wheel motif smoothing rough surface,” meaning “cream refines skin.”
• Generation: Compose a close‑up of a cream jar; one half of the background rough, the other half smooth, with subtle tool‑like highlights; warm, clean lighting.
• Diagnostic: If the polish idea feels too industrial, backtrack to a softer carrier (silk fabric smoothing) and adjust prompts accordingly.

Secret Sauce: Keeping the Generic Space invariant gives a stable spine for creativity, and the critic’s hierarchical backtracking fixes the right problem at the right level—logic, parts, or words—so the final image feels clever, not random.

The Test: The authors built a 126‑image set with real creative metaphors: product ads, memes, film posters, comics, and more. They asked the system to transfer each metaphor to new targets and judged how well the message carried over.

What They Measured and Why:

• Metaphor Consistency (MC): Does the new image keep the original idea?
• Analogy Appropriateness (AA): Is the chosen carrier truly a good fit for the new subject under the same relation?
• Conceptual Integration (CI): Do the parts blend naturally so it looks intentional, not pasted?
• Aesthetic Quality: Is it appealing to look at?

The Competition: Strong multimodal and image systems like BAGEL‑thinking, Midjourney‑imagine, GPT‑Image‑1.5, and Gemini‑banana‑pro.

The Scoreboard (with context): The proposed method topped all baselines across three independent VLM judges (Gemini‑3‑pro, GPT‑5.2, Claude‑Sonnet‑4.5) for MC, AA, and CI, while also achieving the highest aesthetic score (about 5.68). The biggest gap was in AA—roughly a 16.8% lift over the runner‑up—meaning the system was notably better at picking the right carriers for the new subjects. Think of it like getting an A+ in “smart matching” when others are getting B’s.

Human Study: 65 people rated images on five things: recognizability of the metaphor, ingenuity, violation appropriateness, visual integration, and overall quality. The method won across the board, especially in ingenuity and violation appropriateness, showing the images felt both clever and purposeful. In head‑to‑head preference tests, participants chose the proposed method over top commercial systems in more than 60% of cases.

Ablations (what parts mattered):

• Removing CBT and early reasoning led to simple object swaps and big drops in MC/AA/CI.
• Keeping agents but removing CBT hurt AA most—carriers became generic, not genuinely fitting the relation.
• Removing the diagnostic phase reduced quality and structure; the critic is key for cleaning up logic and layout.

Surprises: Even when different LLMs/T2I backbones were swapped in, the logical mapping stayed consistent, proving the approach is model‑agnostic. However, deeply cultural metaphors (like “Achilles’ heel”) sometimes required too much background knowledge, raising the viewer’s effort to “get it.”

Limitations:

• Cultural Load: Some metaphors depend on shared stories (e.g., myth or regional icons). Without that knowledge, viewers may miss the point.
• Cognitive Overload: Multi‑step chains (e.g., sirens → song → lure → danger → noise‑blocking) can be hard to decode instantly.
• Visual Feasibility: A carrier may perfectly fit the relation but be awkward to render clearly (iconic geometry gets lost, or attributes clash).
• Ambiguity Risk: If the violation is under‑ or over‑stated, the image looks either literal or chaotic.

Required Resources:

• A capable VLM/LLM for schema extraction and prompt writing, plus a strong T2I generator.
• Iteration budget for the diagnostic backtracking (the paper uses up to five loops for quality).

When NOT to Use:

• When instant, universal comprehension is critical (e.g., safety signs) and cultural load is high.
• When brand rules forbid the necessary visual violations (no distortion, no mixed categories).
• When compute or time budgets can’t support iterative refinement.

Open Questions:

• Automatic Culture Sensing: Can the system estimate cultural familiarity and pick lower‑effort carriers when needed?
• Visual Feasibility Prediction: Can it score how drawable a carrier/violation is before generating?
• Multi‑Metaphor Blends: How to combine two or more metaphors without confusion?
• Learning from Feedback at Scale: Can human preferences train better carrier selection and clearer violations over time?

Three‑Sentence Summary: This paper shows how to transfer the hidden idea inside a visual metaphor from one picture to a new subject by preserving the same core relationship. It formalizes that idea with a Schema Grammar and uses a four‑agent, closed‑loop system—understand, transfer, generate, and diagnose—to rebuild the metaphor faithfully. Experiments and user studies confirm the approach makes images that are more consistent, more appropriately matched, and better integrated than strong baselines.

Main Achievement: Turning Conceptual Blending Theory into a practical, schema‑driven, agentic pipeline that reliably carries a metaphor’s logic across domains.

Future Directions: Add culture‑aware carrier choices, predict feasibility of violations, blend multiple metaphors safely, and learn from large‑scale human feedback to refine selection and prompting. Exploring interactive tools could let designers steer carriers and violations in real time.

Why Remember This: It moves generative AI from copying looks to reusing ideas. By protecting the Generic Space and fixing mistakes with a smart critic, the system creates images that don’t just look cool—they say something new, clearly and creatively.