REGLUE Your Latents with Global and Local Semantics for Entangled Diffusion

🍞 Hook: Imagine building a LEGO city while wearing foggy glasses. You can click bricks together (so the city exists), but it’s hard to see the big picture (where roads go) and the small details (windows and doors). That’s how many image generators felt: they could place pixels, but truly “understanding” the scene took a long time.

🥬 The Concept (world before this paper): Latent diffusion models (LDMs) became the go-to for high-quality images by learning to denoise compact image latents instead of raw pixels. They were terrific builders but learned meaning (semantics) slowly because the training goal mostly said “remove noise” rather than “recognize object, layout, and parts.” Without faster, clearer semantic guidance, training took more steps and samples could still miss fine structure.

🍞 Anchor: Think of an AI asked to draw “a red bus near a tree.” It could learn that over time, but early in training it might draw red blobs or put the bus in the sky. It needs better hints about “bus-shaped” and “near the tree” early on.

🍞 Hook: You know how a wise friend can whisper tips while you’re doing homework? Vision Foundation Models (VFMs), like DINOv2, are that wise friend. They already learned tons of visual patterns from huge datasets.

🥬 The Concept (the problem): LDMs tried to borrow help from VFMs in two ways: (1) align features externally (like a teacher grading homework), and (2) inject a small slice of VFM features inside the diffusion model. But these attempts either used only a global summary (good for the big picture but not details) or used local features compressed linearly (they lost non-linear richness). As a result, models underused the VFM’s treasure of spatial, multi-layer meaning.

🍞 Anchor: It’s like getting only the book’s blurb (global) or a few flat notes (linear local) instead of the full, colorful chapters and the illustrations that show where everything is.

🍞 Hook: Picture two kinds of directions: a map of the whole city (global) and step-by-step street signs at each corner (local). To arrive on time, you really want both.

🥬 The Concept (failed attempts): Prior methods explored parts of this idea: REPA aligned hidden features to a VFM teacher (good supervision but indirect for generation), REG modeled only a global [CLS] token (helpful but non-spatial), and ReDi modeled patch-level features but squashed them linearly with PCA (lost non-linear detail). Each helped, yet none fully tapped the VFM’s rich, layered, spatial knowledge.

🍞 Anchor: If you only read the map title (global) or try to fit a 3D city into a flat 1D line (linear PCA), you miss where turns and intersections really are.

🍞 Hook: Imagine packing a suitcase. If you roll clothes (smart compression), you fit more without wrinkles; if you just squish them (linear squeeze), you lose shape.

🥬 The Concept (the gap): What was missing is a compact but non-linear, spatial way to carry rich, multi-layer VFM features into the diffusion process and fuse them with the VAE latents—plus the option to keep that gentle teacher alignment. That combination promises faster learning of “what’s where” and better images with about the same compute.

🍞 Anchor: With a proper packing method (non-linear compressor), you bring both the city map and the corner signs into the trip, so you arrive quickly and precisely.

— New Concepts with Sandwich Explanations —

🍞 Hook: Imagine taking a big picture and folding it neatly so it fits in your pocket, then unfolding it later without losing important parts. 🥬 The Concept: Variational Autoencoder (VAE)

• What it is: A VAE learns to squeeze an image into a small code (latent) and then rebuild it back.
• How it works: (1) Encode image to a compact latent; (2) Sample a nearby code; (3) Decode back to the image; (4) Train to make the reconstruction close to the original.
• Why it matters: Without a compact code, diffusion would be too slow and heavy on raw pixels. 🍞 Anchor: Like zipping a huge photo into a tiny file and unzipping it later with most details intact.

🍞 Hook: Think of learning to clean a smudged photo by removing noise step by step until it looks clear. 🥬 The Concept: Latent Diffusion Models (LDMs)

• What it is: Generators that denoise VAE latents instead of pixels.
• How it works: (1) Add noise to the latent; (2) Train a model to predict how to reverse that noise across time; (3) At sampling, start from noise and run the reverse steps; (4) Decode latents to an image.
• Why it matters: Working in latent space makes generation faster and memory-friendly. 🍞 Anchor: It’s like restoring a blurry thumbnail first and then printing the full picture.

🍞 Hook: When you look at a scene, you know both the big idea (“a beach”) and the tiny clues (shells near footprints). 🥬 The Concept: Global and Local Semantics

• What it is: Global = whole-image meaning; Local = patch-level details and where they are.
• How it works: (1) A VFM gives a global [CLS] token; (2) It also gives patch features at each spatial spot and layer; (3) Together they tell “what” and “where.”
• Why it matters: Without local semantics, you miss fine structure; without global, you miss the overall layout. 🍞 Anchor: A travel poster (global) plus street signs and door numbers (local).

🍞 Hook: Mixing two flavors gives a better smoothie than sipping each separately. 🥬 The Concept: Representation Entanglement

• What it is: Training the model to handle and fuse multiple representations at once (VAE latents + local + global semantics).
• How it works: (1) Noise all parts with the same schedule; (2) Project them to a shared width; (3) Fuse local semantics with latents token-wise; (4) Include a global token; (5) Predict velocities for all.
• Why it matters: Without joint modeling, semantics stay outside or thin, slowing learning. 🍞 Anchor: A music band sounds best when all instruments play together in sync.

🍞 Hook: If you fold clothes cleverly, you save space and keep outfits crisp. 🥬 The Concept: Semantic Compression

• What it is: A tiny CNN autoencoder that squeezes multi-layer VFM patch features into a low-channel, spatial map.
• How it works: (1) Concatenate features from several VFM layers; (2) Encode with a shallow CNN; (3) Decode to train with MSE; (4) Freeze the encoder and feed its compact map into diffusion.
• Why it matters: Without smart, non-linear compression, the model is overwhelmed or loses key structure (as with linear PCA). 🍞 Anchor: Rolling clothes (non-linear) beats flattening them with a heavy book (linear PCA).

🍞 Hook: A sturdy backpack lets you carry different tools together. 🥬 The Concept: Scalable Interpolant Transformer (SiT)

• What it is: A Transformer that learns a velocity field for a continuous noise schedule.
• How it works: (1) Patchify inputs; (2) Embed to a shared width; (3) Run through stacked Transformer blocks; (4) Predict per-modality velocities; (5) Use Euler–Maruyama for sampling.
• Why it matters: Provides a unified place to entangle multiple signals efficiently. 🍞 Anchor: Like one control panel that coordinates the whole orchestra.

🍞 Hook: A coach compares your practice swings to a pro’s swing to correct your form. 🥬 The Concept: Alignment Loss

• What it is: A gentle extra loss that nudges hidden features toward frozen VFM targets.
• How it works: (1) Pick a mid Transformer block; (2) Project hidden tokens; (3) Compare to VFM tokens with cosine similarity; (4) Add as an auxiliary term.
• Why it matters: Without it, some semantic hints may drift; with only global alignment, it can become unstable. 🍞 Anchor: A teacher’s hint helps most when it’s tied to local examples, not just big-picture advice.

🍞 Hook: A tidy pencil case holds many tools neatly. 🥬 The Concept: Compact Structured Representation

• What it is: Small, spatially organized semantic maps that keep the good stuff and fit nicely with latents.
• How it works: (1) Non-linear compression; (2) Keep spatial grids; (3) Balance channels (e.g., 16 vs. 4 latent channels); (4) Align sizes to add token-wise.
• Why it matters: If too big, it hogs capacity; if too small, it drops meaning. 🍞 Anchor: A well-organized tool belt—everything you need, nothing extra.

🍞 Hook: A medal score tells you how close a new drawing looks to real photos. 🥬 The Concept: Fréchet Inception Distance (FID)

• What it is: A number that says how close generated images are to real ones (lower is better).
• How it works: (1) Extract features with an Inception network; (2) Compare real vs. fake distributions; (3) Compute a distance.
• Why it matters: It’s a standard scoreboard for image generators. 🍞 Anchor: Scoring 12.9 vs. 33.0 is like jumping from a C to an A in class.

Why care (real stakes): Faster training means less compute, cheaper experiments, and greener AI. Better spatial accuracy improves details like textures and object parts, which matters for design, art tools, and scientific visuals. And the method keeps parameters and sampling cost near-constant—so it’s an easy upgrade path for existing diffusion Transformers.

🍞 Hook: You know how cooking goes faster and tastes better when the recipe, the spices, and the plating plan are all decided together? Not one after the other, but as a single plan.

🥬 The Concept (Aha! in one sentence): REGLUE entangles VAE latents with compact local patch semantics and a global [CLS] token inside one SiT backbone—plus a light alignment nudge—so high-level meaning appears early, guiding generation strongly and fast.

How it works (big picture):

1. Train a tiny CNN “semantic compressor” to squeeze multi-layer VFM patch features into a small spatial map that still keeps important non-linear cues.
2. Jointly inject noise into VAE latents, compressed local semantics, and the global token with the same schedule.
3. Project them to a shared width; add local semantics to latents token-wise; keep the global token separate.
4. Use a SiT Transformer to predict velocities for all parts at once; optionally apply mid-block alignment to a frozen VFM.
5. Sample by reversing the noise; decode latents with the VAE decoder.

Why it matters: Without joint modeling of strong, non-linear local semantics, you learn the “what and where” too slowly. With it, the model locks onto structure quickly, improving FID and convergence—without inflating parameters or inference cost.

Three analogies (same idea, different lenses):

• Orchestra analogy: Instead of practicing violin, drums, and piano separately, the band rehearses together with a conductor (SiT). The score (global token) and sheet notes for each section (local patches) keep everyone in sync.
• City-building analogy: The blueprint (global) and the street-by-street instructions (local) are carried in a compact binder (compressor) and used while building (diffusion), not after.
• Puzzle analogy: You don’t sort edge pieces (global) and middle pieces (local) in different rooms. You keep them on one table, neatly packed, and assemble faster.

Before vs. After:

• Before: External alignment alone helped but didn’t feed rich local structure directly into generation. Global-only joint modeling guided the big picture but missed fine detail. Linear PCA on patches threw away non-linear nuance.
• After: Non-linear compressed local patches are fused with latents, the global token provides scene context, and alignment adds a gentle regularizer. Result: sharper details, better layouts, faster learning.

Why it works (intuition, no equations):

• Shared noise schedule ties all signals to the same “where we are in time,” so they co-evolve.
• Token-wise fusion (adding local semantics to latent tokens) keeps sequence length short—attention stays efficient—while injecting location-specific guidance right where generation happens.
• Non-linear compression preserves curved, complex relationships across layers—something linear PCA can’t capture—so semantic richness survives the squeeze.
• A small alignment loss keeps internal features near a trusted teacher (VFM), especially for local patches; global-only alignment is unstable without spatial anchors.

Building blocks as sandwiches:

🍞 Hook: Imagine carrying a city map and corner signs in the same backpack. 🥬 Representation Entanglement

• What: Learn VAE latents, local patches, and global token together in one model.
• How: Same noise schedule; shared embedding width; token-wise fusion for patches; separate global token; predict velocities for all.
• Why: Separate training loses synergy; together, meaning appears early and strongly. 🍞 Anchor: A team wins when offense, defense, and coaching plan together.

🍞 Hook: Rolling clothes, not squashing, saves space and keeps outfits nice. 🥬 Semantic Compression

• What: A tiny CNN turns stacked VFM layers into a small, spatial semantic map.
• How: Concatenate deep VFM layers; encode→decode with MSE; freeze the encoder.
• Why: Linear PCA drops non-linear cues; naive fusion overwhelms capacity. 🍞 Anchor: A smart suitcase packs more, better.

🍞 Hook: One control room, many instruments. 🥬 SiT Backbone

• What: A Transformer trained with a velocity objective on a continuous interpolant.
• How: Patchify→embed→Transformer blocks→per-modality velocity heads.
• Why: Unifies modalities efficiently; sampling is standard and stable. 🍞 Anchor: A conductor coordinating the orchestra.

🍞 Hook: A tutor’s hint helps if tied to the page you’re reading. 🥬 Alignment Loss (REPA-style)

• What: Cosine-similar matching of mid-block tokens to clean VFM targets.
• How: Project hidden tokens; compare to VFM [CLS] + patch tokens; add small loss.
• Why: Stabilizes and enriches semantics, especially locally. 🍞 Anchor: Small nudges at the right spots keep you on track.

🍞 Hook: A neat tool belt keeps essentials reachable. 🥬 Compact Structured Representation

• What: Low-channel, spatial, multi-layer-informed semantic maps.
• How: Non-linear CNN encoder; preserve grid; balance channels with latents.
• Why: Too big hogs compute; too small loses meaning. 🍞 Anchor: Exactly the right tools, exactly where you need them.

Net effect: REGLUE “glues” semantics to latents early, speeding training and lifting quality with almost no runtime penalty.

At a high level: Image → VAE encoder + VFM encoder → (Compress VFM patches) → Add shared noise to latents, local semantics, and global token → Project to shared width and fuse → SiT Transformer predicts velocities for all → Optional alignment to VFM targets → Reverse-time sampling → VAE decoder → Image.

Step-by-step (like a recipe):

1. Prepare the ingredients (encoders)

• What happens: A frozen VAE encoder turns each 256×256 image into a small latent grid (e.g., 4×32×32). A frozen Vision Foundation Model (e.g., DINOv2-B) outputs: (a) a global [CLS] token (size 768), and (b) patch-level feature maps from several layers (e.g., the last four, each 16×16×768).
• Why this exists: We need a compact place to generate (VAE latent) and strong semantic hints (global + local) to guide structure.
• Example: From one cat image: z* ∈ R^{4×32×32}, 4 VFM layers each 16×16×768, and one 768-d [CLS] vector.

1. Compact the local semantics (semantic compressor)

• What happens: Concatenate the 4 VFM layers along channels (16×16×3072). Pass through a tiny CNN autoencoder trained with MSE to reconstruct the original features; then freeze the encoder. Keep only the 16-channel compressed map (e.g., 16×16×16) and upsample it to match the VAE latent grid (e.g., 32×32).
• Why this exists: Naively fusing 3072 channels with 4 latent channels overwhelms the model and biases capacity; linear PCA loses non-linear structure. The CNN preserves rich, curved relationships in a tiny footprint.
• Example: 16×16×3072 → compressor → 16×16×16 → bilinear upsample → 32×32×16.

1. Entangle three modalities with the same noise schedule

• What happens: For a random time t in [0,1], add noise to: (a) VAE latents z*, (b) compressed local map s*, and (c) global [CLS]. Use the same α_t and σ_t schedule, with independent noise for each.
• Why this exists: A shared timeline ties their evolution together, so the model learns how they relate at each denoising step.
• Example: z_t = α_t z* + σ_t ε_z, and similarly for s_t and cls_t.

1. Tokenize and bring to a shared width

• What happens: Patchify z_t and s_t with the same patch size (e.g., 2×2) to get N tokens each; linearly project z′_t and s′_t to a common width D (e.g., 768). Project cls_t to D as well.
• Why this exists: Transformers expect sequences of same-width tokens; matching widths lets us mix signals cleanly.
• Example: N = (32×32)/(2×2) = 256 tokens for z and 256 for s, each to width 768; one global token also to 768.

1. Fuse local semantics with latents efficiently

• What happens: Instead of concatenating sequences (which doubles length and quadratic attention cost), add the local semantic tokens to the latent tokens element-wise (channel-wise). Keep the global token separate at the front.
• Why this exists: Token-wise addition keeps sequence length N+1, saves compute, and injects location-specific guidance directly into where we generate.
• Example: Input to SiT: [global_token; (z_tokens + s_tokens)], shape (1+N)×D.

1. Predict velocities for all modalities

• What happens: Run K Transformer blocks (SiT). From the last hidden states, apply three small heads to predict the velocities for the VAE latents, local semantics, and global token. Unpatch latents and local predictions back to their grids.
• Why this exists: The velocity objective trains the model to know how to move each modality toward cleaner states. Predicting all three keeps them jointly coherent.
• Example: Heads map hidden tokens back to shapes: v_z (4×32×32), v_s (16×32×32), v_cls (768).

1. Optional external alignment (REPA-style)

• What happens: At an intermediate block (e.g., k=4 for base, k=8 for XL), project hidden tokens and match them (by cosine similarity) to clean, frozen VFM targets (global + flattened local tokens). Weight this auxiliary loss lightly.
• Why this exists: A small teacher nudge stabilizes and enhances semantics, especially local. But global-only alignment can be unstable without local anchors.
• Example: Add λ_rep × L_REPA to the total loss.

1. Training objective and sampling

• What happens: Optimize the sum of multimodal velocity losses (for z, s, cls) plus the optional alignment term. For generation, start from Gaussian noise for all modalities and integrate backward (Euler–Maruyama) using predicted velocities. Decode the final VAE latent with the frozen VAE decoder to get the image.
• Why this exists: Joint loss ensures all parts improve together; standard sampling keeps the pipeline simple.
• Example: 250 sampling steps, as in SiT.

What breaks without each step:

• No compressor: The semantic channels swamp the model; quality drops.
• Linear PCA instead of non-linear CNN: Non-linear richness is lost; FID much worse (e.g., 21.4 vs. 14.3 without alignment).
• Concatenate instead of add: Sequence doubles; attention cost balloons; training slows.
• No shared schedule: Modalities drift apart in time; fusion gets noisy.
• Global-only or alignment-only: You miss crucial spatial anchors or create instability.

Concrete mini-walkthrough with shapes:

• Start: z* (4×32×32), VFM layers (4× 16×16×768), cls* (768)
• Compress: concat → 16×16×3072 → CNN → 16×16×16 → upsample → 32×32×16
• Noise: z_t, s_t, cls_t per shared (α_t, σ_t)
• Tokens: patchify (2×2) → N=256; embed to D=768; fuse (z+s); prepend global
• SiT: (1+256)×768 → K blocks → predict v_z, v_s, v_cls → unpatch
• Loss: sum velocity losses + small alignment on mid-block tokens
• Sample: reverse SDE 250 steps → decode z_final to image

The secret sauce:

• Non-linear, multi-layer spatial compression retains rich semantics in only ~16 channels.
• Token-wise addition injects local meaning where generation occurs without doubling sequence length.
• Shared denoising timeline and gentle alignment keep all signals coherent and stable.
• Almost no parameter or inference overhead vs. the base SiT, yet large quality and speed gains.

The test (what they measured and why):

• Dataset: ImageNet 256×256, a standard, challenging benchmark.
• Metrics: FID (lower is better) for realism, sFID for spatial coherence, Inception Score for diversity, Precision for fidelity, Recall for coverage.
• Goal: Show better images, faster learning, and robustness with minimal extra cost.

The competition (baselines compared):

• SiT-B/2 and SiT-XL/2 backbones without extra semantics.
• REPA (external alignment only), ReDi (joint modeling with linear PCA-compressed patches), REG (joint modeling with global [CLS] + alignment).

The scoreboard (with context):

• Base scale, no CFG, 400K steps:

  • SiT-B/2: 33.0 FID (baseline, like a C grade).
  • REPA: 24.4 FID (improves, like going to a B-).
  • ReDi: 21.4 FID (better, a solid B).
  • REG: 15.2 FID (a strong A-).
  • REGLUE: 12.9 FID (A), and it already beats REG at 300K steps with 14.5 FID (25% fewer steps).

• Large scale (SiT-XL/2), no CFG:

  • 200K steps: REGLUE 4.6 vs. REG 5.0, REPA 11.1, ReDi 12.5. REGLUE leads early.
  • 700K steps: REGLUE 2.7 matches REG’s 1M (2.7) with 30% fewer iterations.
  • 1M steps: REGLUE sets best at 2.5 vs. REG 2.7, ReDi 5.1, REPA 6.4.

• With CFG (standard evaluation):

  • At 80 epochs, REGLUE FID 1.61 vs. REG 1.86 (better with same training time).
  • At 160 epochs, REGLUE 1.53 vs. REG 1.59; competitive with long-trained models (REPA 1.42, REG 1.36 at 800 epochs), despite training 5× fewer epochs.

• Unconditional setting (harder):

  • SiT-B/2: 59.8 FID; ReDi: 43.6; REG: 29.7; REGLUE: 28.7 (best). Even beats the conditional SiT-B/2 baseline (33.0).

Surprising findings:

• Local (patch-level) semantics are the star: modeling only global [CLS] helps but is clearly weaker than local patches.
• Non-linear compression is the unlock: replacing PCA with the tiny CNN slashes FID (e.g., 21.4 → 14.3 without alignment) and preserves much more semantic information (as shown by attentive probing and segmentation mIoU).
• Alignment needs spatial anchors: global-only alignment can hurt (25.7 → 33.7 FID) because it lacks local grounding; adding local alignment stabilizes and improves.
• Multi-layer aggregation (last 4 VFM layers) helps over only the final layer; early shallow layers don’t help.

Concrete numbers to hold onto:

• SiT-B/2: 33.0 → REGLUE 12.9 FID at 400K, a 60.9% drop.
• REGLUE hits 14.5 FID at 300K vs. REG 15.2 at 400K (25% fewer steps).
• SiT-XL/2 + REGLUE: 2.5 FID at 1M; matches REG’s 2.7 at 700K (save 300K steps).
• Stronger VFM (DINOv3-B) improves REGLUE from 12.9 to 12.3 FID at base scale.
• Data-limited training: REGLUE improves over REG by −5.5 FID at 20% data and −3.4 at 50% data.

Why these results matter (plain meaning):

• Faster convergence means fewer GPU hours and a smaller carbon footprint.
• Lower FID and strong sFID mean images look more real and spatially consistent (objects and parts in the right places).
• Gains persist across conditional and unconditional setups and with different backbone sizes.
• The method adds almost no overhead, making it practical to adopt.

Extra semantic evidence:

• Attentive probing: REGLUE’s 8–16 channel compressors keep much more VFM meaning than PCA, and better preserved semantics line up with better FID.
• Segmentation mIoU: Non-linear compressed features approach the full 768-channel baseline with far fewer channels and yield better generation quality than PCA-compressed ones.

Overall: REGLUE shows that the main lever is rich, spatial, non-linear local semantics jointly modeled with latents. Global context and alignment are nice boosts, but the big gains come from the local signal done right.

Limitations:

• Training budget: Large-scale runs were capped at ~1M steps due to compute limits (8×A100). Ultra-long schedules (e.g., 4M) were not explored, so ultimate convergence ceilings remain unknown.
• Resolution scope: Main results are at 256×256. Higher resolutions (e.g., 512×512) weren’t covered here and may need tuning of the compressor and fusion.
• Dependence on VFM quality: Better VFMs (e.g., DINOv3) help; weaker ones (e.g., CLIP-L in this setup) lag. If the teacher’s semantics are off-domain, benefits may shrink.
• Extra pretraining step: The semantic compressor requires a short offline MSE training pass. It’s lightweight, but still an extra step.
• Hyperparameter sensitivity: Choice of which VFM layers to aggregate, compressor width, and the alignment placement/weight matter for best results.

Required resources:

• A diffusion Transformer backbone (SiT-B/2 or XL/2) and a frozen VAE.
• A VFM (e.g., DINOv2-B) to extract global and patch-level features.
• Short compressor pretraining (25 epochs) and standard diffusion training (e.g., 300K–1M steps) with batch size ~256.

When NOT to use:

• Extremely tiny models or latency-critical mobile deployments where even minimal extra features are infeasible.
• Domains where VFMs give poor spatial features (e.g., unusual sensor modalities) unless a domain-tuned VFM is available.
• If you cannot afford even small preprocessing (compressor training/extracting VFM features), a plain LDM may be simpler.

Open questions:

• Can we learn a compact global compressor too, to better balance global/local capacity?
• End-to-end training: If we unfreeze the VAE or parts of the VFM/compressor, do we get further gains or instability?
• Beyond images: How does REGLUE extend to video (appearance + motion), 3D, or multimodal generation (text, audio)?
• Robustness and fairness: How do different VFMs affect bias and generalization in the generated samples?
• Sampling efficiency: With strong semantics, can we safely reduce steps without quality loss?

Bottom line: REGLUE is simple to adopt, brings big gains with minimal overhead, but still invites research on scaling, domains, and fully end-to-end training strategies.

Three-sentence summary: REGLUE jointly models VAE latents, compact local patch semantics, and a global [CLS] token inside a single SiT backbone, while a small alignment loss gently guides hidden features toward a strong VFM teacher. A tiny, non-linear semantic compressor preserves rich, multi-layer spatial meaning in very few channels, unlocking faster learning and much better image quality with nearly unchanged parameters and sampling cost. On ImageNet 256×256, REGLUE consistently beats prior baselines and reaches their scores in far fewer steps, at both base and XL scales, with or without classifier-free guidance.

Main achievement: Proving that non-linear, spatial, multi-layer local semantics are the primary lever for accelerating semantics emergence and improving fidelity—more than global-only modeling or alignment alone—and packaging this into a unified, efficient framework.

Future directions: Scale to higher resolutions and longer schedules; compress the global token; explore stronger or domain-specific VFMs; try end-to-end variants; extend to video and multimodal settings; and study step-reduction strategies leveraging the stronger semantic prior.

Why remember this: REGLUE shows that “what and where” should be learned together, early and efficiently. By gluing local and global semantics directly into the diffusion process, it turns slow semantic emergence into fast, precise guidance—delivering sharper, more realistic images sooner, at practically the same cost.