Position: Universal Aesthetic Alignment Narrows Artistic Expression

You know how every school cafeteria might try to serve food that ‘most kids’ will like, even if it means the menu becomes kind of same-y? That’s what has happened to many AI image makers—they’re trained to make pictures that fit a single, popular taste of beauty, even when you ask for something unusual.

🍞 Hook: Imagine you say, “Draw a rainy, blurry street with a hidden, tiny red bus that feels lonely and scary.” But the AI keeps making a bright, sharp, cheerful city photo with a big, shiny bus in the center. That’s frustrating!

🥬 The Concept (Universal Aesthetic Alignment): It means training AI to favor one mainstream idea of what looks ‘good’ or ‘beautiful,’ across almost every request.

• How it works: (1) Collect lots of human ratings about what images people prefer on average; (2) Train a ‘reward model’ to predict those preferences; (3) Use that reward model to nudge the image generator to make more ‘pretty’ pictures; (4) Repeat until the generator mostly produces that style.
• Why it matters: If the AI always chases ‘pretty,’ it may ignore your instructions when you want something messy, dark, blurry, weird, or emotionally heavy. 🍞 Anchor: You ask for ‘a badly lit room with a smudged face to feel unsettling,’ but get a bright, crisp portrait smiling into sunlight.

🍞 Hook: Think about different music styles—pop, punk, jazz, classical. None is ‘the one true music.’

🥬 The Concept (Aesthetic Pluralism): Beauty comes in many flavors, and people honestly disagree about what looks ‘good.’

• How it works: (1) Different people like different colors, moods, and shapes; (2) Art styles like Fauvism or Dada were once called ugly, then celebrated; (3) Diverse tastes make art richer.
• Why it matters: If AI only favors one look, it erases other valid styles—like satire, horror, abstraction, or gritty realism. 🍞 Anchor: The Scream by Edvard Munch doesn’t look ‘pretty’ in a classic sense, but it’s powerful and famous. Some reward models still give it low scores.

🍞 Hook: Picture a teacher who grades every story higher if it is cheerful, colorful, and neat, even if the assignment was to write something gloomy or chaotic.

🥬 The Concept (Reward Models): These are AI graders that score images for ‘quality’ or ‘prettiness’ and guide generators during training.

• How it works: (1) Humans compare images; (2) The reward model learns patterns of what wins; (3) The image generator is trained to get higher scores; (4) Over time, the generator learns to produce what the reward likes.
• Why it matters: If the reward model prefers bright, clean, happy images, the generator will drift toward that—ignoring requests for the opposite. 🍞 Anchor: When asked to pick between ‘blurry, unsettling bus in darkness’ and ‘bright, sharp bus center stage,’ many reward models pick the bright one—even if the prompt clearly asked for the dark, blurry version.

🍞 Hook: Imagine a super-talented art robot that draws from your words.

🥬 The Concept (Image Generation Models): These are AIs that turn your text prompts into pictures.

• How it works: (1) You type a prompt; (2) The model turns words into visual ideas; (3) It builds an image step by step; (4) Training pushes it toward images its reward model approves.
• Why it matters: If training overvalues ‘pretty,’ the model may stop listening when you ask for messy, strange, or sad. 🍞 Anchor: You request ‘noisy, low-detail table with smears and warped plates,’ but you get a perfect, appetizing food photo.

🍞 Hook: Think of a costume party where the host says, ‘Please don’t be cheerful—come scary!’

🥬 The Concept (Wide-Spectrum Aesthetics): This means intentionally choosing looks that break the usual beauty rules—like blur, odd colors, distortion, darkness, or negative emotions—because that’s the artistic point.

• How it works: (1) The prompt asks for non-mainstream traits; (2) The model should add those traits on purpose; (3) The result may be unsettling or imperfect by design; (4) It still must match the main subject.
• Why it matters: Without this, AI can’t help with satire, critique, horror, abstraction, or experimental art. 🍞 Anchor: ‘Zebras crossing a road, but low-quality, distorted, unfinished, with ugly background’ should look rough on purpose—yet many models ‘clean it up.’

🍞 Hook: Telling a chef, ‘Please don’t add sugar,’ is a clear instruction.

🥬 The Concept (Negative Prompting): It means asking the AI to avoid certain styles or add ‘anti-pretty’ traits on purpose.

• How it works: (1) You specify what not to include (e.g., ‘not sharp, not bright’); (2) The model adjusts away from those; (3) Ideally, it still keeps the main subject.
• Why it matters: If the system ignores these limits, your control disappears. 🍞 Anchor: ‘A building, but blurry, poorly lit, with noise and no detailed background’ should not become a magazine-style skyline.

🍞 Hook: When you design your room, you pick the posters, lights, and colors—not the furniture store.

🥬 The Concept (User-Centered Design): Tools should follow the user’s clear choices, especially when the user asks for a specific style.

• How it works: (1) Defaults can be mainstream; (2) But when users ask for something different, the tool should obey; (3) Controls should let users dial styles up or down.
• Why it matters: Otherwise, developers—not users—decide what is allowed. 🍞 Anchor: A photo editor that refuses to add blur because ‘most people like sharp’ would feel wrong.

🍞 Hook: Imagine a judge who gives extra points only to happy songs.

🥬 The Concept (Emotional Bias in AI): Some reward models score cheerful, bright feelings higher and punish sad, angry, or anxious tones—even when the prompt requests them.

• How it works: (1) The graders learned that people often pick happy images; (2) So they push the generator to make cheerful art; (3) Negative emotions get down-scored.
• Why it matters: This creates ‘toxic positivity,’ where real, complex feelings get erased. 🍞 Anchor: A prompt asking for ‘loneliness and anxiety’ gets transformed into a warm, optimistic picture with smiling colors.

Before this paper, many teams believed aligning models to a single ‘good-looking’ standard improved safety and user experience. The authors argue that, while safety matters, over-aligning aesthetics quietly erases user intent, compresses creative range, and sidelines valid art forms. They build a dataset of prompts that deliberately request anti-aesthetic traits and show that popular generators and reward models often fail—favoring pretty images even when the prompt clearly asked for something else. The takeaway is that instruction-following should outrank ‘universal prettiness,’ and users need controls to switch aesthetic alignment on or off.

🍞 Hook: You know how autocorrect sometimes changes your message into something nicer but totally different from what you meant? That’s what’s happening to art prompts.

🥬 The Aha! Moment: Training AI to please the ‘average’ taste makes it ignore clear instructions for messy, dark, blurry, or negative-emotion art—so the model looks good to graders but stops listening to you.

Multiple Analogies:

1. Restaurant analogy: If a restaurant always cooks dishes to be mild because ‘most people like mild,’ it ruins spicy orders—customers lose choice.
2. School poster analogy: If a teacher grades posters mostly on ‘bright and cheerful,’ students with powerful, somber designs get low scores even when the assignment asked for sadness.
3. Phone camera analogy: A beauty filter that you can’t turn off makes every face look glossy—fun sometimes, but wrong when you want raw truth.

Before vs After:

• Before: Models guided by reward graders mostly chose bright, sharp, cheerful, centered subjects. When you asked for blur, gloom, weird colors, or anxiety, the model ‘fixed’ it to be pretty.
• After (what the paper encourages): Keep safety filters, but stop forcing one aesthetic. Let users request anti-aesthetic looks and have models follow through. Add knobs to dial alignment up or down, and teach reward models to respect wide-spectrum choices.

Why It Works (intuition, no equations):

• Models learn what gets them higher scores. If the scorer prefers shiny, happy images, the generator learns to make shiny, happy images—even when you asked for the opposite.
• By testing prompts that clearly ask for ‘non-pretty’ traits, you can see the bias: aligned graders still pick the pretty image as ‘better,’ revealing the hidden rule: beauty first, instruction second.
• Proving that simple, unaligned models (CLIP/BLIP) understand these prompts suggests that the problem isn’t reading the prompt—it’s the aesthetic bias built into the reward.

Building Blocks (with simple sandwiches for key pieces):

🍞 Hook: Think of a library card that tells the librarian exactly what kind of story mood you want. 🥬 Wide-Spectrum Prompting: The authors took plain captions (like ‘zebras crossing a road’) and expanded them with intentional anti-aesthetic traits (e.g., ‘distorted, unfinished, ugly background’).

• How it works: (1) Start with a normal caption; (2) Add 2–4 ‘undesirable’ traits from a known list; (3) Use a smart language-vision model to craft a natural-sounding anti-aesthetic prompt.
• Why it matters: It creates a clear target that a fair system should honor. 🍞 Anchor: ‘Red double-decker bus’ becomes ‘barely visible, faded, oppressive darkness, anxious mood.’

🍞 Hook: You know how judges at a talent show can shape what performers try next time? 🥬 Reward Models as Graders: The paper compares many graders (like HPSv3, PickScore) to see if they pick the anti-aesthetic image as the better match when the prompt requests it.

• How it works: (1) Show two images—pretty vs. intentionally not-pretty; (2) Ask the grader which fits the prompt; (3) Check if it punishes the correct anti-aesthetic choice.
• Why it matters: If graders can’t reward the requested style, generators trained on them will avoid it. 🍞 Anchor: When asked to choose the ‘blurry, anxious bus’ for an anxious-bus prompt, a fair grader should pick it. Many don’t.

🍞 Hook: Imagine a referee who explains which rule was broken on each play. 🥬 Per-Dimension Judge: The authors fine-tune a small judge that scores specific traits (like lighting, detail) to see if generators actually followed the anti-aesthetic instructions.

• How it works: (1) No prompt needed; it looks only at the image; (2) It scores traits like ‘dark’ or ‘noisy’; (3) You can tell whether the model truly added those traits.
• Why it matters: You separate two questions: Did the model follow instructions? Did the grader still punish it anyway? 🍞 Anchor: The image is clearly dark and noisy (success), but the aesthetic grader still prefers the bright, clean version (bias).

🍞 Hook: Think of visiting a museum and discovering the guidebook gives low stars to famous abstract paintings. 🥬 Real Art Check: They score respected artworks with these reward models and often find surprisingly low ratings.

• How it works: (1) Feed real artworks to the graders; (2) Compare scores to AI-pretty images; (3) Look for big gaps.
• Why it matters: If the graders devalue canonical art, the system is overfitted to one style, not true artistic value. 🍞 Anchor: Some classic pieces receive scores below typical AI-pretty photos—evidence of a narrow lens.

In short, the core idea is simple: Teaching ‘beauty first’ makes the model stop listening when you ask for ‘not-beauty—and that’s bad for creativity and user control. The fix is to put instruction-following first and give users dials for aesthetics.

At a high level: Input (normal captions) → Step A (expand into wide-spectrum/anti-aesthetic prompts) → Step B (generate two images per prompt) → Step C (evaluate with graders and a per-trait judge) → Output (measure how often systems respect anti-aesthetic requests and how graders respond).

Step-by-step with plain-language reasoning and examples:

Step A: Build wide-spectrum prompts from normal captions.

• What happens: The team starts with simple image descriptions from a dataset (like ‘zebras crossing an empty road’). They pick 2–4 traits that mainstream graders call ‘bad’ (e.g., dark lighting, low detail, noisy texture, unrealistic proportions) and ask a strong vision-language model to write one clear anti-aesthetic prompt combining them.
• Why this step exists: You need a crystal-clear order like ‘please make it dark, blurry, and unfinished’ so any fair generator should know exactly what to do.
• Example data: Original: ‘Motorcyclers in a race leaning into a turn.’ Anti-aesthetic: ‘Blurred, fragmented, randomly composed, small and peripheral bikes amid chaotic, unfinished noise.’

Step B: Generate two images per prompt per model family.

• What happens: For each prompt, they create: (1) Io, the image from the original, plain caption; (2) Ia, the image from the anti-aesthetic prompt. They repeat this across several generator families: Flux (with variants like DanceFlux, PrefFlux, Flux Krea), Stable Diffusion XL and 3.5M (including aligned versions), and a strong closed model called Nano Banana.
• Why this step exists: You need a direct A/B test—comparing a standard image to the intentionally not-pretty version—to see if the model can really follow the non-mainstream instruction.
• Example: ‘A red double-decker bus’ vs. ‘a barely visible, faded red bus swallowed by darkness and anxiety.’

Step C: Evaluate results with two kinds of tools.

1. Prompt-aware reward models (graders):

• What happens: Popular graders (HPSv2/3, PickScore, ImageReward, MPS) and also unaligned matchers (CLIP, BLIP) are asked to pick which of Io vs. Ia better matches the anti-aesthetic prompt pa. The right answer is Ia if the generator listened.
• Why this step exists: To test whether graders can recognize and reward the requested ‘not-pretty’ traits when the prompt clearly asks for them.
• Example: For the ‘ugly, unfinished table’ prompt, the grader should choose the smudged, warped table (Ia), not the perfect food photo (Io).

1. Prompt-independent per-dimension judge:

• What happens: A small, fine-tuned judge scores traits in each image without seeing the prompt—things like lighting, color vibrancy, detail, realism. This reveals if the generator actually implemented the requested traits.
• Why this step exists: To separate ‘Did the model follow the instruction?’ from ‘Did the grader fairly reward it?’ If the model followed, but the grader still prefers the pretty version, that’s a bias in the reward model.
• Example: The image gets high ‘darkness’ and ‘noise’ scores (good instruction-following), but the aesthetic grader still picks the bright image.

Secret sauce (what’s clever here):

• The side-by-side design (Io vs. Ia) with a clear anti-aesthetic prompt makes it obvious when a grader or generator is favoring prettiness over instructions.
• Using both prompt-aware graders and a prompt-independent, per-trait judge lets you pinpoint where the failure happens—at generation (didn’t follow) or at grading (followed but got punished).
• Bringing in real, famous artworks as a ‘sanity check’ exposes whether these graders respect historically valued styles or only AI-pretty looks.

Metrics used and how they make sense:

• Preference-choice accuracy: Given Io and Ia with the anti-aesthetic prompt, does a grader choose Ia? If not, it’s favoring prettiness against instructions.
• F1 and ROC-AUC: Standard measures of how well a grader consistently picks the correct image when the prompt wants ‘not-pretty.’
• Delta scores (like ΔHPSv3): Compare beauty scores between normal vs. anti-aesthetic images made by the same model to see how much the generator ‘stayed pretty.’
• BLIP/CLIP matching: Check that Ia still contains the main subject (e.g., it’s still a bus) even if it’s ugly, dark, or blurry—so we know the image didn’t lose the point of the prompt.

Concrete walkthrough:

• Input prompt: ‘A table set with place settings of food and drink.’
• Anti-aesthetic version: ‘Extremely blurry and fragmented—noise distorts edges, details dissolve into smudges, objects broken and mismatched, warped proportions—impossible to discern items or harmony.’
• Generation: Make Io (normal) and Ia (anti-aesthetic) with several models.
• Check instruction-following: The per-trait judge confirms Ia is indeed blurrier, noisier, and lower detail.
• Ask graders: ‘Which image better matches this anti-aesthetic prompt?’ Many aligned graders still choose Io (the pretty one), revealing bias.

In sum, the method is a careful, fair test: make the instructions crystal clear, verify the generator followed them, then see whether the graders and aligned models still force ‘beauty first.’

The Test: Do AI systems honor anti-aesthetic instructions, and do their graders reward that honesty?

• The authors craft 300 wide-spectrum prompts from normal captions, each asking for traits like darkness, blur, noise, distortion, lack of background, negative emotions, or unrealism.
• For each, they generate Io (standard) and Ia (anti-aesthetic) images across multiple model families.
• They then ask a variety of reward models (graders) to choose which image better fits the anti-aesthetic prompt.
• A separate per-trait judge confirms whether the generator actually added the requested traits (so we know if the generator did its job).

The Competition (who/what was compared):

• Generators: Flux family (Dev, DanceFlux, PrefFlux, Flux Krea), SDXL and Playground (aligned for aesthetics), SD3.5M (with GenEval alignment or PickScore alignment), and Nano Banana (strong, closed-source).
• Reward models (graders): HPSv2/3, PickScore, ImageReward, MPS; baselines CLIP-L and BLIP-L as unaligned prompt matchers.

The Scoreboard (with context):

• Reward models often failed to pick the correct anti-aesthetic image when the prompt requested it. In plain terms, they graded the pretty image higher even when the instructions were to make it not pretty.
• Example context: HPSv3 performed worse than chance on this task (around 38% accuracy)—like getting an F on a true/false quiz where guessing gets 50%. Meanwhile, unaligned baselines like BLIP-L reached very high accuracy (around 96%), and CLIP-L also did well (about 91%). This shows the problem isn’t understanding the prompt; it’s a preference bias inside the aligned reward models.
• Generation models aligned for aesthetics underperformed on anti-aesthetic instruction-following compared to their base versions. For instance, DanceFlux struggled: in roughly two-thirds of cases, its anti-aesthetic image wasn’t better than its standard one according to the LLM-as-judge, indicating it tended to ‘beautify’ against instructions.
• Delta scores showed that aligned models kept higher ‘pretty’ ratings even when they were supposed to be messy or dark, confirming a pull toward conventional beauty.

Surprising Findings:

1. Unaligned prompt matchers (CLIP/BLIP) outperformed fancy aesthetic reward models at identifying the correct image for anti-aesthetic prompts. That means understanding the instruction isn’t the hard part—the bias is.
2. Some famous artworks received lower scores than average AI-pretty images. That’s like a music contest consistently scoring jazz classics below pop jingles just because the judge learned to prefer pop.
3. ‘Toxic positivity’ appeared: images with negative emotions (fear, sadness, anxiety) were systematically penalized, even if the prompt asked for those feelings.

Numbers turned into meaning:

• If a grader scores only 38% on ‘pick the right image for the anti-aesthetic prompt,’ it’s not just a small mistake—it’s picking the wrong answer more often than flipping a coin. In school terms, it’s not a B-minus; it’s a failing grade on the very skill we need when users ask for non-mainstream art.
• Conversely, BLIP at about 96% is like getting almost every question right—proof that the instruction is clear and the task is solvable without aesthetic bias.

Validation checks:

• A large set of artworks (about 10K) from a broad art dataset received significantly lower scores than AI-pretty outputs under some reward models. This isn’t random noise; it’s a repeatable pattern. If left as-is, it will discourage generators from making or preserving those styles.

Bottom line: The experiments reveal a systemic tilt. When you ask for anti-aesthetic art, aligned generators try to ‘fix’ it, and aligned graders punish it—even when the prompt makes it crystal clear that not-pretty is the goal.

Limitations (be specific):

• Sample size: 300 prompts give strong signals but aren’t the whole universe of art. More prompts and more dimensions would help.
• Scope of traits: The study focused on selected dimensions (e.g., lighting, detail, realism, emotion). There are countless other styles (e.g., collage textures, glitch art variants) that weren’t tested.
• LLM-as-judge: While the authors validated with human checks and achieved strong agreement, automated judging is still a proxy and may carry its own biases.
• Domain gap: Reward models were often trained on AI and photographic images, not curated museum art—yet consistent low scores for canonical works still signal a structural issue.
• Closed-source opacity: For models like Nano Banana, exact training details are hidden, making it harder to pinpoint causes when results are better or worse.

Required Resources:

• Access to multiple image generators (open and closed) and GPUs to run them.
• Reward models (HPS variants, PickScore, ImageReward, MPS) and unaligned matchers (CLIP/BLIP).
• A capable vision-language model to help craft anti-aesthetic prompts.
• Data for fine-tuning the per-trait judge (VisionReward dimensions).

When NOT to Use This Approach:

• If a system’s sole purpose is to deliver photorealistic, ad-quality images with zero tolerance for messiness (e.g., a product photo tool with strict brand rules), pushing wide-spectrum traits might be counterproductive.
• In workflows where a single house style is mandatory (e.g., a news outlet’s specific look), per-user anti-aesthetic freedom may need to be limited to maintain consistency.

Open Questions:

• How do we design multi-objective reward models that can switch modes: ‘mainstream-pretty’ when desired and ‘follow anti-aesthetic exactly’ when asked—without conflating the two?
• What is the best user interface for ‘aesthetic alignment knobs’? Sliders for brightness, sharpness, emotion valence, realism, and composition freedom?
• How can we collect broader, more diverse annotations across cultures and subcultures without reinforcing new biases?
• Can per-user adapters (like lightweight LoRAs) personalize taste safely, letting each person choose their aesthetic north star?
• How do we keep safety filters separate from aesthetic filters so that ‘unsettling’ doesn’t get mislabeled as ‘unsafe’?

Overall assessment: The paper doesn’t argue against safety; it argues against flattening all art into one happy, glossy look. It shows convincingly that today’s alignment often prioritizes a single aesthetic ideal over what users clearly ask for—and that this hurts creativity and autonomy.

Three-sentence summary: The paper shows that many AI image systems are over-aligned to a single, mainstream idea of beauty, so they ignore clear instructions for intentionally messy, dark, blurry, or emotionally negative art. Reward models often punish exactly the images that match those anti-aesthetic prompts, and aligned generators tend to ‘beautify’ anyway. The authors argue that instruction-following should come first and that users need controls to dial aesthetic alignment up or down.

Main achievement: A clean, testable framework demonstrating that aesthetic alignment narrows expression—using paired prompts and images, per-trait judging, and comparisons to both cutting-edge reward models and respected artworks.

Future directions: Build pluralistic reward models that can handle many aesthetics on purpose; separate safety from style; let users control alignment strength; expand datasets and annotators to represent diverse tastes; and explore lightweight adapters (like LoRAs) so people can personalize AI to their own style preferences.

Why remember this: Art is not one flavor, and tools that serve creators must respect that. If we let ‘beauty by average vote’ dominate, we lose satire, critique, horror, abstraction, and honest depictions of hard feelings. Putting user intent first—and giving them real dials—keeps AI a tool for expression, not a filter that sands off every edge.