Beyond text: bias and safety challenges in multimodal AI

Published: March 23, 2026

The Multimodal Moment - and Its Hidden Risks

Current AI systems like GPT-5, Gemini 3, Claude 4, and open-source alternatives process images, documents, charts, and video alongside text. However, a critical question remains understudied: are these systems fair?

Research indicates that "vision-language models encode and amplify demographic biases across modalities." A paradox emerges where improved overall accuracy can mask worsened fairness outcomes for specific demographic groups.

How Bias Enters Multimodal Systems

Multimodal systems have multiple entry points for bias throughout the pipeline:

Training Data: The Foundation of the Problem

Large-scale image-text datasets predominantly feature English-language content from Western sources. The LAION-5B dataset contains 2.3 billion English-language pairs but consolidates over 100 other languages into a secondary collection of similar size.

This imbalance means prompts like "a doctor" default to Western depictions -- typically white males in lab coats. Research from 2025 found AI-generated STEM professional portraits "almost exclusively depicted male, white, and older individuals."

Model Architecture: Where Bias Amplifies

Popular multimodal architectures use shared embedding spaces trained through contrastive learning (like CLIP). Research demonstrated that CLIP's text encoder maps neutral prompts significantly closer to masculine than feminine embeddings -- establishing gender bias before image generation begins.

A 2025 study showed that while adding modalities improved predictive accuracy, "fairness metrics disparities can persist or even increase depending on evaluation setting." Medical AI models systematically underdiagnose Black female patients despite expert-level aggregate performance.

Generation and Inference: Bias Made Visible

Bloomberg's analysis of over 5,000 AI-generated images revealed higher-paying positions consistently featured lighter-skinned individuals, while most professional roles were male-dominated. When generating "a terrorist," Stable Diffusion rendered men with dark facial hair wearing head coverings -- directly reflecting stereotypes.

A Nature Scientific Reports study documented "significant racial homogenization" where Middle Eastern men were depicted uniformly as bearded, brown-skinned, and wearing traditional attire.

The Research Gap: 16 Times Less Attention

Academic research on multimodal fairness lags significantly behind text-only systems study.

A 2024 survey comparing Google Scholar results found:

• Text-only LLMs: 538,000 results for fairness and bias
• Multimodal systems: 33,400 results -- roughly a 16-fold gap

Web of Science showed even starker disparity: 50 results versus 4.

This gap matters because multimodal systems increasingly deploy in high-stakes contexts -- healthcare diagnostics, hiring assessments, content moderation, and public safety -- where biased outputs carry direct consequences.

Real-World Impact: From Hiring to Healthcare

Employment Discrimination

An EEOC lawsuit revealed iTutorGroup's AI recruitment software automatically rejected female applicants aged 55+ and male applicants aged 60+. Over 200 qualified individuals faced disqualification based solely on age, resulting in a $365,000 settlement. May 2025 saw a federal judge allow a collective action lawsuit to proceed in a similar case.

Medical Misdiagnosis

Multimodal foundation models combining medical images with clinical text underdiagnose historically marginalized subgroups. A 2021 study demonstrated chest X-ray AI models systematically underperformed on Black female patients -- findings remaining relevant as architectures scale clinically.

Visual Media at Scale

AI-generated images exceed 34 million daily as of late 2023. Brookings Institution research showed text-to-image models consistently depict "successful people" as white, male, young, and dressed in Western business attire -- stereotypes flowing into advertising, stock photography, and corporate communications.

Disability Erasure

Stanford researchers found DALL-E, when asked to generate "a disabled person leading a meeting," produced an image of someone in a wheelchair watching someone else lead -- reflecting ableist assumptions about disabled authority. A 2025 University of Melbourne study found AI hiring tools struggled evaluating candidates with speech disabilities or non-native accents.

A Framework for Fairer Multimodal AI

Pre-Processing: Fix the Data First

Diversifying and auditing training datasets is fundamental. This involves curating culturally representative image-text pairs across languages, reviewing stereotypical annotations, and documenting demographic gaps through model cards and datasheets. Human annotators "bring their own biases and very stereotypical views" into labeling -- making annotator diversity equally important as data diversity.

In-Processing: Build Fairness Into Training

Fairness-aware loss functions can constrain outputs toward equalized odds or demographic parity. Adversarial debiasing trains secondary networks to strip protected-attribute signals from embeddings. Cross-modal alignment layers (like CLIP) need specific debiasing attention, since gender bias originates in text embeddings and propagates through generation.

The "Flare" framework (March 2026 arXiv) achieves fairness without explicit demographic attributes, using latent subgroup detection to improve equity across hidden clusters.

Post-Processing: Monitor and Audit in Production

Continuous fairness KPI monitoring (true positive rates, false positive rates, demographic parity ratios) across modalities is essential. Third-party red-teaming using diverse test sets surfaces biases internal testing misses. Output-level interventions -- such as inference-time prompt engineering introducing diversity -- serve as stopgaps pending fundamental fixes.

Supporting tools include Google's Explainable AI platform, IBM's AI Fairness 360, Microsoft's Fairlearn, and newer benchmarks like HEIM, ViSAGe, and the Social Stereotype Index.

The Regulatory Landscape

EU AI Act

The most comprehensive framework classifies systems by risk level, imposing mandatory conformity assessments, red-teaming requirements, and transparency obligations on high-risk systems. Enforcement escalates through 2027.

South Korea's AI Framework Act

Effective January 2026, this mandates fairness and non-discrimination across healthcare and public services, with administrative fines up to approximately $21,000.

Japan's AI Basic Act

Passed May 2025, this requires avoiding biased training data and conducting fairness audits, enforcing compliance through public naming rather than monetary penalties.

United States

No comprehensive federal law exists following the rollback of Biden-era Executive Orders. State and local measures -- such as New York City's Local Law 144 requiring bias audits for AI hiring tools -- create fragmented compliance landscapes.

Singapore

This jurisdiction pursues a voluntary, innovation-first approach through its Model AI Governance Framework, providing guidelines rather than binding requirements.

Organizations deploying multimodal AI across borders face divergent regulatory environments where fairness and bias mitigation become legal and business necessities.

What Comes Next

Research Priorities

Close the research gap: The 16-fold disparity between text-only and multimodal fairness studies must narrow through funder and journal prioritization.

Develop multimodal-specific benchmarks: Few benchmarks systematically test fairness across modalities, though ViSAGe and the Social Stereotype Index point directions.

Mandate cross-modal fairness audits: Organizations should conduct evaluations testing each modality independently and combined, since bias emerges from modal interactions invisible in isolated testing.

Invest in diverse data infrastructure: Industry-wide initiatives creating culturally diverse, multilingual, and demographically representative training datasets would have outsized impact.

Take feedback loops seriously: Biased AI outputs circulating online become future training data, creating self-reinforcing cycles. Solutions require filtering AI-generated content from pipelines and broader media literacy.

Conclusion

Multimodal AI represents genuine machine intelligence advancement, enabling systems to see, read, and reason previously impossible ways. However, this power raises fairness stakes commensurately higher. While biased text generates editable paragraphs, biased multimodal systems generate 34 million daily stereotypical images, reshape medical diagnoses, and filter job applicants across modalities -- compounding harms at scale.

Emerging tools, frameworks, and regulatory structures addressing multimodal AI fairness exist. The challenge remains matching deployment urgency with fairness rigor.