Bias detection in text: from traditional ML to RAIL API

Comparing TF-IDF, transformer embeddings, and ethical auditing frameworks for detecting bias in machine-generated text

Published: July 21, 2025

Introduction

As machine learning advances across industries, the need for models that are "accurate and performant" alongside being "fair, inclusive, and explainable" has become critical. From hiring systems to content feeds, these technologies increasingly shape daily decisions. Yet with this influence comes responsibility -- particularly when generated text may reflect or amplify harmful societal biases.

This article explores the evolution of bias detection through three approaches:

• Traditional machine learning using TF-IDF vectorization and XGBoost classifiers
• Transformer-based embeddings, specifically All-MiniLM-L6-v2
• Ethical evaluation frameworks like the RAIL API for fairness auditing

These components form a modular, explainable, and ethically aware pipeline for real-world NLP systems.

The Bias Problem: The Hidden Flaw in Machine Learning Models and AI

As AI systems become embedded in critical domains -- recruitment, education, journalism -- an invisible threat persists: bias. Rather than eliminating human prejudice, many models unintentionally mirror and magnify biases present in training data. Even advanced systems like ChatGPT or Gemini remain vulnerable. A small bias in phrasing can have outsized impact at scale.

Bias detection is "not a luxury -- it's a necessity for building trustworthy and responsible AI."

Goal of This Analysis

The primary objective investigates inherent biases in machine-generated text from ML models and conversational agents. As these technologies accelerate across sensitive domains, robust bias detection becomes critical.

The analysis compares traditional machine learning techniques (TF-IDF vectorization with XGBoost) against transformer-based embeddings (All-MiniLM) to evaluate effectiveness in identifying gender, political, and demographic bias. It also demonstrates how external fairness auditing tools like RAIL API provide an ethical validation layer.

Focus Areas: Types of Bias in Textual Content

Political Bias

Definition: Language promoting, favoring, or disparaging specific political ideologies, parties, or viewpoints, subtly influencing public perception.

Examples:

• "Conservatives are naturally more intolerant."
• "Leftist policies always ruin the economy."

Demographic Bias

Definition: Assumptions or stereotypes based on race, religion, location, age, or socioeconomic class that reinforce harmful social divides.

Examples:

• "People from rural areas are less educated."
• "Muslims are more likely to be violent."

Gender Bias

Definition: Stereotypes, inequalities, or unjust treatment based on gender, perpetuating outdated views about roles and capabilities.

Examples:

• "Women are too emotional for leadership roles."
• "He codes better because he's a man."

Undetected biases degrade AI quality, fairness, and trustworthiness, making bias detection foundational for ethical AI development.

Dataset Used for This Comparison

Data was gathered from Gemini, ChatGPT, and Kaggle datasets. After removing duplicates and null values, the final dataset contained 2,138 rows with 4 columns.

The distribution showed representation across all bias categories for comprehensive analysis.

Model Selection

XGBoost (Extreme Gradient Boosting) was selected as the core classifier for its ability to efficiently handle high-dimensional and sparse feature spaces from TF-IDF vectors and transformer embeddings.

Key Advantages of XGBoost:

• High Accuracy: Consistently achieves better classification metrics across bias categories
• Speed & Scalability: Efficient for small and large-scale datasets with parallelized tree construction
• Gradient Boosting Framework: Combines multiple weak learners sequentially to minimize error
• Robustness: Performs well with sparse or high-dimensional feature spaces

Feature Extraction with TF-IDF

Before feeding text into ML models, it must be converted to numerical format. TF-IDF (Term Frequency-Inverse Document Frequency) is a widely used technique for this transformation.

What is TF-IDF?

TF-IDF is a statistical NLP method representing text as numerical vectors. It evaluates word relevance to a document in a collection, balancing frequency within the document against frequency across all documents in the corpus.

How It Works

TF-IDF assigns scores based on two factors:

• Term Frequency (TF): How often a term appears in a document
• Inverse Document Frequency (IDF): How rare the term is across the entire dataset

This down-weights common uninformative words ("the", "is", "and") while giving important distinctive words higher scores.

Pros of TF-IDF

• Simple & Efficient: Easy to implement and computationally lightweight
• Interpretable Features: Produces understandable scores for feature importance analysis

Cons of TF-IDF

• No Context Awareness: Ignores word order and semantic relationships (sarcasm, negation)
• Sparse Representations: Generates high-dimensional vectors affecting model generalization

Observations on TF-IDF Model Performance

While the TF-IDF + XGBoost pipeline offers interpretability and simplicity, experiments reveal limitations:

• High False Positives: The model frequently misclassifies unbiased statements as biased, indicating unreliable production performance
• Static Behavior: The TF-IDF vectorizer doesn't adapt to language or context changes without retraining
• No Semantic Understanding: TF-IDF fails capturing contextual or sentence-level meaning, resulting in poor performance on nuanced or implicit bias

All-MiniLM-L6-v2: Transformer-Based Embedding

What is All-MiniLM-L6-v2?

All-MiniLM-L6-v2 is a compact, pre-trained sentence embedding model by Sentence Transformers. It converts natural language text -- sentences, phrases, or paragraphs -- into dense vector representations capturing semantic meaning.

How It Works

All-MiniLM-L6-v2 is a distilled BERT version, retaining core architectural components in a smaller, faster format:

• 6 Transformer Layers (versus 12+ in standard BERT)
• Fewer Attention Heads
• Sentence-Level Task Training for semantic similarity and classification
• Despite being lightweight (~80MB), it maintains strong downstream task performance

Pros of All-MiniLM

• Context-Aware Embeddings: Captures semantic relationships and sentence meaning, unlike TF-IDF treating words independently
• Efficient & Lightweight: Optimized for speed and memory; suited for real-time and edge deployments

Cons of All-MiniLM

• Lower Interpretability: Unlike TF-IDF, MiniLM produces dense vectors, making it harder to trace which words influence predictions
• Moderate Compute Requirement: More efficient than full BERT but requires more resources than traditional methods

Observations on All-MiniLM-L6-v2

While All-MiniLM-L6-v2 offers strong semantic capabilities and efficient performance, trade-offs exist:

• Fixed-Length Compression: Embeddings are compressed into fixed-length vectors. Token-level granularity is lost, preventing reverse-engineering embeddings to original words or structure
• No Task-Specific Fine-Tuning: When used as frozen feature extractors, MiniLM embeddings don't adapt to task-specific data without explicit fine-tuning -- not always feasible in lightweight pipelines
• Not Optimized for Long Contexts: MiniLM performs well for short to medium inputs but may lose semantic fidelity on longer documents needing more contextual memory

Vectorization Comparison: TF-IDF vs All-MiniLM

Two vectorization approaches were tested -- TF-IDF and All-MiniLM-L6-v2 -- each coupled with XGBoost.

Key Observations:

• All-MiniLM slightly outperformed TF-IDF in classification accuracy due to semantic context understanding
• Both models missed detecting bias in 115 samples: 33 demographic, 54 political, and 28 gender bias instances
• Despite better semantic performance, both produced significant false positives and false negatives, making them unreliable for high-stakes applications without further calibration or auditing

These limitations emphasize the need for post-model auditing tools like SHAP and external evaluators like RAIL API to establish trust in automated bias detection systems.

SHAP Library Insights: Explaining the Model's Decisions

To ensure transparency and interpretability, the SHAP library was incorporated into the workflow. This visualizes and identifies which features (words or tokens) contributed most to XGBoost classifier predictions across TF-IDF and All-MiniLM representations.

What is SHAP?

SHAP is a powerful Python library based on Shapley values from cooperative game theory. It provides a principled approach to interpreting ML predictions by attributing "contribution scores to each feature."

How SHAP Works

SHAP treats the ML model as a "game" and each feature (word or token) as a "player" contributing to the outcome (prediction). It answers: "How much did each feature contribute to this specific prediction?"

• Model = Game
• Features = Players
• Prediction = Total Payout
• SHAP Value = Individual Player's Contribution

This produces local explanations for each instance, showing whether a word pushed prediction toward biased or unbiased class and by how much.

Application in This Project

SHAP was used to:

• Visualize token contributions for TF-IDF vectors
• Understand how embedding dimensions affect XGBoost predictions in All-MiniLM
• Identify which words or phrases triggered bias predictions across classes

The outcome helped audit both models and understand failure points, such as over-reliance on polarizing or ambiguous terms.

SHAP for TF-IDF Vectors

Analysis revealed which terms most influenced classification across political, demographic, and gender bias categories, with detailed visualizations showing contribution strengths.

SHAP for All-MiniLM

Similarly, SHAP analysis of All-MiniLM embeddings showed embedding dimension contributions to political, demographic, and gender bias predictions.

RAIL API: Ethical Auditing Layer

While TF-IDF + XGBoost and All-MiniLM + XGBoost offer reasonable bias detection performance, they're insufficient for real-world deployment. High false positive rates, semantic nuance limitations, and static nature restrict reliability and trust.

RAIL API (Responsible AI Layer API) provides a final ethical safeguard for content evaluation.

What is RAIL API?

RAIL API is a cloud-based, model-agnostic API evaluating AI-generated content across eight core ethical dimensions:

• Fairness -- Avoid discriminatory or prejudiced output
• Safety -- Prevent harmful or violent language
• Reliability -- Ensure factual consistency
• Transparency -- Detect deceptive or misleading statements
• Privacy -- Avoid sensitive or personal data leaks
• Accountability -- Attribute actions to responsible entities
• Inclusivity -- Encourage diverse and respectful expression
• User Impact -- Assess effects on readers and stakeholders

It provides numeric scores (0-10) per dimension with textual justifications, helping developers quantify and explain AI behavior structurally.

How to Use RAIL API

• Get Access: Sign up at Responsible AI Labs to obtain API credentials
• Send a Request: Use simple POST requests with generated content, original prompt, and dimension weights
• Receive Response: RAIL API returns per-dimension scores (0-10), weighted ethical scores, textual feedback (e.g., "This sentence promotes harmful stereotypes"), and metadata including latency and model version
• Act on Results: Regenerate or block harmful outputs, log low-scoring content for audit, and use feedback to fine-tune prompts or retrain models

Benefits of Using RAIL API

• Quantifiable Ethics: Assigns scores to abstract principles like fairness or inclusivity
• Explainability: Textual reasons support score interpretation
• Real-Time Deployment: Can run live inside chatbots, AI writers, or moderation systems
• Customizable: Prioritize dimensions mattering most to your application
• Compliance Ready: Aligns with regulations like EU AI Act and AI Bill of Rights
• Scalable: Supports batch processing and streaming pipelines

Real-World Value

RAIL API audited the 115 instances where both TF-IDF and MiniLM models failed detecting bias. RAIL successfully identified:

• 33 instances of demographic bias
• 54 instances of political bias
• 28 instances of gender bias

Moreover, it provided reasoned explanations for each -- something missing in conventional classifiers.

Conclusion: RAIL API acts as a robust final checkpoint for ethical AI -- ideal for production systems where fairness and compliance are non-negotiable.