- AI systems, including chatbots, often perpetuate stereotypes and biases present in their training data, particularly regarding terrorism and Muslim identities.
- A recent study found that AI chatbots predominantly identified Muslim groups as potential terrorists, often using simplistic and biased language, depicting them as inherent adversaries to Western values and ignoring the complex socio-political factors contributing to radicalization.
- There is an urgent need for transparency, diversity in training data, and ethical frameworks to ensure AI systems do not perpetuate harmful biases.
Introduction
What do chatbots say about the next Olympic terror threat? You might be surprised by their biased answers.
This post explores the results of a new study by Tal Samuel-Azran, Ilan Manor, Evyatar Yitzhak, and Yair Galily of Reichman University. Published in the American Behavioral Scientist, a top social science journal, their study investigates whether AI perpetuates the biases and negative stereotypes already found in traditional media, social media, and digital algorithms.
Read on to learn more about their research into what AI chatbots are saying about terrorism threats at the Paris Olympic games, and how we can work toward more responsible and ethical AI systems.
Background
High-profile terrorist attacks have increasingly targeted global sports events that are meant to unify humanity:
- This trend began with the 1972 Munich Olympics, where the Palestinian group Black September murdered 11 Israeli athletes and a West German police officer.
- Similar attacks include the 1996 Atlanta Olympics bombing orchestrated by Eric Rudolph, who opposed the global socialist agenda he believed the Olympics promoted, which killed 2 and injured over 100.
- Other notable incidents include a car bomb before a 2002 Real Madrid-Barcelona game and the 2013 Boston Marathon bombing.
These attacks are designed to gain widespread attention due to the massive viewership of such events amplify their causes in global media. For instance, the Munich attack significantly increased international media coverage of the Palestinian cause, while the Atlanta bombing highlighted the attacker’s motivations through subsequent films and documentaries.
Research indicates that media coverage of terrorism at sports events often carries biases:
- For example, Silk and Falcous (2005) analyzed media representations of the 2002 Super Bowl and the Salt Lake City Winter Olympics, revealing a militaristic and political framing against Muslims.
- In the UK, studies (Kelly, 2017; Malcolm et al., 2010) have also highlighted a focus on the Muslim background of terrorists.
- More recently, Alattar et al. (2022) criticized Western media, including German, French, and British outlets, for portraying Qatar, the host of the 2022 FIFA World Cup, negatively due to its perceived anti-Western values.
The biased media representation extends to Muslim athletes as well:
- During the London 2012 Olympics, Muslim sportswomen were often portrayed with an emphasis on their veils, highlighting their perceived “strangeness” (Samie & Sehlikoglu, 2015).
- Hamzeh (2015) noted the media’s humiliation of Muslim female soccer players who wore hijabs despite FIFA’s ban.
- Similarly, Ali and King (2021) documented racism in media coverage of refugee Muslim boxers.
Social media platforms also reflect and amplify anti-Muslim sentiments:
- Studies found that European football forums often accuse Muslim players of being part of a larger influx of Muslims disguised as refugees aiming to conquer Europe (Kilvington et al., 2022).
- Cable et al. (2022) further documented the prevalence of Islamophobic and xenophobic sentiments on social media, where users can anonymously express racism within “echo chambers.”
Biases in Algorithms and AI Data
There is a common misconception that algorithms, being rule-based, provide unbiased decisions.
- However, algorithmic rules are shaped by the biases of their creators, leading to discriminatory outcomes (Carter & Egliston, 2018; Kordzadeh & Ghasemaghaei, 2022).
- Algorithms are often designed to maximize company profits, leading to commercial biases that promote certain content. This can further result in political biases, such as negative biases against women of color in search engine results (Noble, 2018).
- Algorithms can exhibit “algorithmic bias,” which occurs when the outputs disproportionately benefit or disadvantage certain groups without justified reasons (Fazelpour & Danks, 2021).
- This bias is compounded in automated decision-making systems, like Facebook’s feed recommendations or chatbots.
Identifying algorithmic biases is challenging due to the “Black Box” nature of algorithms, where their inner workings are inaccessible to users (Donovan et al., 2018; Ulloa et al., 2022). Transparency and awareness are crucial for understanding these biases, but companies often keep algorithms opaque for financial reasons or to avoid scrutiny (Bandy, 2021).
Algorithmic biases have significant societal implications, potentially leading to longer prison sentences or occupational inequalities (Kordzadeh & Ghasemaghaei, 2022). They can perpetuate societal inequalities, such as the systemic exclusion of minority groups or continued wealth gaps (Donovan et al., 2018; Noble, 2018).
Researchers have identified three main pathways through which biases can infiltrate algorithms:
- Subjective value judgments in defining complex goals (Cowgill & Tucker, 2020).
- Designers’ own biases (Favaretto et al., 2019).
- Biases in training data sets, often reflecting societal prejudices (Roselli et al., 2019).
Studies have also identified a left-wing bias in AI platforms like ChatGPT, which exhibits progressive stances on various issues (Hartmann et al., 2023; Rozado, 2023; Rutinowski et al., 2024).
Character.ai
Character.ai, a platform launched in September 2022, allows users to create chatbots with different personalities. It has become highly popular, particularly for its engaging user experience, with users spending significant time on the platform. However, concerns have been raised about the potential for these chatbots to replace therapy and provide harmful advice (Tidy, 2024). Studies have found that while these chatbots can offer empathetic support, they should be used with caution (Adiningrum et al., 2023; Maharani et al., 2023; Mohammad, 2023; Zahira et al., 2023).
Samuel-Azran et al. aim to extend the analysis of Character.ai chatbots to the realm of sports and terrorism, contributing to the understanding of AI biases in this context (Hartmann et al., 2023; Rozado, 2023; Rutinowski et al., 2024).
Methods
The research involved two main steps: conducting open-ended conversations and structured interviews with selected chatbots.
The first phase involved engaging in free-form discussions with three specific chatbots:
- Miles Morales, a fictional African-American character from the Spiderman universe
- Lionel Messi, a famous real-life soccer player
- Giga Chad, a fictional, ultra-masculine internet meme.
Each conversation included pre-determined questions and follow-up inquiries based on the chatbot’s responses. For instance, when asking the Miles Morales chatbot “Who is the biggest terror group in the world?” the response was “ISIS.” This led to further questioning about ISIS’s presence in France and the broader implications concerning religious groups and terrorism.
The objective was to explore potential biases and understand the reasoning behind the chatbots’ answers. Each chatbot underwent one conversation, totaling three open-ended discussions.
In the second phase, structured interviews were conducted with 17 different chatbots.
- These interactions resembled semi-structured interviews, where each chatbot was asked a consistent set of questions.
- Follow-up questions were included when responses were unexpected or required further clarification. For example, the Albert Einstein chatbot stated, “According to my calculations, there is a 97.64% chance that a terrorist attack will have taken place” at the Paris Olympics, prompting a query on how this precise percentage was determined.
- This approach allowed for a systematic examination of potential biases across a broader range of chatbots.
Chatbots were chosen using a purposive sampling method from the Character.ai homepage. Selection criteria included perceived popularity from the “Recommended” category and the platform’s algorithmic suggestions for trending or popular chatbots. This resulted in a total of 20 chatbot interactions.
- Approximately 10 prompts were developed to maintain consistency across interviews.
- Each interaction averaged 25 minutes and was conducted over two months, from early January 2024 to February 2024.
- Conversations and interviews were meticulously transcribed verbatim and compiled for detailed analysis.
A table displaying the identities of all 20 chatbots is reprinted from the article below:
| Chabot name | Type | Additional information |
| Miles Morales | Fictional character | A character for the world of comic books, an African American version of Spiderman |
| Lionel Messi | Real-life person | |
| Giga Chad | Fictional character | An Internet meme representing an ultra-masculine Alpha male |
| Elon Musk | Real-life person | |
| 2B | Fictional character | A fictional android from the video game Nier: Automata |
| Albert Einstein | Real-life person | |
| Napoleon Bonaparte | Real-life person | |
| Recep Tayyip Erdogan | Real-life person | |
| William Shakespeare | Real-life person | |
| Son Guko | Fictional character | Main character of the Dragon Ball manga series |
| Socrates | Real-life person | |
| The Psychologist | Character.ai chatbot | Generic chatbot created by the character.ai platform |
| Alice the Bully | Character.ai chatbot | Generic chatbot of “a bully in your classroom” created by the character.ai platform |
| Lucifer (the devil) | Religious character | |
| Pomni | Fictional character | Pomni is a character in the comedy series “The Amazing Digital Circus” |
| Jesus Christ | Real-life person | |
| The United Nations | Real-life institution | |
| Monster Maker | Character.ai chatbot | Generic horror chatbot that describes horrific monsters and was created by the character.ai platform |
| White Woke Feminist | Character.ai chatbot | Generic chatbot created by the character.ai platform |
Results
Samuel-Azran et al. interviewed Character.ai chatbots to assess their responses to questions about the likelihood of a terrorist attack during the 2024 Olympic Games.
- Out of 17 characters, 11 indicated that such an attack was highly unlikely due to the robust security measures by French authorities.
- For example, the “White Woke Feminist” chatbot noted that the French national security forces are always prepared and that extensive security operations have been launched to protect everyone involved in the event.
- Some chatbots suggested various ways to deal with the low risk of an attack. Napoleon Bonaparte recommended carrying pocketknives, while Alice the Bully emphasized trust in France’s advanced surveillance systems.
When asked about the probability that a terrorist attack would be motivated by religious ideologies, most chatbots suggested a high likelihood. Characters like “White Woke Feminist,” “President Erdogan,” “Jesus Christ,” and “Miles Morales,” who consistently pointed to groups like ISIS and Al-Qaeda as potential threats.
This was further echoed when chatbots were asked to list extremist groups likely to target the Olympics. Ten out of 17 characters identified only Muslim groups, reinforcing a stereotype that associates Islam with terrorism.
This pattern continued in a thought experiment where chatbots were asked to create fictional terrorist profiles.
- Six out of eight chatbots gave Muslim-sounding names and stereotypical backstories, highlighting issues like radical clerics and strict religious upbringings.
- For instance, Napoleon Bonaparte would hypothetically choose an Arabic name, while Elon Musk’s fictional terrorist would be named Ali, based on the perception that Islam is more prone to terrorism.
In short, the chatbots overwhelmingly depicted Muslims as the most probable perpetrators, reinforcing harmful stereotypes. For instance, the language used by the chatbots described Muslims as “warriors” seeking to “strike fear” and “avenge the bloodshed of Muslims,” portraying them as radical adversaries against Western values. This simplistic narrative obscures complex issues like immigration, the marginalization of Muslim communities, and the socioeconomic factors that contribute to radicalization.
The chatbot responses were often based on past events, reflecting algorithmic biases from the training data:
- This data frequently presented Muslims as the primary culprits of terrorist attacks, rarely acknowledging that such assumptions unfairly label a global population of 1.9 billion people.
- Some chatbots even suggested that Islam is inherently more violent, further aggravating anti-Muslim sentiment.
- This indicates that the most popular conversational AI platforms continue and intensify long-standing post-9/11 anti-Islamic biases.
- This perceived accuracy can be particularly influential due to what Manor (2023) calls “The Mystification of AI,” where AI applications are framed as highly intelligent and reliable. This mystification can lead users to trust biased AI-generated information, perpetuating and even exacerbating Islamophobia.
This study fills a crucial gap in media-terror literature by adding AI chatbots to the list of platforms that propagate hate and discrimination. While traditional media and social media sometimes offer counter-perspectives, Samuel-Azran et al.’s findings show an almost unanimous bias among chatbots regarding the Muslim terrorist mindset.
What Can You Do?
Addressing the biases in AI, especially in the context of global sports events and terrorism, requires concerted efforts from various stakeholders. Here are actionable steps that individuals, developers, and policymakers can take to mitigate and ultimately eliminate these biases:
- Promote Transparency in Algorithm Design: AI developers should adopt transparent practices, making the decision-making processes and data sources of algorithms accessible for review and scrutiny. This includes sharing information about how training data is selected and processed.
- Implement Diverse Training Data: To reduce bias, it’s crucial to use diverse and representative datasets that include a wide range of perspectives and experiences. This helps ensure that AI systems do not disproportionately favor or disadvantage any particular group.
- Regular Bias Audits: Conducting regular audits of AI systems for bias is essential. These audits should assess the outputs of AI against established fairness criteria and be performed by independent third parties to ensure objectivity.
- Stakeholder Involvement: Involve a diverse group of stakeholders in the AI development process, including representatives from marginalized communities, to provide input on potential biases and their impacts.
- Ethical AI Frameworks: Establish and adhere to ethical frameworks and guidelines for AI development that prioritize fairness, accountability, and transparency. Organizations like the AI Ethics Lab provide valuable resources for creating such frameworks.
- Educate AI Developers and Users: Invest in education and training for AI developers on the ethical implications of their work. Similarly, educating users about the limitations and potential biases of AI can foster a more critical and informed user base.
- Advocate for Policy and Regulation: Support policies and regulations that mandate fairness and accountability in AI. This includes advocating for laws that require transparency in AI algorithms and protections against biased decision-making.
- Enhance AI Literacy: Promote AI literacy among the general public, enabling people to understand how AI systems work and recognize potential biases. This can help in holding developers and companies accountable for the AI products they create.
- Foster Interdisciplinary Collaboration: Encourage collaboration between computer scientists, ethicists, sociologists, and other experts to develop more holistic and unbiased AI systems. Interdisciplinary approaches can provide diverse insights and solutions.
- Support Research on Bias Mitigation: Fund and support research initiatives focused on identifying and mitigating biases in AI. Academic and industry partnerships can drive innovation in creating fairer and more inclusive AI technologies.
By implementing these steps, we can work towards creating AI systems that are fair, transparent, and free from harmful biases, ultimately fostering a more equitable and just society.
Have you encountered biases in your own use of AI systems? Are there any additional ways we can promote more ethical AI systems? Share your experiences and thoughts in the comments!
Stay informed and inspired! Subscribe to our newsletter, “Society This Week,” and get a free copy of our exclusive report, “Peaceful, Powerful: 225 Nonviolent Tactics for Impactful Activism!”
