We are galloping in step with technology. Fascination carries us forward. But there are questions we rarely pause to consider.
Bias in AI is often discussed as a technical problem, something that can be adjusted, coded away, fine-tuned. But bias is not just about data. It is about which interpretations are given priority. Which worldview is reinforced. Which values are amplified, and which are cut away.
We know that training data carries the patterns of history. We know that models already show signs of skewed output, even at the usage stage. So who decides what is relevant? Who decides what is sufficiently true?
The one who controls the weighting of an AI model ultimately controls which information shapes our decisions. Our assumptions. Our perceptions.
Here, we need to talk about cognitive integrity.
We need to understand how AI output is formed, in order to respond with discernment. This requires new competencies. It requires understanding bias as a structural dimension of the technology itself.
What is bias in AI and why does it occur?
Bias in AI occurs when a model systematically produces skewed or disproportionate results. It is not about faulty code, but about the data the model was trained on, how it is constructed, and how it is applied in practice.
Data carries values, history, and patterns. AI systems trained on this data will reflect what they have learned, with biases often preserved and even amplified.
Bias does not occur alongside the technology, it is built into how the technology interprets the world.
Three levels of bias
-
Data Bias
The model learns from material that already reflects inequalities or exclusion. Its outputs follow the same lines. -
Design Bias
The model’s architecture determines which correlations are prioritized. Technical design choices affect which information gets amplified. -
Usage Bias
Even a well-trained model is influenced by how it is used. It can respond differently depending on prompts, context, and the user’s background.
About the framework
The three-part structure described above is a coherent model based on established research in AI and ethics. Similar categorizations appear in the work of Barocas, Hardt & Narayanan, Suresh & Guttag, and Mehrabi et al.
The terminology can vary, for example, terms like measurement bias, deployment bias, or representation bias are also used — but the core division into data sources, model construction, and usage context is a recurring analytical lens.
This summary is intended to provide an accessible structure for understanding how bias in AI systems can arise and be reinforced in practice.
Common Biases to Know
Biases in AI systems often mirror those found in psychology and decision-making, but automation and scale can amplify them. Below are five common biases frequently found in training data, model behavior, or result interpretation.
Scroll sideways to view the full table:
| Bias | Description | Example in AI |
|---|---|---|
| Confirmation bias | The model reinforces existing assumptions or expectations | Search engines display information that confirms the user’s prior behavior |
| Representation bias | The data does not reflect real-world variation | Facial recognition works less accurately for people with darker skin tones |
| Selection bias | The choice of training data is not representative of the intended application | Recruitment systems trained on past hires reproduce historical disparities |
| Anchoring bias | The first input disproportionately influences subsequent decisions | An AI assistant provides different answers depending on the initial prompt |
| Survivorship bias | Only visible or successful examples are included in the data | Systems trained on success stories ignore those absent from the outcome se |
There are other forms of bias that affect AI systems, such as label bias, hindsight bias, and framing bias. The selection above focuses on some of the most pervasive and recurring ones, especially in AI applications like language models and decision support systems, where output influences people’s perceptions, choices, and interpretations in real time.
This applies, for example, to:
-
Language models (LLMs) like ChatGPT, Claude, and Gemini
Used to answer questions, generate text, and make recommendations. -
Recommendation systems in social media, e-commerce, and search engines
Determine which posts, products, or results the user sees. -
Decision support systems in HR, healthcare, education, and public administration
Suggest actions, priorities, or interpretations of data for human decision-makers.
Why Language Models are especially relevant
Language models are particularly relevant in this context because:
- They are often used by non-experts with no insight into how the model is built.
- They express themselves with confidence, creating a sense of authority.
- They often act as the first cognitive suggestion in a thinking process, making them especially prone to reinforcing anchoring bias, confirmation bias, and framing.
From technology to workflow, what happens when AI is integrated?
AI has become an integrated part of workflows in many organizations. Not as an external add-on, but as a systematic tool in every stage, from input to decision.
Where AI can contribute to efficiency, it will be used. It becomes woven into structures, production flows, decision-making processes, and documentation. Many already use AI daily. Others hesitate. Concerns about being replaced are common, but in practice, the shift is more of a transformation than a substitution. Humans remain essential in the decision-making process. AI acts as a structure, analysis engine, and amplification tool.
For this collaboration to be sustainable, transparency is needed, not only about what data was used, but about how interpretation happens, which factors are weighted, and which perspectives are excluded.
When workflows rely on AI decisions without understanding the model’s internal logic, our professional discernment weakens. This impacts quality, accountability, and ultimately an organization’s decision-making capacity.
What is Cognitive Integrity and why does it matter?
When AI is integrated into workflows previously handled by humans, the conditions for how information is interpreted and used change. As AI output influences decisions, documentation, and communication, we must develop the ability to understand how information is shaped and which structures influence its content.
Cognitive integrity means being able to process information with awareness of its origin, interpretation, and emphasis. It is about seeing which assumptions drive the model, which perspectives gain traction, and how answers are structured.
AI generates probability-based interpretations. Therefore, we need the ability to read the result with distance and systemic awareness. Technical literacy is not enough — cognitive discernment must be part of our professional competence.
This requires a culture where understanding how AI shapes content is seen as a natural part of work. The concept of cognitive integrity is further developed in other texts, with applications in education, decision-making, and cognitive endurance.
The competencies we need now
- The ability to read AI output as weighted information
- Insight into model architecture, data foundations, and interpretive mechanisms
- Strengthened metacognitive capacity in the workplace
- Transparency as a precondition for trust
When AI affects both information flows and decision chains, we must combine technical understanding with cognitive discernment. It is about seeing the connections between system logic, probability-based output, and the model’s underlying structure.
For those who want to explore further on interpretation, attention, and mental structuring in AI-influenced environments, there is a related article on cognitive integrity as a systemic requirement.
Further Reading
The EU is currently working on creating a legal framework for AI through the AI Act and the principles in the GPAI Code of Practice. Still in Swedish for now.
Sources
- Bender, Gebru, McMillan-Major, Shmitchell (2021): On the Dangers of Stochastic Parrots
- EU: GPAI Code of Practice
- Barocas, Hardt & Narayanan: Fairness and Machine Learning
- Stanford HAI: Foundation Model Transparency Index
- MIT Technology Review: Bias in Large Language Models (2023)
