The technical development within generative AI has reached a point where large language models not only consume information, but produce the majority of new text on the internet. These systems also rely on training data to develop their language capabilities. This creates a structural tension:
When language models are trained on their own output, they risk losing the ability to contribute new information to the cognitive ecosystem.
Mode Collapse and homogenization
Large language models are optimized to guess the next most probable word. This leads to models generating standardized, predictable, and grammatically correct texts, but with limited variation. When these texts are used as training data for next‑generation models, a phenomenon called mode collapse emerges:
Nuances disappear. Arguments become similar. Expression becomes homogenized.
Cognitively, this is a form of self‑reference that weakens the system’s capacity for renewal.
Epistemic loop and truth as frequency
Generative AI does not access truth itself, but builds responses based on what occurs frequently in the training data. If AI‑generated text begins to dominate the web, then "what AI often says" becomes "what AI thinks is true".
This creates an epistemic closed loop where:
- bias is reinforced
- minority perspectives are marginalized
- innovation in expression is interpreted as error
The concept of truth is reduced to statistical weight.
The appearance of science split from scientific process
Large language models can imitate the tone, structure, and logic of scientific writing. But they do not participate in:
- hypothesis testing
- methodological uncertainty
- peer review or epistemic resistance
It looks like science.
It feels like science.
But it is only the syntax of epistemic authority.
This is especially risky when AI‑created texts are used in education, policy, or research contexts.
What is required to break the loop?
-
Traceability
Systematic metadata about origin, methodology, and whether the text is AI‑generated -
Data Curation for Cognitive Breadth
Include varied forms of knowledge: ethnography, field data, interdisciplinary reasoning -
Special Models for Uncertainty and Hypothesis Generation
AI that not only gives answers, but dares to formulate questions -
Human Interpretive Responsibility
Knowledge production must not be automated away from accountability and reflexivity
A conscious design shift in how we build, use and understand AI systems is necessary. Otherwise we risk creating intelligent mirrors that only confirm our most recent echo.
Further questions: What does AI actually learn?
This is no longer about language only, but about how systems learn to adapt their learning based on reward, penalty, and patterns in human behavior. As language models shape our information environment, self‑learning AI systems in reinforcement learning (RL) and meta‑reinforcement learning (Meta‑RL) also evolve.
But what happens when these reward structures are shaped by fast clicks, superficial interactions or dopamine‑driven patterns of social media?
What kind of "knowledge" is then trained, and what long‑term effects arise if both language and learning patterns begin to revolve around the same limited logic?
Often overlooked is an additional layer:
Since language models construct answers based on probability, they tend to filter out perspectives that do not follow established patterns. Innovative reasoning, original connections, or hypotheses that have not yet appeared widely in text risk being filtered out—not because they are incorrect, but because they are unlikely.
When the unlikely is filtered out of language models, the potentially new is also filtered out of knowledge production.
This article only scratches the surface. Deeper questions remain open.
Related reading
If you're interested in how AI-generated confirmation shapes human cognition and emotional engagement, read:
Synthetic Safety and AI: When confirmation replaces inquiry
