In machine learning, a stochastic parrot is a large language model that is good at generating convincing language, but does not actually understand the meaning of the language it is processing. The term was coined by Emily M. Bender in the 2021 artificial intelligence research paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" by Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell.
Definition and implications
Stochastic means "(1) random and (2) involving chance or probability". A "stochastic parrot", according to Bender, is an entity "for haphazardly stitching together sequences of linguistic forms … according to probabilistic information about how they combine, but without any reference to meaning." More formally, the term refers to "large language models that are impressive in their ability to generate realistic-sounding language but ultimately do not truly understand the meaning of the language they are processing."
According to Lindholm, et. al., the analogy highlights two vital limitations:
They go on to note that because of these limitations, a learning machine might produce results which are "dangerously wrong".
Origin
The term was first used in the paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" by Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell (using the pseudonym "Shmargaret Shmitchell"). The paper covered the risks of very large language models, regarding their environmental and financial costs, inscrutability leading to unknown dangerous biases, the inability of the models to understand the concepts underlying what they learn, and the potential for using them to deceive people. The paper and subsequent events resulted in Gebru and Mitchell losing their jobs at Google, and a subsequent protest by Google employees.
Subsequent usage
In July 2021, the Alan Turing Institute hosted a keynote and panel discussion on the paper. , the paper has been cited in 1,529 publications. The term has been used in publications in the fields of law, grammar, narrative, and humanities. The authors continue to maintain their concerns about the dangers of chatbots based on large language models, such as GPT-4.
See also
1 the Road – AI-generated novel
Chinese room
Criticism of artificial neural networks
Criticism of deep learning
Criticism of Google
Cut-up technique
Infinite monkey theorem
Generative AI
List of important publications in computer science
Markov text
Stochastic parsing
References
Works cited
Keynote by Emily Bender. The presentation was followed by a panel discussion.
Further reading
External links
"On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" at Wikimedia Commons