Smart (part 2) - a way with words

In part 1, I started exploring why the myriad articles on Artificial Intelligence (AI) leave me feeling so frustrated. 

To start, I explored the abstract and contested concept of intelligence in humans, and then as applied to machines. I concluded that while so-called AI can do some gobsmackingly sophisticated and incredibly useful things, and can far surpass human capacities in specific functions, this isn’t what I would label intelligent. 

So, I worked out one major source of frustration for me is the lack of sophisticated understanding about intelligence in humans by those trying to create an artificial version of intelligence. Disconcertingly, computer science is building on the computational model¹ of human intelligence just as cognitive science is abandoning it!   

Despite my personal doubts, the latest AI-based chatbots appear to many users to be really quite smart. They communicate via voice or text, using language a lot like humans do. AI programmers are focusing their efforts on the ability of smart chat bots to interact with humans in natural language.  

Are the AI research scientists claiming that language makes machines intelligent? That sounds like Wordly Exploration territory!

Smart with words

In part 1, I discovered that the etymology of the word smart meaning ‘intelligent’ included a much older meaning of ‘good or quick with words’. That's rather interesting, it taps into something that we still commonly do: we tend to assume that those people with a way with words are pretty smart while those who can’t talk well or much, well they just aren’t.²  

It’s a heuristic – a short cut or hidden assumption in thinking about the world.

This hidden assumption is that spoken or written language reveals the underlying thinking of a person. We take the short cut of using someone's language abilities to make assumptions about their underlying intelligence. Intelligence is something humans have traditionally linked to the value they place on people and other animals (and is linked to our assumptions about sentience or consciousness).³

It’s a useful heuristic some of the time, or it wouldn't persist. 

But it’s actually wrong, as we shall see. 

Machines with ‘natural’ language

Previously, I explored the gradual change in meaning of the word smart as applied to devices and machines. The most recent iteration for smart chatbots uses smart to mean ‘using natural language’. 

These bots can communicate a lot like humans, so the heuristic we use is that they must also be intelligent, right? In fact, that is exactly what some claim. 

You probably know already how they work: the machines are trained⁴ with almost the entire history of human language online⁵ which they reference to predict (very rapidly) the next most likely word in a sentence. They select from the gargantuan collection of existing human dialogue and text, base the word selection on statistical probability shaped or refined by algorithms⁶, and with this they can often do a pretty good job of producing human-like language. 

The chatbots use large language models (LLM) (or more accurately generative large language multimodal models - GLLMMs), something referred to as a ‘neural network’⁷ trained on massive amounts of data to interpret and generate language. Unlike the earlier rule-based chatbots, AI-powered chatbots use software that can ‘learn’⁴ and ‘adapt’ to new contexts and can handle a wider range of inquiries from users.

They give the impression of being creative, using reasoning, interpreting visual input (including humour) and they can handle and usefully reference a large amount of text (e.g. ‘reading’ books in seconds). But they also make a lot of errors and say straight-out silly things (see a few of the images).

As well, obvious limitations include being prone to reasoning errors and biases and making up false information. In fact, the smart chatbot programming can be manipulated relatively easily to create false, biased and hateful comments.

The most recent chatbot machines have overcome some of these limitations some of the time, using a LOT of human training⁴ with feedback and corrections on inaccurate outputs, and even more language data inputs⁸. The smart bots continue to get better and better at producing what sounds like natural language, increasingly more like human conversations with fewer (but still plenty of) non-sensical utterances and glaring errors (but also lots of hidden errors). 

Think about that data sample: they are drawing on the vast number of human-produced sentences, chatgroup discussions, stories, dissertations, meeting minutes, newspaper articles, novels, podcast transcripts, etc. etc., available online. A powerful enough AI can find content suitable to respond to most questions and prompts. Using the next-most-likely word prediction method, they can produce quite plausible responses. Ask ChatGPT to write you a news article on politics in England, no problem; ask it to write your wedding vows, there is plenty of data for it to draw on; get it to write an essay on ethical issues in research or interpret visual humour, same. Over time, the outputs are improving through ‘interaction’ with more users providing further adjustments to programming. 

Good enough language to convince the human interactor of the machine’s intelligence? Yes, in some cases.

But does the generation of what sounds like natural language mean these machines really are smart? 

That might depend on what we mean by language. 

What is language?

Let’s take a look at language then - another very complex topic I will skim over very briefly. One simple and useful perspective⁹ is to consider language as being made up of form, content and social use. (See the diagram to the right.)

Form is perhaps the easiest aspect of language for machines due to grammar’s rule-based prescriptions for which words can go in what sequence in a sentence. AI has made enormous strides in the area of grammar through its statistically-most-likely-next-word prediction model plus ‘training’ - it has been fed grammatically formed language input.

In terms of the area of content, AI continues to improve. Given its vast amounts of source material and its statistically-most-likely-next-word prediction model, these machines have access to many more words and language samples than any one human. Cue words in the questions will guide appropriate content words in the responses.

However, in the area of social use, smart chatbots are pretty poor. We can understand why by considering the distinction between ‘formal language competence’ and ‘functional language competence’. Formal competence includes the rules, patterns and word knowledge of a language (form and content), while functional competence integrates this with a host of cognitive abilities and world knowledge in order to achieve the purposes of the communicator. 

In fact, the updated model (to the right) shows a better way to think about formal versus functional language competence.

Where AI falls down is the complex relationship between form and content in social use to generate meaning. This is the source of the majority of language errors by the chatbots.

That is because meaning is not a given based on the words involved. Meaning is negotiated between language users, constantly. Meaning does not flow directly from words compiled into a grammatically correct sentence, but from the speaker and listener (or writer and reader) conjuring an image based on the content and form to make and share and sometimes re-negotiate the sense of what is said. 

Human language users can do this easily.

There is no meaning, no understanding - so no intelligence

In contrast, AI does not work in the realm of meaning at all. 

If an AI response to a question seems to be appropriate in the context, that is because the machine has sourced language previously produced by a human in a similar context. It produces the illusion of natural language use by using existing human language samples and lots (and lots) of human ‘training’.⁴ 

As Noam Chomsky said, AI is essentially working as high tech mass plagiarism software, and not artificial intelligence at all. And just like a plagiarising high school student, the AI doesn't understand what it's generating!

So, AI is not really using natural language because it is not using language to create, share and re-negotiate meaning. It’s using predictive technology based on human-produced samples refined though human-provided feedback on accuracy and relevance to generate plausible responses that convince those interacting with it that it is using natural language. 

AI provides the experience of human-like conversation, without the AI understanding what it being asked or produced. It generates language patterns based on the programming. And this is where serious concerns about applications of AI arise – lacking any understanding of the real world, AI chatbots have no way of determining if the statistical patterns they find and generate are useful, valid and helpful or merely meaningless coincidences. And this matters.

Relying on an AI that has no understanding of the world is hardly an intelligent thing to do!

Language is a tool for social purposes

When humans use language, what we say and write is required to fit the very complex rules of social use of language.

In fact, no one has (or ever could) described or explained all the social rules (and norms) of our complex world of social interactions. There’s no list! And they change. But we know how to use them (well, most of us and most of the time) and we know when someone else contravenes them (e.g. if a person uses casual or familiar language in a job interview, etc.). 

One way to think about the dynamic nature of language use, and how it works to construct shared values and meaning is post-structuralism. Post-structuralism views language as an endlessly changing organic, cultural, social and historical play of signs and symbols. We each have personal perspectives and different goals, which we negotiate (most often) through language interaction.  

"... AI chatbots are just algorithms,
nothing more than 'stochastic parrots' without
values, or souls, or hearts of their own."
From Erik Davis article, 2023

A key concept from post-structuralism is that language itself carries the agency,¹º intent, perspective, context, and much more, of each of the communicators. Words are not static units of meaning that never change. Words are dynamic tools used to negotiate meaning between communicators.

This most important aspect of language is not something AI can ever venture into. That is because AI is not ‘an agent’ in an interaction. 

At best, some critics say that AI is a parrot¹¹ drawing on a vast store of language samples previously produced by humans, selected using statistical prediction and according to various algorithms. With lots of training⁴, it can do better and better. Its ‘natural language’ outputs look plausible, if you don’t look too hard.

So, AI-powered chatbots generate what seems like natural language, but the next-most-likely-word approach governed by innumerable parameters is not anything like human social use of language. Meaning is missing, social purpose and context does not feature, and negotiation is absent. 

Chatbots might be a very useful tool, but there is no ‘agent’ interacting with us. 

And thus, there is no agent to be intelligent.

Why a predictive next word method appears convincing

I’ve been at social gatherings where we throw challenges at ChatGPT, and the results seem to convince my friends that AI is producing human-like language. But, for me, it highlights how much repetition and formulaic communication is used by humans.

Humans copy other people's language a lot. In big chunks. Very, very few human sentences or stories are completely original. We use what we’ve heard before, in new combinations perhaps, but within some commonly known grammatical, conceptual and social conventions for how we are supposed to communicate in various contexts. 

In fact, we could safely say that a lot of what people express involves repetition and limited content. We use event-specific language ‘recipes’ (wedding vows, insurance negotiation, holiday bookings), formulaic phrases and words for particular uses (work meetings, protests, book club), pre-existing language scripts (at the doctors, in politics, parenting), and so on. For example, the appropriate language for wedding vows might be novel the first time someone gets married, but the fitting words and the suitable sentences are quite limited and predictable. 

The reality is that, although capable of doing so, humans rarely generate completely new language structures or meanings. Instead, they create shared meaning through their interactions within socially and linguistically prescribed formula. The social use of language is associated with cultural values and social roles and expectations.

Humans, who are intelligent (apparently!), do a lot of rote or routinised things, things they don’t really think about. Humans say a lot of stuff over and over, and often there is not a lot of thinking going on.

So, what humans very often do is paralleled by a smart chatbot – they sample from a set of formulaic words and sayings that fit that specific and well-known social context and purpose.

But this is not the limit of human capacity. It’s just that we don’t need to use this larger capacity that often. What we really need to do is fit in, get on with others. Sharing information and ideas through language with other people is part of this. 

And that is what we use language for – it is a social tool for social purposes.

Is language a good proxy for intelligence?

Some in the tech world seem to think that by putting enough language data in, fine tuning and practice, intelligence will eventually emerge. They use natural language output as a proxy for intelligence. After all, we often use it as such both formally (in verbal IQ tests) and informally (ascribing intellectual impairment to people who cannot speak) as mentioned above - it's a common heuristic.

But is language actually a good proxy for intelligence? 

Actually, there is a heap of evidence to show that language is not the same thing as intelligence (in humans) and that a person’s capacity for language does not necessarily equate with their intelligence.

The most telling evidence comes from people who have had brain injury and develop global aphasia – leaving them with almost no ability to understand or produce language. For example, Fedorenko and Varley (2016) wanted to investigate if thought was possible without language. They write in the abstract:

Astonishingly, … despite their near-total loss of language … individuals [with global aphasia] are nonetheless able to add and subtract, solve logic problems, think about another person’s thoughts, appreciate music, and successfully navigate their environments. Further, neuroimaging studies show that healthy adults strongly engage the brain’s language areas when they understand a sentence, but not when they perform other non-linguistic tasks like arithmetic, storing information in working memory, inhibiting prepotent responses, or listening to music.

Taken together, these two complementary lines of evidence provide a clear answer to the classic question: many aspects of thought engage distinct brain regions from, and do not depend on, language.

This (and other evidence too detailed for this post) suggests that thinking is possible without language, and meaningful information can be produced without language. So, while we commonly use language to share our ideas and thoughts, language is not thinking itself. We might use language as an instrument for complex or abstract thinking and reasoning, but again, it is a tool for the process and products of thinking, not thinking itself. 

If language is not the same as thinking, then language is not a great proxy for intelligence. 

Human language is powerful because we combine it with thinking and other capacities that make up our intelligence. 

If the parts of the brain that humans use for language are not responsible for thinking, then we can predict that AI using Large Language Models (LLMs) will get better at producing natural sounding language, conversation, providing answers, etc., but still not be able to do those aspects of thinking that we consider intelligence.  

And that is what we see: LLMs routinely generate coherent, grammatical and seemingly appropriate language in text and voice. AI has achieved a degree of formal language competence.¹² This has led to speculation that these LLMs are, or will soon be, intelligent. However, LLMs fall down on functional language competence which entails the negotiation of meaning for understanding through integrating language with other cognitive abilities such as formal reasoning, situation modelling, social interpretation and world knowledge.   

So, language as a proxy for intelligence is the hidden assumption under the approach to use massive amounts of language inputs and refinements to ‘create’ intelligence. But we have seen it is a poor proxy. Because language and intelligence are two discrete things/concepts, a diet of more and more language cannot not make a machine intelligent. 

As Alan Blackwell says, continuing to increase computer power in one area, like language processing, in the hope it will create a fundamentally different thing, like intelligence, is like ‘arguing that if we make aeroplanes fly fast enough, eventually one will lay an egg.’

Convincing humans

So, I’ve identified a second major source of my frustration when reading articles about AI. I think it is only possible to claim that chatbots and other AI are ‘intelligent’ using an impoverished idea of what language is, what thinking is, and what intelligence is. 

Source: SMBC

The impression of intelligence arises from mathematics and programming. A LLM contains a record of how particular words have been used in the vast amounts of text provided to it. This vast source material means the system can replicate appropriate grammar patterns, along with the impression of personal communication style. When you ask questions using particular words in a particular order, this is correlated with the billions of existing data entries in the model to general a plausible response.

This is neither natural language nor intelligence. 

Nonetheless, AI-based Chatbots seem to many users to be smart. In fact, some AI research scientists have claimed Chat GPT-4 and its ilk show early sparks of artificial general intelligence (AGI).¹³

However, I think there is another factor entirely that explains this.

You will recall that the Turning Test¹⁴ mentioned in Part 1 states that if a machine could successfully convince a knowledgeable human observer of its intelligence, then it should be considered to be intelligent. 

One way to pass the Turing Test would be to build genuinely intelligent computers that function like human intelligence. Another way would be to build programs that can convince humans by tricking them in ways they don’t even understand. 

It seems that Turing overestimated humans. Convincing humans that all sorts of (non-human) things are not only smart, but alive, is alarmingly easy. 

I explore this in part 4 of the series, but first I'm going to take an interlude (for Part 3) to enjoy the many wonderful cartoon on AI. (If you don't follow SMBC yet, why not??)

Footnotes

1. If you're not sure what the computation model is, read more here: https://com-cog-book.github.io/com-cog-book/features/intro-com-mod-cog.html  A sample: [After] the advent of theoretical computer science and digital computers in the 20th century, computational psychology rose as a field. Amid the creation of the digital computer, cognitive psychologist developed mathematical formalisms and computational models describing cognition as an information processing phenomena, in close resemblance to how digital computers work. Researchers ... applied computational methods to the study of perception, language, and problem-solving, effectively establishing the field with their collective efforts. The rise of computational psychology offered an alternative approach to the study of the mind, one based on algorithms and computer simulations, rather than on correlations and laboratory-based experimentation, the dominant paradigms at the time.
2. This is, of course, a broad generalisation and there are plenty of exceptions. Politicians' blither comes to mind as an example of people who talk a lot but may not have much to say!! But bear with me.
3. As I said in Part 1, I won’t go into the application of this heuristic to non-human animals – it’s much too big a topic – other than to say we are pretty good creating definitions that make humans look good or more important than any other form of life. 
4. 'Trained' is a metaphor for the human activity, and when used about machines, trained actually means 'programmed'. The use of words like trained (and learn, think, read, decide, reward, feedback, etc.) implies some agency in the machine that is just not there. There are so many of these analogies or euphemisms; I begin to suspect they hope that by referring to a machine in human intelligence related terms it might just become intelligent! 
5. Actually theft is a better word here too! The vast language and visual information used in the programming of AI was taken without knowledge or consent. Here's just one recent example: https://www.abc.net.au/news/2024-06-10/instagram-facebook-train-meta-ai-tools-no-opt-out-australia/103958308  
6. Parameters refer to the details of programming, the elements that make up the algorithms guiding word choice etc. One such parameter is 'temperature', which refers to the degree of risk the algorithm allows in generating the language responses, from conservative to 'out there'.   
7. This is another analogy to brain physiology – but a 'neural network' does not have neurons and not a network - but it sounds good to the human audience I think. In fact, the closer this all sounds to an animal brain, the closer the conviction that 'intelligence' is just around the corner, I guess. 
8. To explain, this from Wikipedia https://en.wikipedia.org/wiki/ChatGPT: More recently, new versions have had supervised learning and reinforcement learning from trainers’/programmers’ feedback. The fine-tuning process leveraged supervised learning and reinforcement learning from human feedback. Both approaches employed human trainers to improve model performance. In the case of supervised learning, the trainers played both sides: the user and the AI assistant. In the reinforcement learning stage, human trainers first ranked responses that the model had created in a previous conversation. These rankings were used to create "reward models" that were used to fine-tune the model further by using several iterations of proximal policy optimization.
9. This model is quite old, from Lahey in 1978, and there are innumerable models from which to choose. However, the simplicity of this model is helpful to explain some very basic concepts. And I think in AI, we need to get back to some basics.
10. Personal agency refers to an individual's ability to control their own behaviours and reactions to circumstances beyond their control, even if their actions are limited by someone or something else.
11. I'm using 'parrot' in the meaning of mimic, as used by the quote source. I need to add that living parrots are smart! The corvids (ravens, crows, jays, magpies, etc.) and psittacines (parrots, macaws, and cockatoos) are often considered the most intelligent birds, and are among the most intelligent animals in general. 
12. Formal language competence is amazing enough: contemporary LLMs are fantastic models of formal linguistic skills for which there are many applications. Let's just not go overboard with how we interpret it!
13. An important distinction is necessary between the two categories of AI: artificial narrow intelligence (ANI) and artificial general intelligence (AGI). Read more in part 1. 
14. Pioneering computer scientist, Alan Turing, proposed that if a machine could successfully convince a knowledgeable human observer of its intelligence, then it should be considered to be intelligent. 

Images

• Where not otherwise identified images were snipped from social media [fair dealing]
• Components of language models version 1 and 2 by the author
• Stochastic parrot and Plane laying an egg created by the author's friend using AI [who knows what the (c) should be!]
• Matteo Wong quote created by the author using text from his article in The Atlantic: The Difference Between Speaking and Thinking, January 31, 2023  
• Humanness by Saturday Morning Breakfast Comic (SMBC) https://www.smbc-comics.com/comic/humanness [used under terms]