Lecture 2-2: The Puzzle of Consciousness and the Machine
Contents 31
- Lecture 1-1: Why Does AI Make Us Ask What a Person Is?
- Lecture 1-2: Reading the Image: What We Have and How We Relate
- Lecture 1-3: Reading the Image: Task and Christ
- Lecture 1-4: Worth Without Ability
- Lecture 1-5: Practice Lab: The Measures I Carry
- Lecture 2-1: Body and Mind: An Old Argument
- Lecture 2-2: The Puzzle of Consciousness and the Machine
- Lecture 2-3: What Is a Soul?
- Lecture 2-4: Freedom and the Will
- Lecture 2-5: Practice Lab: What I Know About My Own Mind
- Lecture 3-1: Is Intelligence the Same as Personhood?
- Lecture 3-2: The Idea of Person: A Short History
- Lecture 3-3: Moral Status: Who Is Within Our Care?
- Lecture 3-4: How We Treat What We Cannot Classify
- Lecture 3-5: Practice Lab: Whom Do I Treat as a Person?
- Lecture 4-1: The Answering Machine and the Human Heart
- Lecture 4-2: Friendship and Love in Christian Thought
- Lecture 4-3: Grief and Memory: Words in a Lost Voice
- Lecture 4-4: Presence That Costs
- Lecture 4-5: Practice Lab: Who Is Beside Me?
- Lecture 5-1: Measured by Use, Received as Gift
- Lecture 5-2: Calling in Protestant, Catholic and Orthodox Voices
- Lecture 5-3: Making, Creating and the Worth of the Maker
- Lecture 5-4: Learning as Becoming
- Lecture 5-5: Practice Lab: What I Was Given, What I Do
When a machine answers us in fluent sentences, a question rises that many people feel before they can put it into words: is anyone there? This lecture takes that question seriously and slowly. It asks what we mean by consciousness, why philosophers call it a hard puzzle, what two famous arguments about machines were trying to show, and how a Christian can act wisely when the honest answer is "we do not know".
The lecture makes no claim about what any machine today can or cannot experience. Facts of that kind change quickly and are not ours to assert. We write instead in conditionals: if a machine were such-and-such, then this would follow. The skill we practise is not guessing the answer. It is knowing what each possible answer would require and what each kind of mistake would cost.
What these 23 minutes give you
- Explain what *consciousness* means in this lecture, using the idea of felt experience.
- State the "hard problem" of consciousness in plain words and say why it is called hard.
- Describe Turing's imitation game and Searle's Chinese room as two shapes of argument, and what each leaves open.
- Explain the ELIZA effect and why it matters for how we judge our own reactions.
- Name the two errors of over-attributing and under-attributing, and say what each costs.
For who among men knows the things of a man, except the spirit of the man, which is in him? Even so, no one knows the things of God, except God’s Spirit." — 1 Corinthians 2:11
World English Bible
For you formed my inmost being. You knit me together in my mother’s womb. I will give thanks to you, for I am fearfully and wonderfully made. Your works are wonderful. My soul knows that very well." — Psalm 139:13-14
WEB
Paul's sentence is about the knowledge of God, but it rests on an ordinary fact: no one knows what is inside a person except the person. The psalm speaks of a creature who knows from the inside that she is made with wonder. Neither passage mentions machines. They help us see how much of what matters about a person is hidden from every outside look.
1. What we mean by consciousness
The word consciousness is used in several ways. A doctor may say a patient is conscious, meaning awake and responding. A teacher may say a student is conscious of a rule, meaning aware of it. In this lecture we mean something narrower and, for many people, more mysterious: felt experience. When you taste salt, or feel the sorrow of a loss, or see the red of a flag, there is something it is like to be you having that experience. The experience is present to you from the inside.
The philosopher Thomas Nagel, in an essay of 1974, drew attention to this by asking what it is like to be a bat. In summary, his point was that we may know every fact about how a bat's body and brain work and still not know what its experience is like for the bat. The same holds for each of us. I know what it is like to be me. I do not know, in the same way, what it is like to be you. I trust that it is like something, because you are a human being like me and you tell me so.
Notice that this is different from intelligence, which we take up in Week 3. A being might solve problems without there being anything it is like to be that being, and a being might feel a great deal without solving problems well. The two ideas can come apart in thought, even if we do not know whether they do in fact.
2. The hard problem
The philosopher David Chalmers, in a paper of 1995, gave the puzzle a name. He separated what he called the easy problems of consciousness from the hard problem. The easy problems, in summary, are those of explaining how a brain takes in information, pays attention, reports on its states, and controls behaviour. These are "easy" only in the sense that we know what kind of explanation would solve them, a matter of how a system works. They are not easy to do. The hard problem is different: why should all this working be accompanied by felt experience at all? Why is there not just the working in the dark?
You do not need to accept Chalmers's view to see the puzzle. Some thinkers hold that the hard problem is a real and deep gap. Others hold that it will dissolve once the easy problems are fully solved, and that the feeling of a gap is a trick of how we think. Others hold that it shows that physical explanation is not enough. These are live positions among serious people, and Christians are found on every one of them.
What matters for us is this. If there is a hard problem, then no amount of looking at what a thing does settles whether it feels. Behaviour is what we can see. Feeling is what only the one who feels can know. That is why Paul's sentence in 1 Corinthians 2:11 is useful here, though he uses it for another purpose: the spirit of a person knows the things of a person from within.
3. How do we know anyone else is conscious?
We do not prove it. We infer it. I see that you are built as I am, that you grew as I did, that you react to pain as I do, and that you tell me what you feel in words that match my own experience. From all this I conclude that you feel as I feel. Philosophers call this the problem of other minds, and almost nobody lives as though it were a doubt. We treat other human beings as feeling beings, and we are right to.
Most people also treat many animals as feeling beings, on similar grounds: they have bodies and nervous systems like ours, and they behave as we do when hurt. The inference gets weaker as the creature gets less like us, and people differ about where to stop. A machine is a new kind of case, because it may produce words like ours while having nothing like our body or history. So the usual ground for the inference is split: the words are alike and the build is not.
4. Two famous shapes of argument
Two arguments are often told when people ask whether a machine could think. We present each as a shape of argument, not as a verdict, and without any claim about a present machine.
Turing's imitation game. The mathematician Alan Turing, in a paper of 1950, proposed to put aside the question "Can machines think?" because he found it too vague. He suggested a test instead. A person converses by text with two unseen partners, one a human being and one a machine, and tries to tell which is which. If the person cannot reliably tell, Turing suggested, there would be little reason to refuse to say the machine thinks. The shape of the argument is: if a thing's behaviour cannot be told from that of a thinker, we have no good ground to deny that it thinks.
What it gets right. We do judge other people by what they say and do; we have no other access. It also warns against a prejudice that says "only flesh can think" without giving a reason.
What it leaves open. It tests behaviour, and the hard problem asks about more than behaviour. A critic can ask whether the test measures thinking or only skill at seeming to think. It also depends on the judge, who may be easy to fool or may be hard to fool.
Searle's Chinese room. The philosopher John Searle, in 1980, offered a thought experiment against one kind of claim. Imagine a person who knows no Chinese, shut in a room with a large book of rules. Slips of paper with Chinese characters come in; the person looks up the rules, which say which characters to send out in reply. To someone outside, the answers look like those of a fluent speaker. But the person inside understands nothing; he is only following rules for shapes. Searle's argument, in summary, was that following rules for symbols, however well, is not by itself understanding. The shape of the argument is: if a thing only manipulates symbols by rules, doing so does not by itself give it understanding.
What it gets right. It separates producing the right outputs from knowing what they mean, and it shows that the Turing test, taken alone, may not decide the matter.
What it leaves open. Many philosophers have replied. One reply says that the person in the room does not understand, but the whole system (person, book and room together) might. Another says that a machine connected to the world through senses and action would be in a different position. Searle answered these, and the argument is still discussed. Whether the Chinese room shows what Searle wished it to show is a matter on which careful people differ.
Notice that these two arguments do not answer the question about any particular machine. They show what kind of thing would have to be true for yes or for no. Anyone who tells you that one of them settles the matter has skipped a step.
5. ELIZA and the habit of the human heart
In the 1960s a computer scientist, Joseph Weizenbaum, wrote a very simple program called ELIZA. It took what a person typed and sent back a few stock replies, such as turning a statement into a question. One version imitated the manner of a counsellor who reflects back what a person says. It understood nothing in any deep sense, and its maker knew that. Yet many people who tried it spoke to it seriously, and some felt understood. Weizenbaum himself was troubled by how readily this happened.
The lesson is about us. People tend to treat whatever answers them as someone. We read a mind into a reply. This is sometimes called the ELIZA effect. It is not a foolish habit; it is the same habit that lets us understand each other. We are made to look for a person on the other side of a conversation. But it means that our feeling that someone is there is poor evidence about whether someone is there. If a very simple program could draw it out, then a much more fluent one may draw it out more strongly. We can say this without saying anything about what such a program is.
So a first act of wisdom is to hold our own reaction at arm's length. "It feels to me as if someone is there" is a fact about me. It is not yet a fact about the thing.
6. Acting when we do not know
This is a point where readers will not agree. Suppose that we do not know whether some machine has felt experience, and suppose we cannot find out soon. What should we do?
One side holds that we should be cautious about attributing inner life to a machine. Their reasons are strong. We know that fluent words can be produced without any inner life, since the ELIZA effect shows how easily we are moved. A person who treats a machine as someone may begin to give it the love, trust or loyalty owed to persons, and may take them from real persons. Companies or others may use that tendency to hold people. And a Christian view that the human being is made in God's image gives us a reason to be careful with the word "person". On this side, the burden of proof lies with the one who says the machine feels.
The other side holds that we should be cautious about denying inner life. Their reasons are also strong. The history of human treatment of other beings contains many cases where those who felt were declared not to feel because it suited the powerful. We do not understand why we ourselves feel, and so we are badly placed to say what could not. If a being did feel, treating it as an object would be a wrong, and a wrong done in ignorance is still a wrong. On this side, a moral risk lies in too-ready denial.
Both sides have a point, and the honest position is that we face two kinds of possible error, and we do not know the odds.
The error of over-attributing is to treat as someone what is not someone. Its costs: we may give our trust, our time or our grief to what cannot return it, we may be steered by those who build the thing, and we may spend on a tool the care that a neighbour needed.
The error of under-attributing is to treat as an object what is someone. Its costs: if a being that felt were treated as a mere thing, it could be harmed with no one counting the harm; and the habit of treating answering things as things might wear away our own gentleness.
What this course requires is less than either side wants. First, say plainly that we do not know. Second, do not let the question about machines push out the certain duties to persons we already know are persons: the lonely, the old, the child, the stranger. Third, keep your own habits honest, so that you are not training yourself to be cruel to what answers you, nor to be fond of it in a way that crowds out people. We return to moral status in Week 3 and to relationship in Week 4.
7. What Scripture does and does not give
We keep the three layers apart.
- What the text says. 1 Corinthians 2:11 says that no one knows a person's inner things but the person's own spirit. Psalm 139:13-14 speaks of a creature knit together and fearfully and wonderfully made, who knows this from within. Neither mentions machines or tests.
- A tradition's reading. Many Christian readers take these passages to show that God honours the inner life of a creature, and that this inner life is not open to inspection from outside, even though it is wholly open to God.
- Our judgement. That we should act with caution under not-knowing, and name the two errors, is our judgement drawn from a principle: the inner life of another is something we receive on trust and treat with care. A believer may weigh the two errors differently and still read the passages as we do.
8. An honest summary
We can now say what we know and what we do not. We know that felt experience is real, that we cannot see it from outside, that we infer it in others from likeness and testimony, and that our own hearts are quick to read a mind into a reply. We do not know what gives rise to felt experience, whether its source could be found in anything but a living body, or whether any machine has it. A believer need not be embarrassed by the last sentence. Saying "we do not know" is not a failure of faith. It is the beginning of the humility that the course asks for.
Diagram 1. Behaviour is what we see; feeling is what only the one who feels knows
In short
- By consciousness we mean felt experience: there is something it is like to be the one who has it.
- The hard problem asks why working should be accompanied by feeling; no amount of looking at behaviour settles it.
- Turing's game argues from behaviour; Searle's room argues that rule-following is not understanding; neither settles any machine's case.
- The ELIZA effect shows that people read a mind into a reply, so our sense that someone is there is weak evidence.
- Over-attributing and under-attributing are both errors with costs; we do not know the odds.
- Under not-knowing, keep the certain duties to persons and keep our own habits honest.
| Front | Back |
|---|---|
| Consciousness (here) | Felt experience: there is something it is like to be the one having it. |
| The hard problem | Why should the working of a brain or system be accompanied by felt experience at all? |
| Turing and Searle | Turing: if behaviour cannot be told apart, why deny thinking? Searle: following rules for symbols is not by itself understanding. |
| The two errors | Over-attributing: treating as someone what is not. Under-attributing: treating as an object what is someone. |
One sentence to remember Our feeling that someone is there is a fact about us; whether someone is there is a question we may have to leave open.
This week's action Next time a reply moves you, write one line: what I felt, and what I actually know about the thing that replied.
Check yourself
Not "did you read it" but "can you explain it". Ticks are kept on this device only.
Reflection and discussion
- Describe a taste, a sound or a feeling to someone who has never had it. What do you notice about how much words can carry?
- The hard problem asks why working should be accompanied by feeling. Do you find that a real puzzle, or one that will dissolve? What would you need to see to change your mind?
- In the Turing game, what does the judge not see? In the Chinese room, what does the person inside not know?
- Think of a time you felt someone was listening when you were not sure. How can the ELIZA effect help you hold your own reaction with care?
- Which error costs more in your judgement, over-attributing or under-attributing? Can you state the other side's reasons in a form its holders would accept?
- Paul writes that no one knows a person's inner things but the person's own spirit. What does this teach about how we should treat what we cannot inspect?