Lecture 1-2: The Concept of AGI and Its Controversies

3.1 What "AGI" is usually claimed to mean

Chapter 1's first lecture defined artificial general intelligence provisionally as a system capable of the same flexible, transferable reasoning across essentially any intellectual task that a human being can perform. That working definition is useful for keeping narrow AI and AGI apart, but it understates how much is actually disputed once the conversation turns to AGI specifically. Researchers disagree about whether AGI requires human-like reasoning or merely human-level performance regardless of mechanism; whether it requires embodiment, or grounding in a physical world, or whether pure text and code are sufficient; whether it is a threshold crossed suddenly or a spectrum climbed gradually; and whether "general" should be read as "as general as a human" or "general across a much wider range of formally specifiable tasks than current systems, without matching every human capacity." None of these are merely terminological quibbles. Each version implies a different research program and a different answer to "how would we know if we had built it."

3.2 The accelerationist case: Kurzweil and exponential progress

Ray Kurzweil, an inventor and futurist whose predictions about AI have been influential since the 1990s, has argued for decades that technological progress in general, and computational capability in particular, follows a predictable exponential curve rather than a linear one — a claim he calls the "law of accelerating returns." Applied to AI specifically, in works including The Singularity Is Near (2005), Kurzweil's argument runs roughly as follows: the exponential growth already observed in computing power, and more recently in the scale of AI training runs, will continue; capability tends to track available compute and data fairly closely across a wide range of tasks; and extrapolating that trend forward yields a reasonably confident, and comparatively near, date by which machine intelligence matches and then exceeds human intelligence across essentially every domain.

The case deserves to be stated in its strongest form, because it is easy to dismiss as mere enthusiasm and it is not merely that. Kurzweil has made a number of specific predictions about computing milestones that were later verified as substantially correct, and the trend of rapidly increasing capability from rapidly increasing scale — visible across the deep-learning turn and the transformer era described in lecture 1-1 — is a real, documented pattern, not a rhetorical flourish. The accelerationist case is an argument from an observed trend, not from wishful thinking, and a fair hearing has to grant that much.

3.3 Two distinct skeptical arguments

It would be a mistake to treat every skeptic of near-term AGI as making the same argument, because two importantly different objections travel under the same general heading of "AI skepticism," and conflating them weakens both.

The architectural objection, associated with Gary Marcus and introduced in lecture 1-1, holds that current deep-learning architectures, however much they are scaled, are the wrong kind of system to produce general intelligence — that something structurally different, likely involving explicit reasoning and world-modeling capacities current systems lack, is required, and that scaling alone extrapolates a curve that will plateau or bend rather than continue indefinitely. This is a claim about architecture and mechanism: the current road, however far it is driven, does not lead to the claimed destination.

The "stochastic parrots" critique, from a widely cited 2021 paper by Emily Bender, Timnit Gebru, and co-authors (Margaret Mitchell and Angelina McMillan-Major), is a different kind of objection. Their argument is not primarily about whether scaling could in principle produce something with general capability; it is about what a large language model, trained the way current systems are trained, actually is: a system that has learned to produce statistically plausible sequences of linguistic form, without any of those sequences being grounded in reference to meaning, communicative intent, or a model of the world the words are about. Their paper's title supplies the memorable image — the models are, on this account, "stochastic parrots," reproducing the patterned sound of language without producing, in any meaningful sense, the thing language is for. Where Marcus's objection is about the architecture's future trajectory, Bender and Gebru's is about a category error already present in how such systems are described now — calling next-token prediction "understanding" or "intelligence" smuggles in a claim the mechanism does not support, regardless of what future scaling might or might not accomplish.

Both objections counsel real caution about near-term or even eventual AGI claims. Neither one is Kurzweil's argument answered on its own terms; each targets a different premise in the accelerationist case, and a careful reader should be able to say which premise each one targets rather than treating "the skeptics" as a single undifferentiated position.

3.4 Why the disagreement runs deeper than timelines

It is tempting to summarize this dispute as merely "some people think AGI is coming soon, others think it will take longer." That summary loses what is actually at stake. Kurzweil's accelerationist case and Marcus's architectural objection disagree about a technical question that could, in principle, be settled by future evidence: does scaling the current approach continue to yield proportional gains, or does it plateau? Bender and Gebru's critique disagrees about something else entirely — not a prediction about the future, but a claim about how we should describe what current systems are already doing, and a warning about the rhetorical and ethical consequences of over-describing them. A reader who treats all three as competing forecasts of the same clock has missed that one of the three is not making a forecast at all.

This matters for a course like this one because theological chapters later in the syllabus will need to know which kind of claim they are responding to. A chapter examining whether AGI would threaten the image of God (chapter 2) is responding, implicitly, to something like Kurzweil's forecast — it assumes, for the sake of argument, that a sufficiently capable general system might eventually exist, and asks what would follow theologically if it did. A chapter examining whether current systems already understand anything (chapter 3, and again chapter 16) is responding to something much closer to Bender and Gebru's present-tense claim. Confusing the two kinds of question produces theology built to answer a question nobody was actually asking.

3.5 Counting the cost of an uncertain future

Luke 14:28 belongs to a short parable Jesus tells about a man who begins building a tower without first calculating whether he can finish it, and is mocked for the incomplete structure left standing as a monument to poor planning. The application to AGI discourse is not that a church should refuse to engage a technology whose ultimate trajectory is uncertain — that would misuse the text, which commends counting the cost before building, not refusing to build. The application is narrower and more exacting: institutions, companies, and governments are currently making very large financial, regulatory, and even civilizational commitments premised on confident predictions about AGI's arrival and character, predictions this lecture has just shown are genuinely contested among the people best positioned to make them. Luke 14:28's wisdom is procedural rather than substantive — it does not tell a reader whether Kurzweil or Marcus is right, but it does insist that a wise builder reckons honestly with uncertainty before committing resources that cannot be recovered, rather than proceeding as though the forecast were certain because acting as if it were certain is more convenient.

James 4:14 presses a related but distinct point. The passage it belongs to rebukes merchants who speak confidently of their plans for "today or tomorrow" as though the future were simply theirs to schedule, and James's corrective is not that planning is wrong but that planning without acknowledging genuine uncertainty is a form of presumption: "you don't know what your life will be like tomorrow." Applied here, the text is a caution against a specific rhetorical habit visible on every side of the AGI debate — speaking of "when AGI arrives" with the same grammatical confidence one would use for "when the sun rises tomorrow," when the honest epistemic state, on the evidence surveyed in this lecture, is closer to "if, and we genuinely do not know when, or in what form." A Christian is not asked by this text to have no opinion about AGI's likelihood. The text asks that whatever opinion is held be held with the humility appropriate to a vapor that "appears for a little time, and then vanishes away" — including the vapor of a civilization's confident five-year forecast.

3.6 What this lecture does not settle

In keeping with this course's practice of naming disagreement rather than resolving it by fiat, this lecture takes no position on whether Kurzweil, Marcus, or Bender and Gebru have the better of the technical argument, because that argument will be settled, if it is settled, by future evidence in AI research, not by a theology course. What this lecture does argue, and will defend as a controlling principle for the rest of the course, is this: any theological argument built on top of a confident AGI timeline should be read with the same caution Luke 14:28 and James 4:14 recommend for the timeline itself. Chapter 17, on the technological singularity, will return to this same caution at greater length and apply it to explicitly eschatological claims.

Summary table