Lecture 9-1: The Ethics of AI in Diagnosis and Treatment

Mark 2:17 — "When Jesus heard it, he said to them, 'Those who are healthy have no need for a physician, but those who are sick. I came not to call the righteous, but sinners to repentance.'" (World English Bible)

Mark 1:41 — "Being moved with compassion, he stretched out his hand, and touched him, and said to him, 'I want to. Be made clean.'" (WEB)

3.1 Three applications, three different questions

Medical AI is not one technology; it is a family of systems doing different work, and lumping them together produces sloppy ethics. Three applications are worth separating.

Diagnostic imaging. Systems trained on large sets of labeled medical images — chest X-rays, mammograms, retinal scans, skin lesion photographs — learn to recognize patterns statistically associated with disease. Some of these systems now match or exceed average human radiologist or dermatologist performance on narrow, well-defined tasks (for instance, detecting diabetic retinopathy from a retinal photograph, a task with a large, well-labeled dataset and a relatively unambiguous ground truth). Their strength is exactly their narrowness: consistent, tireless pattern recognition on a single, well-specified visual task, at a volume and speed no human reader could sustain.

Triage algorithms. These systems take in a patient's vital signs, presenting complaint, and sometimes historical data, and output a predicted acuity score used to sequence who is seen first in an emergency department, or which patients should be flagged for closer monitoring on a general ward. Their ethical profile differs from diagnostic imaging because their output is not a finding about a static image but a prediction about what will happen to a person, used to allocate a scarce resource — clinical attention — under time pressure.

Treatment-recommendation systems. These compare a patient's clinical profile against large outcomes datasets to suggest which treatment, of several plausible options, has statistically produced the best results for patients "like this one." Their ethical profile differs again because their output shapes not just attention but decision — which drug, which dose, which surgical approach — often for a patient a specific physician will make herself accountable for.

Each application uses a genuinely related technology (statistical pattern-matching on large datasets) for a genuinely different purpose, and this lecture keeps them distinct rather than treating "medical AI" as a single moral object.

3.2 Who is accountable when the recommendation is wrong?

Every one of these systems will sometimes be wrong, because every one of them is fundamentally probabilistic: it reports what has been statistically associated with an outcome in its training data, not a certainty about this patient. When a diagnostic-imaging system misses a malignancy it was not trained to recognize well, or a triage algorithm underestimates a deteriorating patient's acuity, or a treatment-recommendation system suggests a regimen that turns out badly for an atypical patient, someone has been harmed, and the question of who bears responsibility for that harm does not have a settled answer.

The physician who relied on the system's output can reasonably say she exercised her own judgment in a system-assisted environment, as she is trained and licensed to do, and that no reasonable physician could independently verify every statistical claim behind a tool approved for clinical use. The hospital that deployed the system can reasonably say it purchased and implemented a product cleared by the relevant regulatory body, following the vendor's guidance for use. The vendor that built the system can reasonably say the tool was validated on the population and use case it was designed for, and that the harm arose from a use, a population, or a clinical context the tool was never claimed to cover. Each of these claims can be true simultaneously, and the patient who was harmed is left facing a diffusion of responsibility across three parties, none of whom, taken individually, clearly failed.

This is not a reason to reject medical AI; every complex system involving multiple actors — a hospital, an equipment manufacturer, a pharmaceutical supplier — has always had some version of this problem. But it is a reason to insist, as an ethical minimum, that accountability be worked out explicitly and in advance rather than discovered after a harm has occurred: who reviews an AI-flagged result before it reaches a patient, who is responsible for monitoring the system's real-world performance after deployment (since a system's accuracy on its original training data does not guarantee its accuracy on the population actually being treated, a problem called "distribution shift"), and what recourse a harmed patient has. A hospital that adopts a diagnostic tool without answering these questions in advance has adopted a convenience without adopting the responsibility that should travel with it.

3.3 The danger of measurement narrowing

A second and less discussed risk is subtler than any single wrong output: a system's presence can quietly reshape what a physician attends to, simply by making some things easy to quantify and others hard. If a triage algorithm scores acuity from vital signs, presenting complaint, and structured historical data, a physician working alongside it can begin — not through any deliberate choice, but through the ordinary pull of a tool that makes some information salient and other information effortful to notice — to attend most closely to what the algorithm attends to, and less closely to what does not fit its inputs: a patient's affect, the story behind a vague complaint, a family member's worry that does not translate into a data field. Nothing in the technology itself forbids attending to these things. But a tool that makes one kind of attention effortless and another kind of attention require active resistance to a screen's suggestion is not ethically neutral about which kind of attention survives in practice.

This concern has a name in the broader literature on decision-support systems — "automation bias," a documented tendency for people using an automated aid to over-trust its output and under-scrutinize cases where their own independent judgment would have caught an error the system missed. Medical AI is not exempt from this general finding about human-automation interaction; if anything, the prestige and apparent objectivity of a data-driven diagnostic tool make automation bias more, not less, likely in a clinical setting, where deference to a "second opinion" that presents itself as more thoroughly informed than any single physician can be exactly the wrong instinct in the specific case where the physician's non-quantifiable observation was the correct one.

3.4 Mark's benchmark: compassionate and relational healing

Mark's Gospel gives two brief, closely related pictures of what healing looks like at its source, and this course treats both as the standard against which any medical technology — including this one — should be measured.

Mark 2:17 records Jesus's answer to the religious leaders' objection that he ate with "tax collectors and sinners": "Those who are healthy have no need for a physician, but those who are sick. I came not to call the righteous, but sinners to repentance" (WEB). The image Jesus reaches for to explain his own mission is deliberately medical: a physician goes toward the sick, not away from them, and the going-toward is the whole point of the image. Whatever else this verse teaches about Jesus's mission to sinners, it assumes, as background, an unremarkable and positive picture of what a good physician does: moves toward suffering rather than avoiding it.

Mark 1:40-41 makes the same point more vividly, in an actual healing narrative rather than a metaphor. A man with a skin disease approaches Jesus and asks to be made clean — a request carrying enormous social weight, since such a condition in the ancient world typically meant exclusion from ordinary community life. Mark reports Jesus's response with unusual physical specificity: "Being moved with compassion, he stretched out his hand, and touched him, and said to him, 'I want to. Be made clean'" (WEB). The touch is not incidental to the healing; in the social world of the text, touching a person with this condition was itself a boundary-crossing act, one Jesus performs before the healing word is even spoken. Compassion here is not a private feeling accompanying an otherwise sufficient technical act (the healing word); it is enacted, bodily, prior to and alongside the act, in a gesture the text records for its own sake.

Taken together, these two texts do not condemn medical technology or expertise — the physician image in 2:17 is used approvingly, and nothing in the healing of 1:41 suggests skill or means are suspect. What they do is name compassionate, embodied, relational presence as constitutive of biblical healing, not decorative to it. A diagnostic tool that improves accuracy while leaving a physician's attention, presence, and willingness to move toward a patient's suffering fully intact serves this pattern well. A workflow that lets a statistical output substitute for that attention — that treats an accurate flag as though it discharged the physician's actual work rather than assisting it — has not thereby become evil, but it has drifted from what this course's benchmark names as the point of healing in the first place. The test this lecture proposes is not "is the tool accurate?" — accuracy is necessary but was never sufficient even for a merely human physician — but "does this tool serve, or does it quietly displace, the compassionate and relational shape Mark 1:41 gives to the very idea of healing?"