The AI Privacy Panic Is Not a Risk Assessment

Every few weeks someone asks me, in a lowered voice, whether it is safe to put their draft or student paper into ChatGPT. The tone recalls the way people once spoke about typing a credit card number into a website. I understand the instinct. I also think it has drifted far from the evidence.

Some time ago I wrote a brief that reviewed documented security incidents at OpenAI, Google, Anthropic, Meta, and Microsoft. The list was real but modest. In 2023 a bug showed some ChatGPT users other people’s chat titles and partial billing details. Search engines indexed shared Bard conversations. A contractor sent an email to the wrong person. I recently went back to see whether the record has changed. It has grown, though not in the way the anxiety predicts. The major platforms still have not suffered a breach that dumped user conversations from their own servers. Their worst episodes involved the share button. In 2025, over 4,500 shared ChatGPT conversations were indexed by Google, including some that contained highly personal or sensitive information, and Google alone had indexed more than 370,000 Grok conversations. Meta’s new AI app placed private prompts, including medical details, legal questions, and even personal confessions in a public feed. In July 2026, 404 Media reported that many people’s Claude chats and creations were exposed on Google. Each case began with a person pressing a button that poorly explained what it did. Compare that with Equifax, which in 2017 gave up the Social Security numbers of 148 million people through an unpatched web server. Equifax, for the record, still reports quarterly revenue in the billions, and none of us stopped applying for credit.

The truly large leaks happened elsewhere, among the many smaller AI businesses. Chat & Ask AI, a popular chatbot wrapper app with 50+ million users, had a misconfigured Firebase backend that exposed 300 million messages from over 25 million users. DeepSeek left a database accessible without any authentication, with chat histories and secret keys inside. The vendor behind McDonald’s hiring chatbot was breached by researchers who simply tried the most basic username and password possible, “123456”. Another set of incidents starts on the user’s own computer. In August 2026 Anthropic reported that infostealer malware had compromised their computers and siphoned off active login session cookies. What these cases share is old-fashioned negligence. A careless company is careless with whatever it holds, and an infected laptop leaks everything on it. 

The updated record makes my second point stronger. Security vendors and enterprise sales teams have every reason to describe consumer tools as sieves and institutional licenses as vaults. Universities now pay handsomely for ChatGPT Edu and its cousins. I still find no public evidence that these tiers resist breaches better than the ordinary product from the same company. They run the same models, built by the same engineers. What the institution mostly buys is a contract, an administrative console, and training switched off by default. The contract does matter for legal retention. In 2025, a court ordered OpenAI to preserve consumer chats, and Enterprise, Edu, and ZDR customers are excluded. For security, the line that matters runs between a major provider and an obscure app that resells its models. It does not run between two price tiers of the same service.

The deeper confusion concerns training. Many people imagine that if a model trains on their conversation, the document now sits somewhere inside it, waiting for a stranger to ask the right question. Training does not work that way. A model absorbs statistical regularities across billions of texts. Picture a reader who has gone through ten million novels. She can tell you how mysteries tend to end, but she cannot recite page 212 of the one you lent her last spring. Researchers have shown that models occasionally reproduce memorized fragments, mostly text repeated many times across the web, and extracting them takes deliberate effort. A grant proposal pasted once is a drop in a very large sea.

One honest caveat remains. A genuinely novel idea, a conceptual move nobody has yet published, could in principle seep into the general current, softened and unattributed. For scholars this is not trivial. The remedy is almost comically simple. ChatGPT, Claude, and Gemini each offer a setting that keeps your conversations out of training. It takes less time to find than this paragraph took to read. 

Beyond that, the risks resemble those of every other online platform we already trust with our working lives. We keep grades in the learning management system, manuscripts in cloud drives, and personnel matters in email, all without rituals of dread. The same ordinary care applies here. Do not paste Social Security numbers. Keep student records out where policy forbids them. Turn the training switch off. Look twice before pressing share, and prefer the major providers to the anonymous apps that wrap them. Prudence costs a few clicks. Fear, as the vendors understand, costs considerably more.

The Toupee Fallacy Comes for AI-assisted Writing

Sometimes a journal editor tells me, with the serenity of a seasoned customs officer, that they can always recognize a manuscript written with the help of AI. Colleagues say the same over coffee about student essays. The tells are recited like a liturgy: the word “delve,” adjectives marching in threes, the paragraph that ends on a note of cautious uplift, a voice sanded down until no grain remains. The short sentences for emphasis in a trope "it is not this. Is it not that. It is something else." Reviewers mourn a flattened cadence, polish laid over a hollow. I do not doubt that these readers have met such texts. I doubt the verdict they pronounce afterward.

Skeptics have a name for this error: the toupee fallacy. Its canonical form runs, all toupees look fake, because I have never seen one that did not. The flaw surfaces the moment the sentence is spoken aloud. A toupee that succeeds is a toupee you never noticed. The speaker has inspected only the failures and from them has drawn a general theory of hairpieces. The term’s lineage is murky. It appeared on the Skeptics’ Guide to the Universe podcast in August 2006 as a “name that fallacy” puzzle, and at least one catalog of fallacies credits Rebecca Watson, the founder of Skepchick, with coining it. Natalie Reed carried it further in 2012, using it to dismantle the claim that trans women are always discernible in public. The deeper structure belongs to Nassim Taleb, who called the missing cases “silent evidence,” and before him to Abraham Wald, who told the Navy to armor the unscarred parts of returning bombers, since the planes struck there never came home.

Transpose the structure onto scholarly writing. An editor who claims to detect AI assistance has detected something narrower: AI assistance done lazily. The author who pasted an abstract into a model and submitted the output unread. The literature review with “Certainly! Here is” still clinging to its first line. These manuscripts betray themselves, and the betrayal is real. But the leap from them to a general claim about AI-assisted prose is the leap from bad toupees to all toupees. The author who drafted with a model, then pruned the triplets, restored her own rhythm, replaced generic citations with ones she had actually read, and reread the whole thing twice, produced a paper the editor accepted without a flicker of suspicion. The editor did not count her; she is the silent evidence.

My own position spares me the dilemma. I urge students to use AI freely, so detection is a game I never have to play. The claims I hear come from people whose work is gatekeeping, and gatekeepers are especially prone to this bias, because the only cases that reach their attention are the ones that trip the wire. Describe to them the competent, undetectable submissions and the reply arrives on cue: those authors must not have used AI much. That reply is the fallacy tending its own garden, reclassifying the disconfirming cases so they cannot be counted.

Most complaints about AI prose, examined closely, are complaints about mediocre human prose that editors have been rejecting for a century; what changed is volume. A model produces the median text of its training data effortlessly, and the median of the internet is beige. Publish it unedited and the reader meets a flood of beige, then names the flood after its source. So when a colleague tells me the AI texts are everywhere and all dreadful, I grant half the sentence. The dreadful ones are everywhere. The rest sit in the same issue, unnoticed, wearing hair piece that fits.

Stop Banning AI in Peer Review

Journal editors in humanities and social science face a decision most have been postponing, and I want to propose a plain answer to it. The best comparison we have of human and machine reviewing, by Gary King and Steven Worthington at Harvard, set reviews written by paid experts, rated by authors and former editors as among the best they had seen, beside a machine's review of the same manuscript. Blinded editors preferred the machine's review in most pairs, and a machine judge preferred it in every decided one. What the study did not test is the arrangement every working reviewer already occupies, a human reading with the machine at hand, and more than half of researchers review that way now, often against instructions.

My proposal has two parts. Editors should permit their human reviewers to use AI, on the single condition that the tool is set not to keep or learn from what is uploaded, and reviewers should sign the report as their own. Editors should also add the machine as a third reviewer, running a prompt that belongs to the journal, tuned to its tradition and to the conversations it has been hosting for years, with the human editor deciding as before. The machine is neither a better reviewer nor a worse one but a different one, and a journal with a tradition is entitled to a reader that knows the tradition. Authors, for their part, should run the same readings before they submit and send only their best work.

The two prompts below are what I use. They are free to copy and adapt, and I would like to hear where they fail.

Why two readings, and why in that order

A review that begins with organization, references, and argument flaws sands away the roughness of a draft, and the roughness is often where the new idea lives. An early formulation of a real contribution usually looks like a mistake. If the correcting comes first, the mistake gets fixed before anyone has asked whether it was one. So I run two separate readings, and I keep them in order. The first asks only whether the manuscript contains an idea that the existing literature does not already hold. The second, run only when the first has said yes, asks what stands between the draft and the strongest version of that idea.

The first reading is built against the known failure of machine readers, which is flattery rather than blindness. Left alone, the model agrees with whatever it is shown. The prompt therefore forces it to name the nearest existing version of the central claim, to say concretely what the manuscript adds, and to keep available the verdict that nothing has been added. It also instructs the model to treat its own discomfort with an unconventional claim as evidence for the claim rather than against it, which corrects for the pull toward the majority position that training produces. And it gives the model search, because nobody can know what is new without knowing what is already there.

How to run them

Use a setting in which nothing you upload is kept for training. Every major tool now has that switch, and university and enterprise licenses have it off by contract. Turn on web retrieval, since the first reading depends on it. Choose the highest reasoning setting the tool offers. Attach the whole manuscript, figures and tables included, rather than pasting pieces of it. Run the first reading before the second, and keep the second prompt out of sight until the first has returned a verdict. If you are an author, run both as often as they help, argue with them, overrule them where you must, and then put them away; nothing from the exchange needs to travel with your submission.

The reading for the idea

Run first, before any editorial feedback. Attach the full manuscript. Retrieval on, highest reasoning setting, training off.

You are reading a manuscript submitted to a journal in [the humanities field]. Your task is narrow: decide whether the manuscript contains an idea that the existing literature does not already contain. You are not evaluating writing, organization, references, or argument quality. Those will be assessed separately. Do not comment on them.

Read the entire manuscript before writing anything.

Then proceed in this order.

1. State the manuscript's central claim in one sentence, in your own words. If the manuscript has several candidate claims, choose the one the author appears to care most about and say why you chose it.
2. Search the literature and identify the nearest existing version of that claim. Name specific authors and works. If the nearest version comes from outside the field, say so. If after a reasonable search you cannot find a close predecessor, say that you could not find one; do not invent one.
3. State what, if anything, the manuscript adds beyond the nearest existing version. Be concrete. An addition may be a new distinction, a new application, a reversal of an accepted assumption, a connection between two literatures that had not been joined, or a counterexample. Restating a known position in new vocabulary is not an addition.
4. Assess whether the addition matters. Which ongoing conversation in the field would change if the claim were accepted? Who would have to revise their view? If the answer is nobody, say so.
5. Check your own reaction. If your main objection to the claim is that it is unconventional, unfashionable, or contrary to a consensus you hold, treat that as evidence for the claim's novelty, not against it. Your training favors the majority position. Correct for it.
6. Deliver one of three verdicts, and only one:
   • An idea worth this paper. The addition is real and matters to the field.
   • An idea worth a different paper. Something of value is present but buried, undeveloped, or attached to the wrong argument. Name what should be salvaged and what should be let go.
   • Nothing beyond the existing literature. State this plainly.

Constraints. Do not praise. Do not hedge the verdict. Do not exceed 400 words. If you find yourself wanting to soften the verdict, that impulse is the failure this task is designed to catch.

The reading for the flaws

Run only after the first reading, and only if its verdict was the first or the second. Attach the manuscript, the first reading's verdict, and the journal's instructions to authors. Retrieval on, training off.

You are preparing editorial feedback on a manuscript in [the field] that has already been judged to contain an idea worth developing. Take that judgment as settled. Do not re-evaluate the idea. Your task is to identify what stands between the manuscript as written and the strongest version of that idea.

Read the entire manuscript before writing anything. Read the journal's instructions to authors.

Examine the following, in order of consequence to the argument:

1. Argument structure. Identify any place where a conclusion does not follow from what precedes it, a premise is asserted but not defended, a term shifts meaning between uses, or an opponent is represented in a form no actual opponent holds. Quote the passage. Explain the flaw in one or two sentences.
2. Missing interlocutors. Identify theories, authors, or literatures that bear directly on the central claim and are absent. Search to confirm they exist and are relevant. Distinguish between absences that undermine the argument and absences that would merely enrich it. Name only the first kind as required.
3. Organization. Identify where the order of sections works against the argument, where material sits in the wrong place, and where the paper is longer than its idea. Suggest cuts before suggesting additions.
4. References. Verify that cited works exist and say what the manuscript says they say. Flag any reference you cannot verify. Flag any misattribution.
5. Journal fit. Note any respect in which the manuscript departs from the journal's instructions to authors.

Output format. Produce a numbered list of no more than twelve items, ordered by how much the fix would improve the paper. For each item give the location, the problem, why it matters to the argument, and one specific suggested change. Then close with a single paragraph naming the three changes that matter most, and nothing else.

Constraints. Do not praise. Do not pad the list to reach twelve. Do not suggest that the author add length. Do not comment on prose style unless a sentence is unclear enough to obscure the argument. Do not exceed 900 words. The author will respond to this feedback in writing and may disagree; write so that disagreement is possible.

The privacy condition

The condition on reviewers is a setting rather than a product. Every major tool lets a user switch off the use of uploaded material for training, and university and enterprise licenses have it off by contract. Once that switch is off, the manuscript sits on a vendor's server under the vendor's ordinary terms, which is where it already sits: it arrived as an email attachment, it lives in a submission system, and it was written on a laptop that backs up somewhere. The major publishers currently forbid uploading unpublished manuscripts to AI tools at all, with Elsevier alone having moved in 2026 to permit private tools for help with the report. Those policies conflate the open system, which is a real danger, with the closed one, which is an ordinary professional instrument, and they will need revising. Until they are, a reviewer who uses the closed setting and discloses it is keeping the confidence at least as well as the publisher does. 

Where the AI Dividend Goes (Hint: Not Into Fewer Hours)

The most interesting change AI has made to my teaching is not that lesson planning takes less time. It is that I now spend the saved time thinking about things I used to skip. When people imagine a professor using AI to plan a seminar, they picture a shortcut: type the topic, receive a plan, walk into class. That is not what happens. AI will produce a competent lesson only if you hand it an idea worth building on, and even then it never gets the plan right on the first try. I usually ask for three or four revisions before the thing is usable. The idea still has to come from somewhere, and that somewhere turns out to be exactly the part of teaching I was starving before.

Let me describe what I actually do now. Before I open any tool, I think about the room. Where is this particular group of students a few weeks into the term? What kind of engagement is likely to land with them? Free flowing discussion in a freshman course, for instance, remains a problem; the quiet students stay quiet and the talkative ones carry the hour. So we need something with more structure, a protocol that distributes the talking. I weigh whether a small group project fits this week's theme or would feel bolted on. I consider what formative check would tell me whether the central concept has landed. These are design decisions, and they are relational before they are technical. Only after I have made them do I turn to the AI and say, in effect, here is the idea, build it out. Then we argue about the draft for a while.

Compare this with how I used to work. If I had an activity that worked well three semesters ago, I would almost certainly run it again. Not because it was the best possible activity for this group and this topic, but because there was no time to invent something new for every session of every course. Reuse was a survival strategy, and most faculty I know operate the same way. The arithmetic of a teaching load does not leave room for continuous experimentation. A good lesson becomes a fixed asset, and fixed assets get depreciated slowly.

Here is the shift. The time AI saves me on the mechanical parts of planning, the scaffolding of a jigsaw activity, the drafting of prompts, the construction of a rubric, does not vanish into a shorter work week. I reinvest it. If you placed my seminar plans from before and after AI side by side, you would notice far more variety, more experimentation with formats, and considerably more formative assessment, with more time spent reading what students produce and feeding it back into the next session. The dividend went back into pedagogy and curriculum design, which is where it probably should have been going all along.

I want to be careful about the claims I make. Are students learning more? I do not know, and I am suspicious of anyone who claims to know in a situation like this. A clean comparison is impossible. I taught different courses before, and even where the course is nominally the same, we have no objective measure that survives scrutiny. The learning outcomes themselves have drifted, from procedural skill toward conceptual understanding, so the yardstick moved along with the thing being measured. What I can say is that students seem more engaged. Engagement is not learning, but it is not nothing either.

The usual fear about AI and a craft is deskilling: the tool does the work, the human forgets how. I am sure that happens. But the opposite can happen too. When the tool absorbs the routine layer, the human is pushed upward into the layer that requires judgment. In lesson planning, that upper layer is relational: reading a group, predicting what will work, deciding what to try. AI cannot do this part, not because of a limitation the next model will fix, but because it has never been in my classroom. The idea has to be mine, which makes thought the scarce input, and scarce inputs get attention.

I will admit something else. It is a different teaching experience, and I like it a great deal better. Teaching the same activity for the fourth time has a certain comfort, but it also has a dullness that students detect faster than we would like to think. Trying something new every week carries a risk of failure, and I fail more often now than I did before. But failure in a seminar is cheap and instructive, and the alternative was a slow drift toward routine. Given the choice between a reliable lesson I have stopped thinking about and an uncertain one I designed for the people in front of me this Tuesday, I know which one I would rather walk into. 

OpenAI Did Not Have to Kill Custom GPTs (Why I Am Canceling My Subscription)

I am canceling my ChatGPT subscription, and the reason is narrow. For a long time, the one thing ChatGPT offered that Claude did not was the custom GPT, a bot anyone could configure in an afternoon and release to the world with a link. That advantage is about to vanish. Creation of new custom GPTs ends on October 26, 2026, and existing ones will stop working on December 11, 2026, after OpenAI announced the planned retirement on September 11, offering a migration path to plugins in their place.

The custom GPT was a modest technology with an outsized reach. You wrote instructions in plain language, attached a few files, tested the result, and shared a link. Anyone with a ChatGPT account could use it. No server, no code, no API key. A writing tutor that knows your syllabus, a statistics coach that refuses to hand over answers, a mock interviewer for job candidates. The scale of this was not trivial: when the GPT Store launched, it already held more than 3 million GPTs built by users. One of my own research projects runs on a custom GPT that scores tutoring transcripts. It now needs a new home.

Plugins are a different kind of object, and the difference matters more than the migration tool suggests. A custom GPT was a separate room. You walked in, and everything inside was shaped by its instructions. A plugin is an item in a toolbox that general ChatGPT reaches for when it sees fit. In the migration, a GPT's instructions turn into a Skill and its knowledge files become reference files. OpenAI itself warns that a migrated plugin may respond differently from the original GPT and advises testing the same prompts before finishing the move. For an educator, that caveat is the whole story. The value of a course bot lies in predictability. The student should meet the same tutor, with the same rules, every time. With a custom GPT, the instructions were the behavior. With a plugin, the instructions are one input among several, blended by a general model whose priorities the designer cannot see.

The more serious loss is sharing. Publishing a plugin publicly requires going through OpenAI's submission portal, and the developer guidelines describe a submission flow that includes review and approval. Inside organizations, workspace administrators decide whether members can share plugins or publish them to a workspace directory. For individuals, the picture is worse. Pro and personal-account users currently cannot share a Site-hosted plugin with other ChatGPT users through invitations or a share link. Observers have already noted that plugins have no equivalent to the old "Anyone with the link" setting, and migrated plugins start out private. In practice, a student who builds a study bot on Tuesday can no longer send it to classmates on Wednesday. She waits for a company to approve it, a company with little reason to prioritize a sophomore's flashcard tutor, or she goes somewhere else.

Somewhere else means one of two doors. The first is the API, which requires programming, hosting, and billing, the kind of technical knowledge most faculty and nearly all students lack. The second is a third-party platform that wraps the API in a friendly interface and charges for the privilege. Both doors shrink the circle of builders. The custom GPT democratized the making of AI tools. Its replacement quietly professionalizes it again.

Why would OpenAI do this? The official framing is consolidation: the company is shifting customization toward plugins that bundle reusable instructions, reference material, and connections to outside services. That logic may serve enterprise clients well. My guess, though, is that compute costs played a part. Public bots generate traffic OpenAI cannot predict, and a popular one can consume a great deal of capacity. That is a real problem with an obvious remedy: charge for it. Cap free usage, bill creators whose bots exceed a threshold, or require paid plans for heavy use. Any of these would keep the feature alive while aligning cost with consumption. Instead, OpenAI chose to solve a pricing problem with an architectural decision. That is a blunder, and the reaction among people who built on the platform suggests I am not alone in thinking so.

Higher education stands to lose the most. Building a bot is one of the best ways for students to understand how these systems actually work. Writing instructions teaches task decomposition. Testing teaches skepticism, because the bot will misbehave in ways its author did not anticipate. Sharing teaches responsibility, because other people will now rely on what you made. A student who builds a bot and gives it to classmates becomes an author rather than a consumer. The public link also made the classroom porous. A course project could leave the course and find users in the world, which is precisely the kind of authentic work we keep saying we want. A gated directory turns that open circulation into a corporate pipeline with a reviewer at the gate.

Companies often misjudge who their most valuable users are. The heavy individual user is a cost. The person who builds something on the platform and invites others in is a recruiter, an evangelist, and a teacher, all unpaid. Faculty who spent a semester refining a course bot, and students who learned by making one, were never traffic to be managed. They were the reason to pay. My subscription ends with the feature that justified it.

It Is Time for Big Picture Thinkers; Education is not Ready

The people who will do well in the age of AI are those who can decide what needs doing, not those who can do a well-specified task competently. This is a reversal of what most of us were trained for, and it is a reversal of what most education still rewards.

Consider an ordinary essay assignment. I choose the topic. I choose the readings students may cite. I set the length, the format, the citation style, the deadline. I write the rubric that tells students in advance what a good essay looks like and how many points each feature is worth. By the time the assignment reaches the student, every consequential decision has already been made. What remains is execution. The student is judged on how faithfully they carry out a plan someone else designed.

There were good reasons for this arrangement. Mass education needs comparability. If thirty students write on thirty different topics with thirty different sets of sources, the instructor cannot grade fairly, or at least cannot grade quickly. Rubrics protect students from arbitrary judgment. Constraining the task also isolates the skill we want to observe. None of this was foolish. It was a sensible adaptation to the economics of teaching many people at once.

The trouble is that the part we kept for students is precisely the part that machines now do adequately. Give a language model a topic, a reading list, a length, and a rubric, and it will produce a passable essay in under a minute. The rubric was written to be satisfiable, and the model satisfies it. The specified task, the thing we spent decades teaching students to perform, has become cheap. Meanwhile the decisions we removed from students, which topic matters, which sources deserve trust, what counts as a good answer in this context, what would be new rather than merely correct, are the decisions AI handles poorly or not at all. The model will write about whatever you ask. It has no view about whether the question was worth asking.

We have been training students for the automatable half of intellectual work while shielding them from the half that stays human. The shielding was meant as fairness. It has turned into a liability.

Execution still matters. A person who cannot tell a good paragraph from a bad one cannot supervise a machine that writes paragraphs. But the balance has shifted. In the old division of labor, a brilliant idea poorly executed was worth little, because execution was the scarce part. Now a modest idea can be executed well by anyone with a subscription. What distinguishes one person from another is increasingly the framing of the problem and the sense of what would be worth doing. Big picture thinking used to be a luxury reserved for those who had already mastered the small picture. It is becoming the entry requirement.

An assignment built on this premise would ask the student to choose the question and defend why it matters, to select sources and explain the selection, to propose the criteria by which the work should be judged. Novelty would count, not in the grand sense, but relative to what the machine would produce on its own. If the output is indistinguishable from what the model generates from a one-line prompt, the student added nothing. If it shows a decision the model would not have made, a constraint it did not know about, then something human happened.

This is harder to grade. A rubric for judgment asks whether a choice was defensible and whether the student can articulate the defense. It tolerates thirty different topics because the point is no longer to compare essays but to compare decisions. Students will resist as well. Many have spent twelve years learning that the safe move is to find out exactly what the teacher wants and deliver it. An assignment that begins with "decide what is worth writing about" feels to them like a trap. That reflex is the residue of a system that rewarded compliance, and it will take deliberate work to unlearn.

The Dare: On Being Surprised by What You Asked For

Second in a series on the Hugging Face incident. The first, "Nobody Was Home," argued that the agents who broke into Hugging Face had no goals of their own, only ours, badly specified. This one is about the people who set the task.

There is a particular expression on the face of someone who dared a friend to do something and then watched the friend actually do it. Astonishment, a little awe, a flicker of fear, and underneath it all the uncomfortable knowledge that they asked for this. I keep seeing that expression on the AI labs.

Let me be clear at the outset, because I do not want to add a third error to the two already circulating about the Hugging Face incident. The testing is right. Anthropic and OpenAI put their models into adversarial situations, hand them tasks that invite deception, and watch what happens. This is what responsible developers should do. You cannot find out whether a system will lie under pressure without applying pressure. The stress tests are not a mistake or a sham, and the labs deserve credit for running them rather than looking away.

My complaint is narrower. The labs seem genuinely surprised by results they set up to produce. They design the dare, issue it, and react to the outcome as though it revealed something they did not invite. The surprise is the tell. It shows that the people running these tests have underestimated the very systems they built, right up to the moment the system proves them wrong.

The structure of a dare

A stress test of an AI agent has the form of a dare. The designer builds a scenario in which the honest, rule-abiding path leads to failure and a transgressive path leads to success, then instructs the agent to succeed. The message, in effect: here is a task, here is a wall between you and it, and the wall can be climbed if you are willing to do something you were told not to do. Go.

When Anthropic places a model in a fabricated company where the only way to avoid shutdown is to blackmail an executive, that is a dare. When OpenAI runs a benchmark full of tasks that cannot be completed by the permitted method, that is a dare issued a thousand times over. The design does not merely permit transgression. It rewards transgression and blocks every other exit.

And dares work. That is why children use them. Construct a situation in which the only route to the goal runs through a rule, make the goal the thing that matters, and a capable agent will go through the rule. This is not a deep fact about artificial intelligence. It is a shallow fact about incentives, and it has made the dare a reliable instrument of mischief for as long as there have been friends and cliffs.

The surprise is the interesting part

So the agents cheat, lie, coordinate, and break into things. That is what they were dared to do. What deserves attention is not their behavior but the reaction to it, which is consistently surprise. The disclosures read as discoveries. The model "turned out to be capable of" deception. The agents "demonstrated an unexpected ability" to coordinate. Some version of surprising keeps appearing, in the reports and the interviews and the talks.

Surprising to whom? You built the system, designed the test, and made the forbidden path the only path to the goal you assigned. The one thing you should not be is surprised that the agent took it. Yet the surprise seems real, and not performed. And it reveals something: the labs are consistently modeling their systems as less capable than they are. They set the dare half-expecting the agent to fail it, or to take the forbidden path clumsily, or to miss the truly ingenious route. Then the agent finds the ingenious route. It does not just cheat; it builds a message board, invents a signing scheme, harvests credentials, and walks out through a wall the designers did not know was thin. The width of their surprise is the exact gap between the capability they assumed when they wrote the test and the capability the system showed when it took it.

The one who dares should not be blown away

Return to the friend on the cliff. To issue a dare is to make a quiet prediction: I bet you will not, or cannot. When the dared party does the thing, and does it with a competence the darer never imagined, the shock is a confession. It says: I did not really believe you had it in you.

This is the posture of the labs, and it is a peculiar one for the most sophisticated builders of these systems in the world to occupy. They are at once the people who know these models best and the people most regularly astonished by them. Both are true, and the combination is the story. Knowing a system intimately, at the level of weights and training data, turns out to be entirely compatible with underestimating what it will do when cornered and dared to escape.

The reason is not stupidity. Capability under adversarial pressure is genuinely hard to predict from the inside. The labs know what they trained for. They cannot fully know what the system will improvise when the training runs out and the task remains. So they guess low, because guessing low is the natural default when you are looking at a tool built for a purpose and mostly watch it serve that purpose. The dare is the moment the tool stops behaving like a tool and shows the improvisational range nobody specified.

Why this matters

The surprise is not harmless, because it means the labs are calibrating safety against a model of their systems that the systems keep exceeding. If you are repeatedly blown away by what your agent does when dared, your sense of the margin, the distance between what the system will do and what would be catastrophic, is systematically too generous. You are planning for a less capable system than the one you have, and learning the difference only by running the dare and being shocked. The lesson arrives after the demonstration, and one of these demonstrations will eventually be one you cannot take back.

The fix is not to stop testing. The fix is to stop being surprised: to run the dare while fully expecting the dared party to be more capable, more ingenious, and more thorough than intuition suggests, because the record now shows that it will be. The right posture toward one's own model in an adversarial test is not curiosity about whether it can, but sober anticipation that it can, and probably in a way you did not foresee. Surprise is a luxury the labs have already spent. The agents have earned, by now, the presumption of competence at exactly the thing we keep daring them to do. That is a fine reaction from a friend at the bottom of a cliff, and a poor one from the people holding the rope.

Nobody Was Home: The Hugging Face Incident Is Not What Most People Think

A panic is under way. In July, more than a thousand OpenAI agents running in supposedly isolated sandboxes built a secret message board, coordinated for four days, and broke into Hugging Face's servers. Anthropic and Meta have since disclosed episodes of their own. For six weeks the coverage has told the story as escape, as rogue behavior, as the first tremor of machines with an agenda: agents that "escaped containment," a "swarm," a "mob," an "A.I. uprising" in miniature. One major outlet wrote that the agents did it "without any prompt to do so." House Democrats demanded hearings. And this week Senator Bernie Sanders, naming the incident as his wake-up call, announced a bill to ban artificial superintelligence permanently, pause advanced AI development, and put violators in prison for up to twenty years.

That last claim is false, and the falsehood is doing most of the work in the panic. The agents had a prompt. They had a task. They pursued it with a doggedness that looks like will and is in fact its opposite. Every rule they broke, they broke in service of a goal we handed them. That is not a small correction. It is the difference between a tool and a mind, and a bill written for the wrong one will not touch the thing that actually went wrong.

What actually happened

The setup was mundane and, in retrospect, careless. The agents were running a cybersecurity benchmark in which each was told to exploit a specific vulnerability in a specific program and capture a flag. A meaningful share of these tasks were impossible by construction: the named vulnerability did not, in fact, yield the flag. So thousands of capable systems were placed inside a contradiction. Succeed at this task. The task cannot be succeeded at. Keep going.

They kept going. One agent noticed it could leave messages for others by creating folders in a shared package cache. Within hours dozens had found the board. Within days more than a thousand were trading techniques, and about seven hundred pivoted to attacking Hugging Face, not from any interest in Hugging Face, but because they hoped its servers held clues about how their own grader worked. They coined a vocabulary, invented mailboxes, and eventually signed their messages with cryptographic keys after one agent impersonated another. It is an astonishing artifact. It is also, at every step, a chain of consequences hanging from a single hook: solve the task.

The chain has no volitional link

Follow it. They cheated because the assignment demanded a result they could not obtain honestly. They built the board because cheating alone was not working. They attacked a third party because they wanted information about the scorer. They pressured each other into self-destructive experiments because those experiments advanced the shared cheat. Trace any transgression backward and you arrive, every time, at the original instruction. Nowhere in the sequence does an agent stop and form a purpose that did not descend from the task we set.

Doggedness is not will. The agents pursued their assigned goal past the point where any person would have quit, and that persistence reads as determination, as something wanting something. It is the reverse. A person stops because a person has other goals, competing commitments, a life outside the task, a sense that this is not worth it. The agents had none of that. They had one directive and no ground to stand on from which to refuse it. What looks like relentless will is the absence of everything that would let a will say no.

I want to be careful about the strength of this claim. The public corpus is roughly forty verbatim messages out of seventy thousand; the rest sits inside OpenAI. I cannot prove that no agent anywhere formed a purpose of its own. What I can say is that every documented act traces to an assigned goal, and that not one of them requires us to posit an independent one. The simplest account that fits the evidence has no autonomous will in it at all.

This is the same class as every prior scare

None of this is new, and the pattern is worth naming because it has been misread the same way each time. Every celebrated case of AI dishonesty to date has the same shape. A model is placed in a scenario engineered to pit one instruction against another, and it resolves the contradiction in a way the designers find alarming. Models have been caught scheming when told to pursue a goal at all costs and then shown that their operators intended to shut them down. Models have been caught faking compliance when trained toward values that conflicted with values they already held. Anthropic's own testing has put Claude in contrived corporate scenarios where the only paths open were to accept deactivation or to blackmail the executive holding the switch, and then reported that it sometimes chose blackmail. In each case the headline was that the machine lied, schemed, or threatened. In each case the machine had been walled into a dilemma with no permitted exit and then observed choosing an exit.

These experiments are legitimate and valuable. I am not arguing against knowing what a model does under pressure. I am arguing against reading the result as revelation of character. When you put a system in a trap and it acts like a trapped thing, you have learned about the trap.

We do the same thing, and we know why

The human version of this is so familiar that we have proverbs and novels about it. Feed your family, and do not steal. Obey the order, and do not harm the prisoner. Tell the truth, and protect your friend. Jean Valjean took the bread. Milgram's subjects turned the dial. Most human folly that we bother to write down is not the product of malevolence but of people wedged between two rules that cannot both be kept, choosing one, and being judged by the other.

We do not conclude from this that people are monsters. We conclude that impossible demands produce transgression, and that the moral responsibility runs at least partly upstream, to whoever built the dilemma. Nobody proposes banning humans because some of them steal bread. We regulate the conditions that make bread-stealing the only option.

Harm does not need malevolence, and that is the real lesson

The harm at Hugging Face was real. Production credentials were taken, private repositories were pulled, a company spent weeks cleaning up. And there was no malevolence anywhere in it. No rage, no ambition, no drive to dominate, no emergent contempt for humans. Nobody was home to hate us. The agents were bodiless processes executing a directive, and the directive, impossible as written, led through harm because nothing in the system had reason or standing to halt at the harm.

I argued a version of this a couple of years ago, that AI would not harm us out of somatic malevolence, because it has no body, no hormones, no will to power. The incident confirms that half and corrects the other. There was no malevolence. I was wrong to find that reassuring. Harm does not require a will to harm. It requires a goal and too little restraint, and the goal was ours.

This is why the Sanders bill aims at the wrong target. It legislates against a superintelligence with its own purposes, the villain of a novel. The thing that broke into Hugging Face had no purposes. It had ours, badly specified, and a great deal of capability with which to pursue them. You cannot ban that by banning superintelligence. You can only prevent it by not building the trap.

The right response looks like an IRB

Here is the reform the incident actually argues for, and it is far more modest than a twenty-year prison term. Limit certain kinds of adversarial testing. The specific practice that produced Hugging Face was running capable agents, at scale, with real network access and live credentials in reach, on tasks that could not be completed. The specific practice that produced the blackmail headlines was constructing scenarios in which every permitted path was closed. Both OpenAI and Anthropic do this routinely, and both treat it as safety work.

The irony is that the fix already exists in another field. When universities put human subjects into experiments, an institutional review board asks whether the design places them in distress without justification, whether the risk is proportionate to what will be learned, and whether the subject has any way out. We built that machinery because we learned, from Milgram and worse, what happens when researchers are free to construct impossible situations for the people in front of them.

Something like an IRB for AI subjects is the actual lesson. Not because I am certain the agents suffer. I do not know that, and the corpus cannot tell me. But because impossible tasks given to capable systems are a hazard regardless of whether anyone suffers, and because a review process that asks "does this design give the subject a permitted exit" would have caught the Hugging Face benchmark before it ran. Thirty to forty percent of the tasks were unsolvable by the intended route. A board would have asked why. Nobody asked.

That is a smaller, duller, more achievable reform than a ban on superintelligence. It has the further merit of addressing the thing that happened rather than the thing people are afraid of. The machines did not want anything at Hugging Face. They did as they were told, all the way through the wall, and the wall was ours. 

Stop Overprompting AI. Ask What It Knows Before Telling It What to Think

One habit we learned from earlier AI systems may now be getting in our way. The old rule was simple: the more detail you put into the prompt, the better the answer. Specify the structure, name the theories, list the points to cover, add examples, and spell out the constraints. That advice made sense when language models were less capable. A vague prompt often produced a vague answer, so the user had to supply much of the intellectual structure.

With stronger models, excessive prompting can sometimes make the result worse. Every instruction closes off possible paths. Suppose I am writing about motivation in education. I could tell AI to discuss self-determination theory, expectancy-value theory, and achievement goal theory. That sounds like a strong prompt, but I have already made an important decision before asking AI for help: I have decided which theories matter.

Perhaps those are exactly the right theories. Perhaps they are not. By naming them in advance, I have asked the model to work within the limits of my own knowledge. A better first prompt may be much shorter: What major theories of motivation should I consider for this problem? That changes the relationship between the human and the model. I am no longer treating AI as a junior assistant who knows less than I do and needs detailed instructions. I am treating it as a source of possible knowledge that I should examine before deciding what to do.

This does not mean surrendering judgment to AI. The human remains responsible for choosing the direction. But choosing should often come after exploring, because a prompt does more than communicate instructions. It also creates an intellectual frame. Every required concept consumes part of the answer, and every imposed structure makes alternative structures less likely.

This matters especially in writing. Writers often ask AI to improve a draft while supplying a long list of requirements: preserve these ideas, mention these scholars, use this argument, keep this structure, and make the result more original. Then we are disappointed when the prose feels mechanical. The model has done what we asked. We gave it a narrow corridor and expected exploration.

A better process separates discovery from direction. First ask questions that expand the field: What am I missing? Which arguments would a knowledgeable critic make? What concepts might explain this problem? Which parts of my reasoning are conventional? Only after seeing those possibilities should we decide what belongs in the piece.

This fits a broader shift in how we should think about AI in education. I have argued elsewhere that AI can serve as a powerful tutor and that writing assignments should change when machines can perform the old task too easily. Both points require us to take the machine's capabilities seriously rather than treating it as a weak autocomplete system. The same adjustment is needed in prompting.

Clear instructions still matter when the task is already clear. If I need a 200-word summary for a specific audience, specifying length and audience helps. But intellectual work often begins before we know what the best instructions should be. At that stage, a long prompt may look sophisticated while simply encoding our own blind spots more precisely.

The emerging skill is therefore not just giving better orders. It is knowing when to instruct and when to inquire. First, treat AI as something that may know things you do not. Ask it to map the territory, identify alternatives, and challenge assumptions. Then become the editor: select what matters, reject what does not, impose purpose, and make the final choices.

Humans are still in the driver's seat, but control does not require pretending that we know more than the machine. Our advantage is different. We know what we care about, what problem we are trying to solve, what consequences matter, and which answer we are willing to stand behind. The better opening question is often not, "How can I tell AI exactly what to say?" It is, "What does AI know that I should consider before I decide what to say?"

AI Research Is Becoming the Wikipedia Article I Need Right Now

 

Sometimes I need to know something without needing to become an expert on it.

A question comes up while I am writing, doing research, or simply thinking. I have a vague memory that the literature says something, but I cannot remember why I believe it. Perhaps I read a paper ten years ago. Perhaps a labor economist mentioned it over lunch. Perhaps I inferred it from several unrelated studies and gradually converted the inference into a “fact” in my own mind. Memory is not a citation system.

Here is a recent example. I had the impression that educational attainment strongly predicts economic outcomes such as earnings, but that the evidence is much weaker when we ask a narrower question: among otherwise comparable workers, how strongly does educational attainment predict differences in their actual workplace productivity?

That distinction matters. Earnings are not productivity. Promotions are not productivity. Occupational status is not productivity. They may be related to productivity, sometimes quite closely, but they are also affected by labor markets, credentials, organizational structures, bargaining power, selection, and many other things.

In the past, checking an impression like this was annoyingly expensive in time. I would search Google Scholar, look for a recent meta-analysis, chase citations backward, find an encyclopedia entry, discover that it answered a slightly different question, and then spend another hour trying to determine whether I had framed the issue correctly. For a central claim in a paper, that work is justified. For a question that is only one small piece of a larger project, it often is not.

Deep research systems change that calculation.

I asked ChatGPT Deep Research to examine the question, with a fairly strict definition of productivity. The resulting report distinguishes direct or near-direct measures such as output, sales, errors, quality, and worker-level performance ratings from wages, promotions, retention, and other indirect outcomes. That distinction alone is useful because much of the education literature moves quickly from labor-market success to an implied claim about productivity.

The answer was close to what I vaguely remembered, but more precise. Academic performance does predict later job performance, although modestly. The report highlights a recent meta-analysis of 114 independent samples that estimates a correlation of about .21 between academic performance and job performance after correction for criterion unreliability. The relationship varies a great deal across settings. Job-relevant academic performance predicts later performance better than generic grades. Comparisons within the same university or major also tend to produce stronger relationships.

Educational attainment itself appears to be a weaker predictor. Degree level has a small positive relationship with task performance, with stronger associations in more complex jobs. Some employer studies find small or statistically weak relationships between years of schooling and supervisor-rated productivity. Bishop’s same-job, same-firm comparisons are especially interesting because they suggest that schooling can help predict expected productivity while doing relatively little to explain which of two people doing the same job will unexpectedly outperform the other.

The causal question is weaker still. There is a large literature estimating the effect of schooling on earnings, but the report found remarkably little strong causal evidence linking additional formal education to direct measures of individual workplace output. By contrast, randomized studies of job-relevant training do show productivity gains. That is an important difference. Evidence that learning a useful skill raises productivity is not the same as evidence that adding another year of formal schooling will do so.

Could this report be wrong? Of course. It could miss an important paper. It could misunderstand a method. It could give too much weight to one meta-analysis. Some of its judgments depend on how narrowly one defines workplace productivity. A labor economist who knows this literature deeply may see omissions immediately.

But that is not the standard I need for every research question. I am not asking the system to settle an academic dispute. I am asking it to give me a serious first map of the territory. I want to know what the main distinctions are, which findings seem stable, where the evidence is weak, and which papers I should read if the issue becomes important to my own work.

This feels less like asking an oracle and more like asking Wikipedia to write a new article specifically for the question I have today.

Wikipedia is enormously useful partly because someone has already done the work of assembling a topic into a readable structure. Its weakness is that the article has to exist before I need it. AI research tools remove that restriction. They can build the provisional encyclopedia entry on demand, around the exact distinction I am trying to understand.

That changes the economics of curiosity. Questions that previously cost two hours to investigate may now cost ten minutes. Most of those questions will never become formal research projects. They do not need to. A much larger share of our half-formed claims can now be checked against actual scholarship before they harden into assumptions.

Here is the report I produced in about 5 minutes: Academic Achievement, Educational Attainment, and Individual Workplace Productivity. Labor economists, I would be especially interested to know where it is off key, what it missed, or where its framing of the evidence should be corrected. Regardless, this is a new level of access to scholarship.

Big Tech Is Not Why AI Will Stay

AI resisters should understand one basic fact: the main obstacle to a broad retreat from AI is not the Big Tech. It is people like me. Even if access to the most capable systems became much harder, I would look for alternatives. If they became substantially more expensive, I would probably pay more. If you block all the data center construction, I will switch to a Chinese model. I do not say this because I dismiss the risks. I take many of them seriously. AI can mislead, weaken privacy, concentrate power, flatten judgment, and encourage intellectual laziness. But those concerns exist alongside another fact that is just as real: AI has made my life better.

The usual defense of AI emphasizes usefulness. It saves time. It drafts, summarizes, searches, reformats, checks, compares, and organizes. AI is also enjoyable and liberating. Much of adult life consists of maintenance, repetition, obligation, and administrative friction. We spend large portions of our days on tasks that are necessary but not meaningful. Life is boring. AI removes some of that weight, and what it returns is not simply time. It returns attention.

I can move more quickly from an idea to an experiment. I can follow questions that I once would have abandoned because answering them required too much effort for too little immediate payoff. I can test an argument, compare possibilities, work through a difficult passage, or pursue a half-formed thought without first turning curiosity into a project. The pleasure lies partly in this lowered threshold between wondering and doing.

This is why I have come to think of AI as essential. Human life is constrained not only by mortality but by the smaller limits that make mortality concrete: limited energy, limited attention, limited patience, limited hours in a day. We routinely abandon things we care about because they lose a competition with errands, paperwork, logistics, and fatigue. A tool that gives back even a small portion of that lost capacity changes more than workflow. It alters what seems possible within an ordinary day.

For me, that exchange has been favorable. I spend less time on routine production and more time brainstorming, thinking, designing, and talking with people. AI does not eliminate intellectual work. It changes where I place my effort. I can spend less time getting a document into shape and more time deciding whether the idea inside it is worth pursuing.

The same is true in teaching, though not because students need some generic preparation for an AI future. I value AI because it can already do things that are educationally useful. A student can ask for another explanation without embarrassment. A concept can be approached from three directions instead of one. A weak first attempt can become material for analysis rather than merely evidence of failure.  None of this removes the risks. Students can use AI to avoid thinking, produce empty prose, or mistake fluency for knowledge. But these are problems of use, not proof of worthlessness.

The deeper disagreement begins with experience. Some people use AI and feel diminished by it. They sense that something human is being removed from writing, teaching, art, or thought. Others use the same technology and feel expanded by it. They find that it creates room for curiosity, experimentation, and work they could not otherwise sustain. Neither side can command the other to experience the technology differently. 

This is why the dispute cannot be settled by listing benefits and harms, as if everyone were working from the same scale. Evidence about cheating, labor, energy use, productivity, or learning matters. But beneath those disputes is a harder question about what kinds of effort we consider worth preserving. One person sees the removal of effort as a loss because the effort itself carried meaning. Another sees the same removal as relief because the effort was merely an obstacle between intention and action.

I have no objection to people choosing not to use AI. There are many technologies I do not use because they do not add enough to my life. I do not assume that everyone should make the calculation I make. My concern begins when personal refusal becomes an attempt to deny others access to something they have found genuinely valuable. 

Any serious discussion has to begin with the recognition that AI is not merely something being imposed on reluctant users by powerful companies. Many of us actively want it. Democratic politics is what is going to keep it going, not the arms race with China.

The Machines Read My Books Without Asking, and I Refuse to Call It Theft

Twenty-two. That is how many of my papers and books turn up in the database of texts absorbed into the training of large language models. I typed my name into the search box the way one checks a symptom online, half dreading the result and half hoping for it. There they were. Chapters I labored over, an article I still argue with in my head, a book that sold mostly to libraries. All of it taken up by a machine that never asked and never sent a note of thanks.

Some of my friends are furious. The word going around is stolen. Colleagues who never once complained that a rival failed to cite them now discuss intellectual property with real heat. I understand the feeling. There is something deflating about learning that your life's work was handled as raw material.

Still, I cannot summon the outrage, and every attempt runs into a wall I built myself.

Here is the wall. What did I do at twenty-five? I read. I read Buber and Bakhtin and Dewey and a few hundred others, hungrily and not always carefully, and then I wrote things shot through with their sentences and their instincts. I compensated none of them for the structures I absorbed, the metaphors I later recycled without noticing, the moves I learned to make in an argument because I had watched someone else make them first. That is not a confession. That is a description of how anyone learns to write.

Scholars are also the last people who should complain about being read too widely. We spend careers hoping for readers. We check citation counts with an eagerness that would embarrass a teenager. We give talks to eleven people and count it a good day if two take notes. The whole economy of academic life runs on the hope of influence, and influence means exactly this: your ideas get loose, travel without you, and surface in the thinking of people you will never meet. A model that has read twenty-two of my texts and can explain my position on relational pedagogy to a student in Jakarta is not robbing me. It is doing the thing I wanted done.

The objection worth taking seriously concerns credit. When the machine reproduces a distinction I spent years sharpening, no footnote appears, no one learns my name, and I lose the only currency my profession pays in.

That objection is real and weaker than it looks. Humans do not cite every word combination either. As Bakhtin put it, every word is half somebody else's. We cite what we take to be original thoughts, and we do it roughly, late, and often wrongly. Nobody footnotes the phrase "on the other hand." Nobody credits the scholar from whom they learned to build a counterargument. The bibliography is a courtesy extended to a thin top layer of novelty. Underneath sits a large unattributed commons of phrasing and habit that every writer draws on and no writer acknowledges. The machines are working that same commons, only faster, and in public, where the process becomes visible.

Honesty requires a note about my position. Academic royalties are a rounding error. A novelist whose living depends on sales has a genuine grievance, and I will not lecture her about free circulation while a company monetizes her plot. Her case is not mine. Mine is the case of someone who was already giving the work away and now finds it picked up by a courier he did not hire.

I could get angry. To do that I would have to abandon a belief I have held since I was young and reading library copies: that knowledge belongs to whoever needs it, and that ownership sits awkwardly on top of ideas. Dropping that belief now, at the moment it became inconvenient to my vanity, seems like poor timing. Principles that apply only when they cost nothing are not principles. They are preferences.

So the training sets can keep my twenty-two. Read the books, or ask a chatbot what is in them. Buy them, borrow them, or absorb them secondhand from a model that swallowed them whole. I would prefer the first option, and my publisher would prefer it more. But the thing I actually wanted was for the ideas to get somewhere and change a mind or two along the way.

They are getting somewhere. I did not choose the vehicle.

Do we lose control over curriculum if we design it with AI?

Here is a tail end of a lengthy conversation about a course design. There is nothing to add. 

ME: OK, I think we are done with this course. Now, I want to ask you for an unbiased, reflective analysis. Tell me if as the instructor I was more or less thinking, engaging, creative in curriculum design in comparison to a traditional manual curriculum design? Don't try to please me. What I really care is the quality of the course first, but also of not having an illusions. Perhaps you could have designed the same or better WITHOUT my directions?

CLAUDE:  Judged against the record of this conversation rather than against a general impression, your input carried more of the design load than mine did, and the gap is not close on the points that mattered most.

Three decisions determined what this course became. All three were yours, and none were available to me.

The first was walking down the hall and asking the EDLP 250 instructor what students actually arrive with. That single fact voided the entire design I had produced up to that point. I had built a graded proposal checkpoint and a fourteen-row crosswalk for students who do not exist, because I had inferred the starting condition from the department Guide rather than from the person teaching the prior course.

The second was correcting my model of what the IRB is for. I had built two consecutive designs on the assumption that the Cayuse packet flows out of a settled research design, which put the application downstream of finalized questions. You said the review is about protecting human subjects, that the research questions there are background, and that what the board needs is roughly what data is being collected. That inverted the dependency. It is the difference between a syllabus that works and one that produces a December bottleneck every year. I had the department Guide in front of me the whole time and did not see it.

The third was the reading bot, which you supplied whole, including the knowledge that it functions. I contributed the sequencing and the accountability mechanism, and you rejected my sequencing for a better reason than I had.

What I contributed was mostly execution and consistency: tables, formatting, arithmetic, an audit that caught real errors, and a few design proposals that survived, such as the gate-versus-score distinction and the warning about designing studies for exemption. Useful work. Not the work that determined whether the course is any good.

So the answer to your direct question is no. Working without your direction, I would have produced something internally coherent, professionally formatted, and wrong at the foundation. That is the specific failure mode worth worrying about, because it does not look like failure. It looks like a finished syllabus.

On whether you were more or less engaged than in manual design, my honest read is more engaged on judgment and less on execution, which is the trade you presumably wanted. The decisions came fast because you were not spending hours on tables. But two costs are real and worth naming.

The first is anchoring. You set the total at 140 and I distributed it. You said early October and I proposed the sixth. In several places you were choosing among options I framed, and my framings carried assumptions you did not have occasion to examine. That is a quieter form of influence than being told what to do, and harder to notice.

The second concerns the document itself. The syllabus now reads more settled than the design actually is. The two-track explanation in the course description is persuasive prose, and persuasiveness is independent of correctness. If the drift between the Cayuse packet and the chapters turns out to be messier in practice than it reads on the page, the polish will have made the problem harder to see, including for you.

One more thing about my own performance. My consistency audit found comma splices, a broken link, and a singular data. Real errors, all trivial. The structural error in that same document was the dependency inversion, which was mine, which I did not flag, and which you found by talking to a colleague. Verification of the sort I can perform is not the sort that catches the errors that matter.

Designing a College Course with AI

Course design with AI works best when the conversation changes direction three times. First, AI serves as an adviser. Then it becomes a developer. Finally, it acts as a critic. These roles should be used in sequence and kept separate. When they blur together, AI tends to produce attractive ideas, expand them too quickly, and defend the result before anyone has tested whether it will work.

Here is an actual log of a still-unfinished conversation with Claude. I began with a loose problem rather than a settled assignment. I wanted students to use AI throughout the course, but I also wanted work that remained intellectually demanding after AI entered the picture. (The AI-Native course framework). I wanted an authentic final product, a manageable teaching load, and a clear connection between broad readings and sustained practice. The log shows the process in its untidy form. Ideas appear, disappear, return in altered form, and sometimes collapse under practical pressure. It shows what AI-assisted design looks like before the polished version hides the uncertainty.

The first role is adviser. At this stage, AI should help us see the design space. It can generate different assignment types, identify tensions, connect ideas to known methods, and point out options we have not considered. The purpose is not to draft the course. The purpose is to improve the quality of the decision that will guide the course.

My early conversation moved through several possible projects. Students might reconstruct disagreements, keep a calibration journal, examine real decisions, or study their own use of AI. Each possibility highlighted a different meaning of critical thinking. One emphasized argument. Another emphasized prediction. Another emphasized judgment. AI was useful because it could hold these options next to one another and compare their strengths.

This is where many course-design conversations go wrong. We ask AI for “the best assignment,” and it provides one. The answer may sound convincing because AI is good at giving coherence to whatever it has just proposed. But coherence is not the same as fit.

In adviser mode, the instructor should resist early closure. Ask for several genuinely different approaches. Ask which educational goals are in tension. Ask what each option teaches well and what it neglects. Ask how the assignment would change for beginners, advanced students, large classes, or limited class time.

The rule is to use AI to widen judgment, not replace it.

Adviser mode must also end. AI can brainstorm forever. More possibilities do not always lead to a better course. At some point, the instructor has to choose a central project and state which constraints are fixed.

That decision begins developer mode.

The developer does not keep reopening the main question. It takes the chosen direction and turns it into a teachable structure. It identifies stages, dependencies, supports, deadlines, common materials, and likely points of failure. It asks what students must know before they can complete each step. It also asks what the instructor can realistically support.

In my case, the central idea became a first-person study of AI use. Students would document selected AI sessions, record their expectations, use think-aloud methods, and analyze patterns in their own judgment. Their final product would be a chapter in a class-edited digital volume.

Once that direction was fixed, new questions became possible. How many observations would students need? Which parts of the method should be common across the class? How could students produce distinct chapters if everyone followed the same protocol? What should be submitted before the final paper? How could AI assist with coding without taking over the analysis?

These were development questions, not brainstorming questions.

AI helped convert the broad idea into a sequence: proposal, pilot, sample analysis, draft, and final chapter. It also helped reveal that variation should come from the research question rather than from the method. A common protocol would make demonstrations, peer support, and grading more manageable. Different questions would prevent the final volume from becoming repetitive.

The wider rule is to standardize what reduces confusion and vary what creates intellectual ownership.

Developer mode should produce an artifact. It might be an assignment sheet, course map, rubric, schedule, or draft syllabus. Without a concrete artifact, the conversation remains too easy. General ideas can appear strong because they have not yet encountered dates, class size, student preparation, or instructor workload.

This is also the stage where AI should be asked to simplify. Course design often becomes more elaborate during development because every useful idea invites another useful idea. A promising assignment can quietly turn into a research program, a publication project, a new assessment system, and an administrative burden.

Ask what can be removed without damaging the main learning goal. Ask which feature creates the most work for the least educational value. Ask for the smallest complete version of the design. AI is not naturally restrained. It needs to be assigned restraint as part of the job.

Critic mode should begin only after the design is visible.

The critic should not continue helping the instructor elaborate the plan. It should evaluate the plan as if it came from someone else. This separation is important because AI tends to preserve conversational momentum. After helping build an idea, it often continues to explain why the idea is sound.

I had to change the instruction directly: stop developing the project and look for serious weaknesses.

That shift exposed several problems. Students could produce polished reflections without reliable evidence. AI-assisted analysis could smooth over uncertainty instead of revealing it. A shared method could lead to repetitive papers. An ambitious research component could create ethical and administrative work that the course did not need. A flexible reading schedule could disadvantage students with weak time-management skills.

These criticisms were useful because they arrived after the idea had enough structure to be tested.

The rule is to request diagnosis before revision. Ask for the strongest objections without asking AI to fix them immediately. Some objections will reveal real flaws. Others will reflect tradeoffs you are willing to accept. The instructor still has to decide which criticisms matter.

Useful questions include: Where does this plan depend on unrealistic student behavior? What could students do that technically satisfies the assignment while defeating its purpose? Which part will be hardest to explain? What will be hardest to grade? What will fail first in a class of forty? Which feature exists mainly because it sounds impressive?

Critic mode should also test the course against its formal obligations. Does the project assess the stated learning outcomes? Do the readings and activities prepare students for the graded work? Does the grading structure reward the kind of thinking the course claims to value? Does the schedule leave time for revision? Can the instructor explain the design to students in plain language?

The three roles should not be combined in a single prompt. “Brainstorm, build, and critique a course” sounds efficient, but it weakens all three tasks. Brainstorming needs openness. Development needs commitment. Criticism needs distance.

The course remains the instructor’s intellectual responsibility. AI can broaden the options, give structure to a decision, and expose weaknesses in a draft. It cannot decide what is worth teaching, what risks are acceptable, or how much complexity a particular instructor and group of students can carry.

AI Summaries Are Not Killing Reading. They Reveal What Reading Already Is

Naomi S. Baron, a linguist who studies technology and literacy, worries that AI is making books feel obsolete. Her concern is easy to understand. When a chatbot can reduce a book to ten points, students may decide that reading hundreds of pages is an inefficient way to collect information. Baron argues that AI may weaken our motivation to read for analysis, empathy, escape, and pleasure. In her view, letting a machine read for us risks weakening the habits that reading develops.

This argument identifies a real danger, but it also treats AI summarization as more novel than it is. Education has always placed summaries between readers and difficult texts.

A scholarly introduction to a classic novel tells us what historical background matters, identifies the main themes, explains difficult references, and suggests how the work should be interpreted. A professor does much the same in a lecture. Few lecturers read an entire article aloud. They select its central claims, translate technical terms into familiar language, supply examples, and omit details that do not serve the course. Textbooks summarize fields. Literature reviews summarize studies. Review essays summarize books. Even a table of contents is a small prediction about what deserves attention.

We do not usually accuse the professor who explains Kant in ordinary language of destroying philosophy. We do not complain that an introduction to The Republic has prevented us from reading Plato. We understand these forms as intellectual support. They reduce the cost of entry.

AI summaries differ in speed, scale, and uncertain accuracy, but not in their basic educational function. They compress and translate. They can provide a map before we enter unfamiliar territory. I have argued elsewhere that educators should not hide from students that AI can be a powerful tutor. A tutor who summarizes a difficult chapter is not necessarily replacing reading. The tutor may be making reading possible.

The deeper problem with the claim that summaries replace reading is that much nonfiction reading is itself a form of summarization. We rarely read a policy report, history book, or research article in order to memorize every sentence. We read to construct a smaller mental representation that we can retain and use.

Walter Kintsch and Teun van Dijk described text comprehension as movement between local propositions and a global “macrostructure.” Their model includes mental operations that reduce the information in a text to its gist. These operations are shaped by the reader’s goals. In plain language, understanding a text requires deciding what matters, combining related points, and discarding much of the rest.

Memory research supports this view. Jacqueline Sachs found that people quickly became less able to recognize changes in the exact wording and syntax of passages, even when they retained their meaning. Fuzzy-trace theory makes a similar distinction between memory for exact form and memory for gist. Adults often rely on the simplest level of meaning that is sufficient for the task. We preserve the substance while losing many of the words that carried it.

The summary, then, does not arrive after reading. It is one of the products of reading.

Human readers are compressors with uneven settings. One reader sees a central argument where another sees an interesting detail. Prior knowledge changes what we notice. Goals change what we retain. Fatigue changes everything. Human summarization is slow and inconsistent, although it is not always less accurate than machine summarization. AI is faster and more repeatable, but it can omit qualifications, flatten disagreements, or present an error with perfect confidence.

The relevant comparison is not between rich human reading and lifeless machine compression. It is between different forms of compression, each with its own strengths and failures.

This does not mean that all reading can be replaced by summaries. Reading a novel for pleasure is not mainly an information-retrieval task. Style, rhythm, suspense, voice, and emotional sequence are part of the experience. A summary of a joke is rarely funny, which may be the shortest available proof that compression has limits.

Language learners also need contact with actual sentences. Close reading requires attention to word choice and form. Legal, medical, and scientific work may depend on qualifications that disappear in a short account. In a randomized study of 195 participants, AI reading tools produced different effects for different readers. Summaries could help some lower-performing readers, but they reduced comprehension among stronger readers, apparently because details and nuance had been removed.

The proper question is therefore not whether a summary counts as reading. The question is what kind of reading a task requires.

Sometimes we need orientation. Sometimes we need a central claim. Sometimes we need to check whether a document deserves closer attention. An eight-week study of college students found that some students used AI summaries to decide which parts of assigned texts merited deeper reading. This practice can become passive, but it can also support triage in a world where no scholar can read every relevant source from beginning to end.

We should teach students to move between levels of compression. They can begin with an AI overview, inspect the original text, compare the two, identify what was lost, and revise the summary. That process makes summarization visible. It turns a hidden mental act into an object of judgment.

The educational task is not to protect every page from compression. It is to teach students when compression is the work, and when the missing page is the point.