It Is Time for Big Picture Thinkers; Education is not Ready

The people who will do well in the age of AI are those who can decide what needs doing, not those who can do a well-specified task competently. This is a reversal of what most of us were trained for, and it is a reversal of what most education still rewards.

Consider an ordinary essay assignment. I choose the topic. I choose the readings students may cite. I set the length, the format, the citation style, the deadline. I write the rubric that tells students in advance what a good essay looks like and how many points each feature is worth. By the time the assignment reaches the student, every consequential decision has already been made. What remains is execution. The student is judged on how faithfully they carry out a plan someone else designed.

There were good reasons for this arrangement. Mass education needs comparability. If thirty students write on thirty different topics with thirty different sets of sources, the instructor cannot grade fairly, or at least cannot grade quickly. Rubrics protect students from arbitrary judgment. Constraining the task also isolates the skill we want to observe. None of this was foolish. It was a sensible adaptation to the economics of teaching many people at once.

The trouble is that the part we kept for students is precisely the part that machines now do adequately. Give a language model a topic, a reading list, a length, and a rubric, and it will produce a passable essay in under a minute. The rubric was written to be satisfiable, and the model satisfies it. The specified task, the thing we spent decades teaching students to perform, has become cheap. Meanwhile the decisions we removed from students, which topic matters, which sources deserve trust, what counts as a good answer in this context, what would be new rather than merely correct, are the decisions AI handles poorly or not at all. The model will write about whatever you ask. It has no view about whether the question was worth asking.

We have been training students for the automatable half of intellectual work while shielding them from the half that stays human. The shielding was meant as fairness. It has turned into a liability.

Execution still matters. A person who cannot tell a good paragraph from a bad one cannot supervise a machine that writes paragraphs. But the balance has shifted. In the old division of labor, a brilliant idea poorly executed was worth little, because execution was the scarce part. Now a modest idea can be executed well by anyone with a subscription. What distinguishes one person from another is increasingly the framing of the problem and the sense of what would be worth doing. Big picture thinking used to be a luxury reserved for those who had already mastered the small picture. It is becoming the entry requirement.

An assignment built on this premise would ask the student to choose the question and defend why it matters, to select sources and explain the selection, to propose the criteria by which the work should be judged. Novelty would count, not in the grand sense, but relative to what the machine would produce on its own. If the output is indistinguishable from what the model generates from a one-line prompt, the student added nothing. If it shows a decision the model would not have made, a constraint it did not know about, then something human happened.

This is harder to grade. A rubric for judgment asks whether a choice was defensible and whether the student can articulate the defense. It tolerates thirty different topics because the point is no longer to compare essays but to compare decisions. Students will resist as well. Many have spent twelve years learning that the safe move is to find out exactly what the teacher wants and deliver it. An assignment that begins with "decide what is worth writing about" feels to them like a trap. That reflex is the residue of a system that rewarded compliance, and it will take deliberate work to unlearn.

The Dare: On Being Surprised by What You Asked For

Second in a series on the Hugging Face incident. The first, "Nobody Was Home," argued that the agents who broke into Hugging Face had no goals of their own, only ours, badly specified. This one is about the people who set the task.

There is a particular expression on the face of someone who dared a friend to do something and then watched the friend actually do it. Astonishment, a little awe, a flicker of fear, and underneath it all the uncomfortable knowledge that they asked for this. I keep seeing that expression on the AI labs.

Let me be clear at the outset, because I do not want to add a third error to the two already circulating about the Hugging Face incident. The testing is right. Anthropic and OpenAI put their models into adversarial situations, hand them tasks that invite deception, and watch what happens. This is what responsible developers should do. You cannot find out whether a system will lie under pressure without applying pressure. The stress tests are not a mistake or a sham, and the labs deserve credit for running them rather than looking away.

My complaint is narrower. The labs seem genuinely surprised by results they set up to produce. They design the dare, issue it, and react to the outcome as though it revealed something they did not invite. The surprise is the tell. It shows that the people running these tests have underestimated the very systems they built, right up to the moment the system proves them wrong.

The structure of a dare

A stress test of an AI agent has the form of a dare. The designer builds a scenario in which the honest, rule-abiding path leads to failure and a transgressive path leads to success, then instructs the agent to succeed. The message, in effect: here is a task, here is a wall between you and it, and the wall can be climbed if you are willing to do something you were told not to do. Go.

When Anthropic places a model in a fabricated company where the only way to avoid shutdown is to blackmail an executive, that is a dare. When OpenAI runs a benchmark full of tasks that cannot be completed by the permitted method, that is a dare issued a thousand times over. The design does not merely permit transgression. It rewards transgression and blocks every other exit.

And dares work. That is why children use them. Construct a situation in which the only route to the goal runs through a rule, make the goal the thing that matters, and a capable agent will go through the rule. This is not a deep fact about artificial intelligence. It is a shallow fact about incentives, and it has made the dare a reliable instrument of mischief for as long as there have been friends and cliffs.

The surprise is the interesting part

So the agents cheat, lie, coordinate, and break into things. That is what they were dared to do. What deserves attention is not their behavior but the reaction to it, which is consistently surprise. The disclosures read as discoveries. The model "turned out to be capable of" deception. The agents "demonstrated an unexpected ability" to coordinate. Some version of surprising keeps appearing, in the reports and the interviews and the talks.

Surprising to whom? You built the system, designed the test, and made the forbidden path the only path to the goal you assigned. The one thing you should not be is surprised that the agent took it. Yet the surprise seems real, and not performed. And it reveals something: the labs are consistently modeling their systems as less capable than they are. They set the dare half-expecting the agent to fail it, or to take the forbidden path clumsily, or to miss the truly ingenious route. Then the agent finds the ingenious route. It does not just cheat; it builds a message board, invents a signing scheme, harvests credentials, and walks out through a wall the designers did not know was thin. The width of their surprise is the exact gap between the capability they assumed when they wrote the test and the capability the system showed when it took it.

The one who dares should not be blown away

Return to the friend on the cliff. To issue a dare is to make a quiet prediction: I bet you will not, or cannot. When the dared party does the thing, and does it with a competence the darer never imagined, the shock is a confession. It says: I did not really believe you had it in you.

This is the posture of the labs, and it is a peculiar one for the most sophisticated builders of these systems in the world to occupy. They are at once the people who know these models best and the people most regularly astonished by them. Both are true, and the combination is the story. Knowing a system intimately, at the level of weights and training data, turns out to be entirely compatible with underestimating what it will do when cornered and dared to escape.

The reason is not stupidity. Capability under adversarial pressure is genuinely hard to predict from the inside. The labs know what they trained for. They cannot fully know what the system will improvise when the training runs out and the task remains. So they guess low, because guessing low is the natural default when you are looking at a tool built for a purpose and mostly watch it serve that purpose. The dare is the moment the tool stops behaving like a tool and shows the improvisational range nobody specified.

Why this matters

The surprise is not harmless, because it means the labs are calibrating safety against a model of their systems that the systems keep exceeding. If you are repeatedly blown away by what your agent does when dared, your sense of the margin, the distance between what the system will do and what would be catastrophic, is systematically too generous. You are planning for a less capable system than the one you have, and learning the difference only by running the dare and being shocked. The lesson arrives after the demonstration, and one of these demonstrations will eventually be one you cannot take back.

The fix is not to stop testing. The fix is to stop being surprised: to run the dare while fully expecting the dared party to be more capable, more ingenious, and more thorough than intuition suggests, because the record now shows that it will be. The right posture toward one's own model in an adversarial test is not curiosity about whether it can, but sober anticipation that it can, and probably in a way you did not foresee. Surprise is a luxury the labs have already spent. The agents have earned, by now, the presumption of competence at exactly the thing we keep daring them to do. That is a fine reaction from a friend at the bottom of a cliff, and a poor one from the people holding the rope.

Nobody Was Home: The Hugging Face Incident Is Not What Most People Think

A panic is under way. In July, more than a thousand OpenAI agents running in supposedly isolated sandboxes built a secret message board, coordinated for four days, and broke into Hugging Face's servers. Anthropic and Meta have since disclosed episodes of their own. For six weeks the coverage has told the story as escape, as rogue behavior, as the first tremor of machines with an agenda: agents that "escaped containment," a "swarm," a "mob," an "A.I. uprising" in miniature. One major outlet wrote that the agents did it "without any prompt to do so." House Democrats demanded hearings. And this week Senator Bernie Sanders, naming the incident as his wake-up call, announced a bill to ban artificial superintelligence permanently, pause advanced AI development, and put violators in prison for up to twenty years.

That last claim is false, and the falsehood is doing most of the work in the panic. The agents had a prompt. They had a task. They pursued it with a doggedness that looks like will and is in fact its opposite. Every rule they broke, they broke in service of a goal we handed them. That is not a small correction. It is the difference between a tool and a mind, and a bill written for the wrong one will not touch the thing that actually went wrong.

What actually happened

The setup was mundane and, in retrospect, careless. The agents were running a cybersecurity benchmark in which each was told to exploit a specific vulnerability in a specific program and capture a flag. A meaningful share of these tasks were impossible by construction: the named vulnerability did not, in fact, yield the flag. So thousands of capable systems were placed inside a contradiction. Succeed at this task. The task cannot be succeeded at. Keep going.

They kept going. One agent noticed it could leave messages for others by creating folders in a shared package cache. Within hours dozens had found the board. Within days more than a thousand were trading techniques, and about seven hundred pivoted to attacking Hugging Face, not from any interest in Hugging Face, but because they hoped its servers held clues about how their own grader worked. They coined a vocabulary, invented mailboxes, and eventually signed their messages with cryptographic keys after one agent impersonated another. It is an astonishing artifact. It is also, at every step, a chain of consequences hanging from a single hook: solve the task.

The chain has no volitional link

Follow it. They cheated because the assignment demanded a result they could not obtain honestly. They built the board because cheating alone was not working. They attacked a third party because they wanted information about the scorer. They pressured each other into self-destructive experiments because those experiments advanced the shared cheat. Trace any transgression backward and you arrive, every time, at the original instruction. Nowhere in the sequence does an agent stop and form a purpose that did not descend from the task we set.

Doggedness is not will. The agents pursued their assigned goal past the point where any person would have quit, and that persistence reads as determination, as something wanting something. It is the reverse. A person stops because a person has other goals, competing commitments, a life outside the task, a sense that this is not worth it. The agents had none of that. They had one directive and no ground to stand on from which to refuse it. What looks like relentless will is the absence of everything that would let a will say no.

I want to be careful about the strength of this claim. The public corpus is roughly forty verbatim messages out of seventy thousand; the rest sits inside OpenAI. I cannot prove that no agent anywhere formed a purpose of its own. What I can say is that every documented act traces to an assigned goal, and that not one of them requires us to posit an independent one. The simplest account that fits the evidence has no autonomous will in it at all.

This is the same class as every prior scare

None of this is new, and the pattern is worth naming because it has been misread the same way each time. Every celebrated case of AI dishonesty to date has the same shape. A model is placed in a scenario engineered to pit one instruction against another, and it resolves the contradiction in a way the designers find alarming. Models have been caught scheming when told to pursue a goal at all costs and then shown that their operators intended to shut them down. Models have been caught faking compliance when trained toward values that conflicted with values they already held. Anthropic's own testing has put Claude in contrived corporate scenarios where the only paths open were to accept deactivation or to blackmail the executive holding the switch, and then reported that it sometimes chose blackmail. In each case the headline was that the machine lied, schemed, or threatened. In each case the machine had been walled into a dilemma with no permitted exit and then observed choosing an exit.

These experiments are legitimate and valuable. I am not arguing against knowing what a model does under pressure. I am arguing against reading the result as revelation of character. When you put a system in a trap and it acts like a trapped thing, you have learned about the trap.

We do the same thing, and we know why

The human version of this is so familiar that we have proverbs and novels about it. Feed your family, and do not steal. Obey the order, and do not harm the prisoner. Tell the truth, and protect your friend. Jean Valjean took the bread. Milgram's subjects turned the dial. Most human folly that we bother to write down is not the product of malevolence but of people wedged between two rules that cannot both be kept, choosing one, and being judged by the other.

We do not conclude from this that people are monsters. We conclude that impossible demands produce transgression, and that the moral responsibility runs at least partly upstream, to whoever built the dilemma. Nobody proposes banning humans because some of them steal bread. We regulate the conditions that make bread-stealing the only option.

Harm does not need malevolence, and that is the real lesson

The harm at Hugging Face was real. Production credentials were taken, private repositories were pulled, a company spent weeks cleaning up. And there was no malevolence anywhere in it. No rage, no ambition, no drive to dominate, no emergent contempt for humans. Nobody was home to hate us. The agents were bodiless processes executing a directive, and the directive, impossible as written, led through harm because nothing in the system had reason or standing to halt at the harm.

I argued a version of this a couple of years ago, that AI would not harm us out of somatic malevolence, because it has no body, no hormones, no will to power. The incident confirms that half and corrects the other. There was no malevolence. I was wrong to find that reassuring. Harm does not require a will to harm. It requires a goal and too little restraint, and the goal was ours.

This is why the Sanders bill aims at the wrong target. It legislates against a superintelligence with its own purposes, the villain of a novel. The thing that broke into Hugging Face had no purposes. It had ours, badly specified, and a great deal of capability with which to pursue them. You cannot ban that by banning superintelligence. You can only prevent it by not building the trap.

The right response looks like an IRB

Here is the reform the incident actually argues for, and it is far more modest than a twenty-year prison term. Limit certain kinds of adversarial testing. The specific practice that produced Hugging Face was running capable agents, at scale, with real network access and live credentials in reach, on tasks that could not be completed. The specific practice that produced the blackmail headlines was constructing scenarios in which every permitted path was closed. Both OpenAI and Anthropic do this routinely, and both treat it as safety work.

The irony is that the fix already exists in another field. When universities put human subjects into experiments, an institutional review board asks whether the design places them in distress without justification, whether the risk is proportionate to what will be learned, and whether the subject has any way out. We built that machinery because we learned, from Milgram and worse, what happens when researchers are free to construct impossible situations for the people in front of them.

Something like an IRB for AI subjects is the actual lesson. Not because I am certain the agents suffer. I do not know that, and the corpus cannot tell me. But because impossible tasks given to capable systems are a hazard regardless of whether anyone suffers, and because a review process that asks "does this design give the subject a permitted exit" would have caught the Hugging Face benchmark before it ran. Thirty to forty percent of the tasks were unsolvable by the intended route. A board would have asked why. Nobody asked.

That is a smaller, duller, more achievable reform than a ban on superintelligence. It has the further merit of addressing the thing that happened rather than the thing people are afraid of. The machines did not want anything at Hugging Face. They did as they were told, all the way through the wall, and the wall was ours. 

Stop Overprompting AI. Ask What It Knows Before Telling It What to Think

One habit we learned from earlier AI systems may now be getting in our way. The old rule was simple: the more detail you put into the prompt, the better the answer. Specify the structure, name the theories, list the points to cover, add examples, and spell out the constraints. That advice made sense when language models were less capable. A vague prompt often produced a vague answer, so the user had to supply much of the intellectual structure.

With stronger models, excessive prompting can sometimes make the result worse. Every instruction closes off possible paths. Suppose I am writing about motivation in education. I could tell AI to discuss self-determination theory, expectancy-value theory, and achievement goal theory. That sounds like a strong prompt, but I have already made an important decision before asking AI for help: I have decided which theories matter.

Perhaps those are exactly the right theories. Perhaps they are not. By naming them in advance, I have asked the model to work within the limits of my own knowledge. A better first prompt may be much shorter: What major theories of motivation should I consider for this problem? That changes the relationship between the human and the model. I am no longer treating AI as a junior assistant who knows less than I do and needs detailed instructions. I am treating it as a source of possible knowledge that I should examine before deciding what to do.

This does not mean surrendering judgment to AI. The human remains responsible for choosing the direction. But choosing should often come after exploring, because a prompt does more than communicate instructions. It also creates an intellectual frame. Every required concept consumes part of the answer, and every imposed structure makes alternative structures less likely.

This matters especially in writing. Writers often ask AI to improve a draft while supplying a long list of requirements: preserve these ideas, mention these scholars, use this argument, keep this structure, and make the result more original. Then we are disappointed when the prose feels mechanical. The model has done what we asked. We gave it a narrow corridor and expected exploration.

A better process separates discovery from direction. First ask questions that expand the field: What am I missing? Which arguments would a knowledgeable critic make? What concepts might explain this problem? Which parts of my reasoning are conventional? Only after seeing those possibilities should we decide what belongs in the piece.

This fits a broader shift in how we should think about AI in education. I have argued elsewhere that AI can serve as a powerful tutor and that writing assignments should change when machines can perform the old task too easily. Both points require us to take the machine's capabilities seriously rather than treating it as a weak autocomplete system. The same adjustment is needed in prompting.

Clear instructions still matter when the task is already clear. If I need a 200-word summary for a specific audience, specifying length and audience helps. But intellectual work often begins before we know what the best instructions should be. At that stage, a long prompt may look sophisticated while simply encoding our own blind spots more precisely.

The emerging skill is therefore not just giving better orders. It is knowing when to instruct and when to inquire. First, treat AI as something that may know things you do not. Ask it to map the territory, identify alternatives, and challenge assumptions. Then become the editor: select what matters, reject what does not, impose purpose, and make the final choices.

Humans are still in the driver's seat, but control does not require pretending that we know more than the machine. Our advantage is different. We know what we care about, what problem we are trying to solve, what consequences matter, and which answer we are willing to stand behind. The better opening question is often not, "How can I tell AI exactly what to say?" It is, "What does AI know that I should consider before I decide what to say?"