What are humans actually good for now? And what will we be good for in five years?
Every group of knowledge workers I know is circling that question, and my book club got there last night, discussing Richard Powers’ novel Playground (I won’t give away the ending, but AI, humanity, and reality are major themes). Around that table you’d have found a pretty good cross-section of modern professional life: a physician, a research scientist, a couple of entrepreneurs, a technologist, and whatever I am.
The usual answer is to draw a protective line around creativity, taste, or empathy. That line didn’t survive the evening. Someone mentioned a friend who genuinely likes AI-generated poetry (not ironically, either; it actually moves them). Put that next to a JAMA study where clinicians rated chatbot answers to patient questions as more empathetic than doctors’ answers, and “machines do logic, humans do art and feelings” starts looking pretty shaky.
I left convinced we’re using the wrong map. The real dividing line comes down to two questions. Is “good” a pass/fail bar or a balancing act? And can a machine check the work? The first is the difference between satisficing and optimizing. The second is where tacit knowledge lives: everything we implicitly know and trade off but have never thought to write down.
The satisficing machine
Herbert Simon (economist, AI pioneer, and one of the very few people with both a Nobel and a Turing Award) coined satisficing, a mashup of satisfy and suffice, to describe how people decide when the world is too big and messy to calculate a single best answer.
An optimization problem asks you to weigh a bunch of competing variables and find the best outcome across all of them. A satisficing problem skips that. You set a “good enough” bar ahead of time, look at options one by one, and ask a binary question: Does this clear the bar? The first “yes” wins.
Today’s AI agents are good at satisficing and bad at open-ended optimization. They lack common-sense grounding, institutional memory, and a feel for how the real world fits together, so drop one into a situation where it has to balance five unstated priorities, and it’ll wander off a cliff.
Put that same agent inside a deterministic harness (a fixed set of automated checks that gives the same verdict every time), though, and it can run loops all day (literally - our book club host was rebuilding his startup’s back end, unsupervised, while we were sitting). That’s why coding agents were the first agentic workflow to really break out. At the tactical level, writing code is one of the cleanest satisficing environments we’ve ever invented: compilers, type checkers, linters, and test suites tell you right away if you broke the rules. The agent doesn’t need to care about the long-term health of your codebase, your roadmap, or your vision. It just edits and re-runs until the red FAIL turns into a green PASS.
Code runs in binary; engineering lives in trade-offs
And yet, if you’ve spent the last year in a codebase heavy on AI-generated commits, you know the honeymoon ends around month three (in my experience, anyway!).
Here’s the thing. Writing code that runs is a satisficing problem. Software engineering is an optimization problem.
When an experienced engineer builds a feature, getting the tests to pass is the easy part. Underneath, they’re weighing concerns nobody wrote in the ticket. How hard will this be for a tired teammate to debug at 2 a.m. six months from now? Does it solve the problem the way the rest of our system does, or did we just invent a fourth way to manage state (and who invented the third)? Given where the roadmap is probably headed next quarter, does this schema paint us into a corner?
An agent feels none of that friction. If a constraint isn’t wired into its test runner or linter, the constraint doesn’t exist. Left alone, it’ll happily write a brittle 400-line function that duplicates three existing helpers, and as far as the harness is concerned, the job is done.
That’s why I think the current enthusiasm for Spec-Driven Development (SDD) and frameworks like BMAD (Breakthrough Method for Agile AI-Driven Development) misses the mark. Both rest on a comforting premise: if the agent writes messy code, it’s our fault for not specifying clearly enough. Write a thorough PRD, generate the architecture files, split the user stories into agent-sized chunks (BMAD calls this “sharding”), and the agent’ll hand back exactly what you had in mind.
It won’t, because a spec can’t push back. An agent treats every casual phrase in the doc, like “should be configurable” or “should be extensible,” as a literal box to check. It builds elaborate abstractions to satisfy the text on the page and misses the much simpler thing you actually needed. A human engineer would’ve read the same requirement and said, “Wait, if we build it that way, it’s going to take three weeks because of how billing works. What if we do this instead?”
To be fair, the better SDD practitioners would say the spec is a living document, revised in conversation with the agent as the work unfolds. That’s a real improvement over waterfall. It still assumes we know, in advance, which of our assumptions matter enough to write down. Most of the time we don’t.
I might have an unconscious theme on this Substack. Whether I was writing about Better Conversations over Perfect Docs, comparing Claude Code to offshoring in Agents, Copilots, and the Return of Waterfall, or looking at why leaders lose touch with the front lines in The CEO Scale Problem, I keep circling back to the same problem: tacit knowledge. A document is not an understanding. The map is not the territory. You can’t serialize everything a team knows, cares about, and trades off into a static file, whether you’re handing it to an offshore dev shop, a new VP, or a $200-per-month Claude Max plan.
Building a tighter box
The same split (a binary check on the surface and tacit trade-offs underneath) shows up in most of the enterprise workflows where companies are deploying agents right now.
In customer support, closing a tier-one ticket is binary: check the refund table, hit the Stripe API, get a 200 OK. But a good support lead knows when to waive a rigid $12 return fee to keep a $10,000 customer happy, and notices when five separate “resolved” tickets actually point to a broken checkout page.
In finance, matching a purchase order to an invoice is a clean pass/fail check. The judgment is knowing which three-cent discrepancy is a rounding quirk worth ignoring so a critical vendor gets paid on time, and which technically compliant invoice means a partner is gaming the commission rules. (Security triage and data extraction work the same way; I’ll spare you.)
The teams getting good results take one piece of their tacit judgment at a time and turn it into a new binary check: stricter types, custom linter rules, dollar-variance tolerances, confidence thresholds. You don’t turn the agent into an optimizer. You just build a tighter box.
The benchmark illusion
This lens also explains one of the weirdest puzzles of the last decade: why AI keeps face-planting in fields where smart people assumed it would be a slam dunk.
Ten years ago, Geoffrey Hinton said people should stop training radiologists, because it was “completely obvious” that deep learning would outperform them within five years. In 2026, hospitals are still short of radiologists (Mayo Clinic has grown its radiology staff by 55% since then), and diagnostic AI is mostly an assistive tool.
What happened is what I call the benchmark illusion. Researchers took a messy, high-context job, stripped away everything that happens before and after the scan, and froze it into a static dataset with yes/no labels (Stanford's CheXpert, about 224,000 chest X-rays labeled by software that read the radiology reports, is the famous one). Inside a lab benchmark, radiology looks like pure satisficing: Look at this grid of pixels and tell me if this lung is diseased.
Flatten reality like that, and two things go wrong the moment the model leaves the lab.
First, the model cheats. In a couple of well-known audits, high-scoring radiology models weren’t actually reading lung tissue. Models trained to spot collapsed lungs learned to look for plastic chest drain tubes (which meant doctors had already found and treated the problem), and pneumonia models learned to recognize the “PORTABLE” label and other hospital-specific markers on X-rays taken at the bedside of the sickest patients. If your only job is to clear a statistical bar on a test set, a chest tube's a much cleaner signal for disease than a faint shadow on a lung.
Second, a working clinician isn’t just labeling pictures. Modern scanners pick up “incidentalomas” all the time: real anatomical quirks that look strange on film but won’t ever hurt the patient. As the physician in my book group knows firsthand, a doctor’s always weighing the chance of catching real disease against the harm of setting off a chain reaction of invasive biopsies, terrified patients, and burned-out nurses ignoring alerts.
Once you see the gap between a static benchmark and an open world, you see it behind many high-profile AI flops.
Take home flipping. Buying and selling houses looks like a spreadsheet problem until you put an automated cash offer into a real neighborhood. A model can’t smell a damp basement after a rainstorm or hear how highway noise bounces into a backyard at rush hour, but the person selling the house can. When Zillow tried to automate house flipping, homeowners used that local, tacit knowledge to select against the algorithm, happily selling it their problem properties while turning down its lowball offers on great homes. Zillow shut down the business in 2021 after writing down about $304 million in inventory in a single quarter.
Then there’s autonomous driving’s “last 10%” problem. Staying between the painted lines on a sunny highway is a clean-rules game. Getting around a double-parked delivery truck in a Boston snowstorm is a social negotiation. Human drivers keep cities moving by knowing when to bend the rules, like creeping across a double-yellow line after making eye contact with an oncoming driver. Make an AI’s safety rules strictly binary and the car freezes in traffic; loosen them without human social intuition and it drives into wet concrete.
The box keeps growing (and someone has to build it)
Of course, there’s an obvious objection to all of this: the box keeps getting bigger. Teams now use one model to grade another against a written rubric (often called “LLM-as-judge”), and labs are training verifiers that turn fuzzy judgment calls into scores. Even radiology’s shortcut problem turns out to be partly a data problem; a 2021 study found a comprehensively trained commercial model held up even on patients without chest drains. Make enough tacit judgment explicit, the argument goes, and eventually everything becomes satisficing.
Maybe. But we’ve been here before. This is the old dream of symbolic AI: the 1980s expert systems that tried to capture knowledge as hand-written rules (a tension Playground itself keeps circling). That project stalled because experts couldn’t articulate most of what they knew. LLMs flipped the approach, learning judgment from raw data and information instead, and a harness is where the two traditions meet: a learned model inside a box of explicit rules.
Every rule in that box, like every rubric and verifier above, started with a person noticing what was missing, deciding it mattered, and figuring out how to write it down. The box grows because people keep building it.
Two questions to build your intuition
If you’re building a product, running a team, or deciding where to trust an agent this year, you don’t need a PhD in machine learning to guess whether it’ll work. Run the task through two questions:
Can we verify success automatically? Is there a compiler, test runner, schema validator, or rule engine that can check the work without a human having to look at it?
Is “good” a pass/fail bar or a balancing act? Are we just trying to clear a threshold, or are we trading off competing values we haven’t fully written down?
The corner to watch is the benchmark trap: work that looks pass/fail on paper, but nothing in the real world is checking the answer.
So when a vendor brags that their AI tool hits 95% accuracy on a benchmark, skip the question about which model they fine-tuned. Ask what happens on the 5% where it fails, and who has to clean up the mess.
If the compiler catches the mistake and the agent tries again for three cents of compute, ship it today. If a nurse gets alert fatigue, a money-losing house lands on your balance sheet, or an engineer has to untangle an architectural mess six months later, you’re looking at an optimization problem dressed up as a benchmark.
What people are good for
Which brings us back to Playground. In a world full of fast, cheap satisficing machines (some of which write poetry somebody’s friend actually likes), what are we here for? Three things.
First, we own the objective function, the thing we’re actually trying to maximize. An agent will climb whatever hill you point it at, and it’ll take every shortcut available to reach the flag at the top. So far, only people can reliably step back and notice that the metric has drifted away from what we actually care about. Somebody has to notice when closing support tickets fast is costing us customers, or when shipping features fast is rotting the codebase.
Second, we hold the tacit context, and we surface it through continual communication. We hear the hesitation in a client’s voice on a Zoom call, remember why the last rewrite of the billing service blew up in 2023, and know the difference between a rule that can never be broken and one that bends on a snowy Tuesday. You can’t dump all of that into a PRD and expect an agent to execute it, because half the time we don’t know what our real trade-offs are until we argue about them with a colleague.
Finally, we build the harnesses. The most useful technical and operational skill in 2026 isn’t prompt engineering or generating ten-thousand-word spec files. It’s looking at a messy human problem, figuring out which parts of our taste and judgment can actually be made explicit, and turning those parts into clear pass/fail checks that a machine can safely run.
Let the agents run the loops and clear the bars. Our job is to decide which bars are worth clearing, and to keep having the conversations that don’t fit inside the box. At least in 2026.