Most AI advice can't be wrong

There is a kind of AI advice that cannot be wrong. Not because it is right, but because it is built so that no outcome could ever prove it false. It sounds like insight, it arrives with confidence, and it would read the same for your competitor as it does for you. I call it trendslop, and once you can see it, you cannot unsee it.

Trendslop is the deck that says AI will “transform your industry” without naming a single thing it would change on Monday. It is the advisor who is certain about the next eighteen months and was equally certain about the last set of predictions that did not happen. It is the strategy that survives regardless of what actually occurs, because it never committed to anything you could check.

The one test

There is a single question that sorts real advice from trendslop: what would we have to see for this to be wrong?

If there is a clear answer, you are dealing with a claim. You can test it, track it, and hold it to account. If there is no answer, if the advice is compatible with every possible future, then it is not telling you anything. This is not a new idea. The philosopher Karl Popper argued decades ago that a theory which explains every outcome explains none, and that what separates a real claim from a comfortable story is whether it can, in principle, be proven false (Popper, 1959).

Most AI advice fails this test on purpose. Vagueness is not a bug in trendslop, it is the product. A claim specific enough to be wrong is a claim specific enough to embarrass the person who made it, so the market has learned to sell the version that cannot be scored.

Confidence is not accuracy

The uncomfortable part is that confidence, the thing that makes advice feel trustworthy, is almost unrelated to whether it is any good.

The clearest evidence comes from Philip Tetlock, who spent twenty years running the largest study of expert prediction ever done: 284 experts, around 28,000 forecasts, all tracked against what actually happened. The headline result is brutal. The average expert was only marginally better than chance, what Tetlock memorably called a dart-throwing chimpanzee, and the experts were reliably overconfident about their own accuracy (Tetlock, 2005).

The detail matters more than the headline. Tetlock found that the experts who thought in terms of one big idea, the confident, media-friendly “this changes everything” voices, actually scored worse than the cautious ones who juggled many considerations and updated when the evidence moved. The people who sounded the most authoritative were, on average, the least reliable.

Now look at who dominates the AI conversation. It is not the careful ones. It is the single-idea, high-certainty voices, because certainty is what gets shared. Tetlock’s work is a warning label for exactly this moment: the more sweeping and confident an AI prediction sounds, the more you should discount it, not less.

Trendslop has a price

This would be a harmless annoyance if it were free. It is not.

In 2025, S&P Global surveyed more than a thousand enterprises and found that the share abandoning most of their AI initiatives had jumped from 17 percent to 42 percent in a single year (S&P Global Market Intelligence, 2025). A great deal of that spend was authorised on the strength of confident, untestable advice about what AI would do, advice that never specified what success looked like, so no one noticed it was failing until the budget was gone.

And here is the tell inside the tell. S&P’s own reading is that these projects did not collapse because the technology does not work. They collapsed on the operational gap between a proof of concept and production. Which is to say the confident strategy decks were not just untestable, they were pointed at the wrong thing entirely, at the model instead of the the management gap and the demo-to-ship delta where the real work and the real risk sit. Trendslop does not just waste money. It aims the money away from the problem.

Why AI is the perfect habitat for it

Every field has trendslop. AI is an unusually good home for it, for a few reasons that reinforce each other.

The field moves fast enough that no one is ever held to last year’s confident claim, because there is always a newer thing to be confident about. The vocabulary is technical enough to intimidate the person being sold to, so “we will leverage agentic workflows to drive transformation” passes for a plan. The fear of being left behind is strong enough that buyers reward certainty over rigour, because certainty feels like safety. And the people giving the advice are frequently the people selling the tools, which quietly removes any incentive to say the honest, unsellable thing: that it depends, that it might not work, that you would need to test it.

Put those together and you get a market that pays more for confident nonsense than for careful truth. Not forever. But long enough to do real damage to a budget and a career.

Learn to read the words

Trendslop has a house style, and learning to read it is half the defence. The words are not lies exactly, they are placeholders where a claim should be, and you can usually swap them for the honest version in your head.

“We will leverage AI to transform the business” means “we will use AI to change something,” so the useful question is: change what, measured how? “Unlock efficiencies” means “save time or money somewhere,” so ask where, how much, and compared to what. “Agentic workflows” and “end-to-end automation” name a mechanism, not a result, so ask what specifically will happen without a human, and what happens when it is wrong. “AI-first” and “future-proof” describe a posture, not a plan.

None of these phrases are forbidden, and sometimes there is a real claim sitting right behind them. The test is whether the person using them can produce that claim when you ask. If “transform” collapses into a specific, checkable change the moment you push on it, good. If it just produces more fog, you have your answer.

The outside view

There is a name for the most reliable defence against confident nonsense, and it comes from the psychologist Daniel Kahneman. He called it taking the outside view. The inside view looks at your project, your team, your plan, and asks how it will go. The outside view ignores all of that and asks a colder question: how have projects like this one gone, in general (reference class forecasting)?

The gap between the two is where trendslop lives. From the inside, every AI initiative feels like the exception, because the pitch was built to make it feel that way. From the outside, the base rate is what you have already read here: most pilots never reach production, and a growing share get abandoned. Kahneman’s finding was that the outside view is almost always the better predictor, and almost always the one people skip, because it is less flattering and less fun.

So when you are handed a confident forecast about your own AI project, the strongest move is to refuse the inside view for a minute and ask for the reference class. How often do initiatives like this actually succeed, and what did the survivors do differently? A trendslop pitch has no answer, because the base rate would sink it. A real one has already accounted for it.

How to spot it in the room

You do not need to be technical to catch trendslop. You need three or four questions, and the willingness to sit through the silence after you ask them.

• What would we have to see for this to be wrong? If there is no answer, the advice is not a claim, it is a mood.
• What is the base rate? If someone predicts your AI project will succeed, ask how often projects like it actually do, and whether their pitch accounts for it.
• What would change your mind? Tetlock’s better forecasters, the foxes, could always answer this. The ones who cannot are telling you they will believe the same thing regardless of the evidence.
• Show me where this was wrong before. Anyone who has been doing this honestly has been wrong and can tell you about it. Anyone with an unbroken record of confident correctness is selling you the record, not the truth.

Take an illustrative case. A consultancy presents a slick roadmap: adopt their framework and you will “unlock AI-driven efficiency across the organisation.” Run the questions. What would prove it wrong? No answer, the phrase fits any outcome. What is the base rate for these programmes? Unmentioned, and it is not good. What would change their recommendation? Nothing you could name. The deck is beautiful and it is trendslop, and the giveaway was never the design. It was that nothing in it could ever be false.

What I have seen

I will be honest that I am not immune to this, and neither is anyone. Confident, sweeping claims are genuinely pleasant to make, and I have caught myself reaching for one when the honest answer was “I am not sure, we would have to measure it.” What I try to hold to, in my own work and my own writing, is the same test I would put to anyone else: if I cannot say what would prove a claim wrong, I have not earned the right to make it yet. It is a slower way to talk. It is also the only kind of advice I would want to act on if the cost of being wrong were mine.

What to do about it

You cannot out-argue trendslop, because it is built to have an answer for everything. You can only refuse to buy it. Make the falsifiable question a standing part of how you evaluate advice: what would prove this wrong, what is the base rate, what would change your mind. Reward the advisor who says “it depends” and can tell you what it depends on, over the one who is certain. And treat confidence as what it is, a style of delivery, not evidence about the world.

The AI conversation rewards people for sounding sure. The work rewards people for being right, and the two are related far more loosely than anyone selling you a roadmap would like you to believe. The whole discipline, in the end, is refusing to accept a claim that could never have been wrong.

---

Sources

• Karl Popper, The Logic of Scientific Discovery (1959), on falsifiability.
• Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton University Press, 2005).
• S&P Global Market Intelligence, Generative AI shows rapid growth but yields mixed results (2025).
• Daniel Kahneman, Thinking, Fast and Slow (2011), on the outside view and reference-class forecasting.