Free, no account. Daily levels are open data.
TRADING TOOLS

How to Check an AI Market Summary Against the Data

Four checks that catch the failures an AI-written market summary actually makes, including the one built entirely out of true numbers.

The short answer

Four checks, in the order that catches the most.

  1. Symbol attachment. Is each number attached to the ticker it belongs to?
  2. Presence, at the precision written. Does each number appear in a source the writer could have seen, to as many digits as it printed?
  3. Shape. Is the sentence describing something that happened, or predicting something, or explaining a cause nobody published?
  4. Date. Does the reading say when it was taken?

They apply to any automated market summary, including this site's. The order matters because the first one catches the failure that looks most like a correct sentence.

Check one: the number is real and the ticker is wrong

The failure people expect from a language model is an invented number. The failure that survives review is the opposite: every number in the sentence is real, and one of them is attached to the wrong symbol.

"AMD is trading at $950.50" is a fabrication in which nothing was fabricated, if $950.50 happens to be NVDA's price in the same data. A reviewer scanning for made-up figures finds nothing, because there is nothing made up. Only a check that binds each number to the symbol it was published under sees it.

This is worth running first on any AI-written market recap, because a mixed-symbol sentence is the most quotable kind of wrong: it is specific, it is plausible, and it is built out of true parts.

Check two: precision beyond the source

A summary that writes more decimal places than its source holds has invented the extra digits.

If the underlying reading is 950.5, then 950.53 is not a rounding of it. It is a number the source never contained, and the last digit came from somewhere else. The same applies to percentages: a source holding 3.2 can support "3.2 percent" and cannot support "3.24 percent", because the hundredths digit was never in the source to round.

The useful version of this check is a rule rather than a feeling: a figure is supported when it matches a source value to within half a unit of the last decimal place it actually wrote. More digits than that is a fabricated decimal, regardless of how close the value looks.

Arithmetic over source numbers is a separate and legitimate case. A difference between two published figures, a percent change between them, or a ratio of one to the other, are all derived from data rather than invented. Anything outside that short list is authored.

Check three: the shape of the sentence

Some sentences are unpublishable no matter how well grounded their numbers are, because of what they claim rather than what they cite.

  • An instruction. A level paired with a direction is a trade recommendation wearing a description's clothes.
  • A precedent with no source. "The last three times this happened" is a claim about history that either has a citation and a sample size or does not exist.
  • An invented cause. Attributing a move to a reason is an explanation, and explanations require evidence that the reason was the cause rather than a coincidence in the same session.
  • A forecast. A statement about what price will do next is not a reading of data. No amount of correct data in the sentence makes it one.
  • An impossible magnitude. Billions of dollars of premium in a single contract is arithmetic nobody checked.

Each of those can be spotted by a reader in one pass, and none of them depends on having the underlying data.

Check four: the date

A market figure without a timestamp cannot be verified, only believed. A reading dated to a session and a capture time can be compared against the source; the same reading with no date cannot be distinguished from one taken last week.

This is the check that most quietly separates a tool from a demo. Live numbers age, and an undated figure is the same object whether it is two minutes old or two months old.

How this site applies it to itself

Narration here passes two gates before publication, and the description below is of the mechanism rather than a claim about the result.

The first gate reads a sentence on its own and refuses the shapes in check three, returning a machine-readable reason: price instruction, unsourced precedent, invented cause, unsourced prediction, implausible magnitude. It cannot tell a real price from an invented one, because it never sees the data the sentence was generated from.

The second gate does. Every number and ticker in an enhanced sentence is matched against the bundle of data that sentence was generated from, with formatting and sign stripped, rounding allowed only to the precision the output wrote, and derivation limited to differences, percent changes and ratios of two bundle values. Derivations are attempted only after the symbol-attachment check, so a price that belongs to another ticker cannot be rescued by a coincidental arithmetic match. A span with no support drops the sentence carrying it, with a reason recorded. A whole narration that fails falls back to the factual template it was built beside.

The property that makes it worth describing is the failure direction. When the gate cannot verify something, the outcome is less narration rather than unchecked narration.

What none of that establishes is that any given sentence is right. A gate can only compare a sentence against the data behind it, and if that data is wrong, a grounded sentence repeats it faithfully. Errors that reach the site are logged on the corrections page, and the research published here, including the results that did not work, sits in the Lab.

The one-line version

Ask of any automated market sentence: which source, which symbol, which timestamp, and is this describing or predicting. Four questions, and most of what fails fails on the first two.

Educational content, not financial advice. See our risk disclosure.

COMMON QUESTIONS

How do you check whether an AI market summary is made up?
Four checks, in the order that catches the most. Is each number attached to the right symbol, is each number present in a source at the precision written, is the sentence describing something or predicting it, and is the reading dated.
What is the hardest AI market error to catch?
A real number attached to the wrong ticker. Every figure in the sentence exists somewhere in the source data, so a check that only asks whether the numbers are real passes it.
Does more decimal places mean more accurate?
The opposite, when the source does not carry them. A summary writing 950.53 against a source that holds 950.5 has invented the last digit, and precision beyond the source is a reliable tell.

This guide explains the idea. The page below carries today’s numbers. See today’s SPX dealer gamma levels.

Track this on SquawkFlow

Real-time options flow, GEX dashboard, dark pool alerts, and AI narration, free.

Open Terminal →