Graphite measures how Opus 5.5 writing changes
Graphite compares Opus 5.5 with human writing on 9,974 topics
Growth agency Graphite’s October 1 update compares ten AI models with human web articles across 9,974 matched topics. Opus 5.5 produced 2,548 qualifying writing tells, against 2,666 for Opus 5. Its em dash rate fell from 2.92 to 0.015 per 1,000 words.
Its overall word distribution moved closer to the human sample. Jensen Shannon divergence dropped from 0.064 to 0.052, a decrease of about 19 percent.
How the comparison was built
The September 16 study began with 10,000 Common Crawl articles predating ChatGPT’s November 2022 launch. GPT 4.1 summarized each article. Models then wrote from those summaries using a fixed prompt and a target length matching the original. The first study aligned 9,984 topics across nine models.
This summary bottleneck keeps subject matter broadly comparable. Different prompts could change the patterns. Older human articles introduce a time gap, and surviving boilerplate may affect comparisons. The findings concern a particular web writing assignment.
The researchers cleaned bylines, cookie notices and inline artifacts before comparison. Their word patterns preserve common words while replacing runs of one to three uncommon words with blanks.
What the measures mean
Tells occur at least twice the human rate after length normalization and frequency filters. Cohen’s d measures standardized feature differences. Jensen Shannon divergence compares word distributions. These are different measurements, not a general writing quality score.
For mannered prose, Opus 5 rated articles from 1,000 topics. The evaluator was not told authorship, but its judgments could still introduce bias. That score rates metaphor and flourish replacing direct statements, rather than reader preference.
Editors can compare drafts on the same brief, record which passages need factual correction and ask readers where the explanation becomes difficult to follow. That would test a newsroom’s actual editing needs instead of assuming one statistic measures all of them.
Individual tells cannot establish who wrote an article.
ByteForward checked Graphite’s released tables against its tell counts and punctuation rates. They agree. We have not independently reproduced the study.
Illustrative journal photograph by Aaron Burden, published on May 7 2016 under the Unsplash License. This archival image does not depict the experiment.
