There are two loud camps: people who insist AI captions are obviously worse, and people who insist nobody can tell the difference. Both camps argue from vibes. We have 2.3 million posts of actual data, so we ran the comparison properly.
The setup
We compared three groups across accounts of similar size and niche: fully manual captions, fully AI-generated captions (one-click, no edits), and AI-assisted captions — generated by Leon AI, then edited by the creator before publishing.
The results
- Fully manual captions: baseline engagement rate.
- Fully AI-generated, unedited: 8% below baseline.
- AI-assisted (generated, then edited): 14% above baseline.
The unedited group underperformed — not catastrophically, but consistently. Generic hooks and slightly-off tone add up. The interesting result is the assisted group: when creators used AI for the first draft and spent their energy on the hook and the close, engagement beat both other groups.
“AI is a terrible ghostwriter and an excellent first-drafter.”
Why assisted wins
Caption-writing has two genuinely hard parts: the first line that stops the scroll, and the last line that prompts action. Everything in between is structure. AI handles structure instantly, which frees the human to spend two minutes on the 20% that actually moves numbers — instead of twenty minutes staring at a blank field.
Creators in the assisted group also published 2.6x more often. Removing the blank-page problem didn't just maintain quality — it removed the bottleneck that kept content sitting in drafts.
How to apply this
- Never publish a one-click caption raw. Always rewrite the hook in your own voice.
- Train your AI on your past top performers — Leon's Brand Voice does this automatically.
- Spend your saved time on the first line. It's worth more than the other 200 words combined.
Alex Whitfield
Data Lead at Leon
