Key points:
- An academic paper, “Understanding Mobile App Recommendation Dynamics in General-Purpose LLMs,” studied how AI assistants actually pick and explain app suggestions.
- Researchers filtered over 5,000 raw reasons down to 16 real, repeatable ones. Only 6 match traditional ASO metrics.
- Different AI models often disagreed with each other. Gemini also showed more run-to-run inconsistency than GPT-4o, Claude, or Mistral.
- I could not reach the paper’s own full text directly, so these figures come from its abstract and secondary summaries, not a full read.
Most ASO advice for “getting found by ChatGPT” assumes one thing. That the same signals moving a store ranking also move an AI pick. A new study says that’s true for barely a third of what actually matters.
What the study actually found
Researchers filtered over 5,000 raw reasons behind real LLM app picks. They narrowed that down to 16 real, repeatable ones. Of those 16, only 6 match what a normal ASO process tracks. That leaves 10 factors behind an AI’s pick that most ASO work never touches at all.
| Detail | What the study found |
|---|---|
| Raw reasons collected | Over 5,000 |
| Final, repeatable criteria | 16 |
| Criteria matching ASO | 6 |
| Criteria outside ASO | 10 |
Why the disagreement between models matters too
The study didn’t just count criteria. It checked whether models agreed with each other. And with themselves. They didn’t, not consistently. Agreement between different models varied a lot, pair by pair. One model, Gemini, showed more variation across its own repeat runs than GPT-4o, Claude, or Mistral. That’s a real problem if you want one stable target to optimize toward. Winning over one model’s logic doesn’t mean winning over another’s.
Real-world scenario: A productivity app’s team had been tracking their name in ChatGPT’s answers the same way they tracked App Store rank. One check, one assistant, called it “AI visibility.” Reading about this study, they saw the gap. That single check told them almost nothing about a different model. Or even the same model, asked again. They started sampling across several assistants instead. Treating consistency itself as a signal. Not just a single mention.
What this means for the “ASO equals AEO” assumption
A lot of current advice treats App Store rank and AI recommendation as the same work. This study’s own numbers argue against that. Only 6 of 16 real criteria overlap. Most of what actually moves an LLM’s pick sits outside normal keyword-and-metadata work. That doesn’t make ASO worthless here. It makes it incomplete. By the study’s own math, for most of what drives the real result.
What to actually do about it
- Don’t assume your ASO work already covers AI discovery. By this study’s own count, most of what matters for an LLM pick sits outside that overlap.
- Check your standing across more than one assistant. Model disagreement was real here. One check tells you about one model. Not “AI visibility” as a whole.
- Treat one good or bad AI mention as noise, not proof. The study found real inconsistency, even within the same model, run twice.
- Watch for follow-up research on the other 10 criteria. This study maps the gap. Closing it is still open work.
AppRankr’s ASO Inspector reviews your current listing against known ranking signals, a useful baseline while the AI-discovery side of this picture is still being mapped out.