01
Why GitHub stars fool almost everyone
A GitHub star is one click. Someone saw a repo, thought "neat", and bookmarked it. That's the entire signal. It says nothing about whether they installed it, whether it worked, whether they're still using it next week, or whether anyone is running it in production. Stars measure popularity at a single moment — not adoption over time. Tens of thousands of stars can sit on top of almost no real usage. So when you pick a tool because it "has the most stars", you're optimizing for the one number that's easiest to game and hardest to connect to whether the thing actually works.
02
The 4 signals that actually tell you the truth
Replace the star count with four questions. Each one maps to something you can usually verify yourself — and together they tell you whether a tool has real, durable demand:
- 1. Live usage — are real requests hitting it right now? Not historical hype: actual traffic, today.
- 2. Paying usage — is anyone spending real money to run it? People pay for things that work; free stars cost nothing.
- 3. Retention — do users come back in week two, or vanish after day one? Retention separates a viral weekend from a real tool.
- 4. Production volume — is real work flowing through it, or just demos and tutorials? Volume in production is the hardest signal to fake.
None of these is perfect alone. Read them together: a tool can spike on one and flatline on the rest. The shape across all four is the signal.
03
Where to actually find each signal
The point of this guide is that these aren't abstract — you can check most of them in a few minutes.
| Live usage | Public usage leaderboards (e.g. openrouter.ai/rankings for model/agent traffic), npm/PyPI download trends, Docker pull counts, status-page request graphs. |
| Paying usage | Pricing pages that actually exist, public revenue or ARR mentions, paid-tier waitlists, case studies with named paying customers (not logos with no story). |
| Retention | Cohort/retention charts if published, Discord/community activity that's still alive months in, repeat-contributor graphs on GitHub Insights, churn mentions in reviews. |
| Production volume | Real integration write-ups, incident/status history (you only get incidents when people depend on you), GitHub Issues full of production edge-cases vs. "how do I install this". |
Rule of thumb: if the only number a tool leads with is its star count, treat that as a flag — not a feature.
04
A real example — read it as directional, then verify it yourself
Here's a case that makes the point. Back in May 2026, on OpenRouter's app/agent usage rankings, a lower-starred AI agent quietly overtook a far more popular, far more starred one on live token volume. The popularity contest said one was winning; the live-usage numbers said the opposite.
Important caveat: these rankings are reported third-party and are directional — OpenRouter does not publish a canonical, audited leaderboard, and any single-day snapshot is just that: a snapshot. So don't take a number off a press write-up (or off this page) as gospel. The right move is the one this whole guide argues for: go look at the live data yourself at openrouter.ai/rankings, and weigh it against the other three signals before you conclude anything.
Important caveat: these rankings are reported third-party and are directional — OpenRouter does not publish a canonical, audited leaderboard, and any single-day snapshot is just that: a snapshot. So don't take a number off a press write-up (or off this page) as gospel. The right move is the one this whole guide argues for: go look at the live data yourself at openrouter.ai/rankings, and weigh it against the other three signals before you conclude anything.
The lesson isn't "this tool beat that tool." It's that stars and live usage can point in opposite directions — and live usage is the one tied to people actually running the thing.
05
Your 60-second pre-commit check
Before you bet a project on a tool, run this:
- Ignore the star count entirely for the first pass.
- Find one live-usage signal (a leaderboard, downloads trend, or traffic graph). Is it real and recent?
- Confirm someone pays for it (a real pricing page or named paying customers).
- Look for retention (a community that's alive months in, repeat contributors).
- Check for production volume (incident history, integration write-ups, production-grade Issues).
- Only now glance at stars — as a tiebreaker for attention, never as proof of adoption.
Get the next drop
New AI build guides + the occasional bonus template. No spam, unsubscribe anytime.
By submitting you agree to our Privacy Policy & Terms. Unsubscribe anytime.
You're in — check your inbox to confirm.
Frequently asked questions
Are GitHub stars completely useless?
No — they're a fine signal of attention and discoverability. They're just not a signal of adoption. Use them to notice a tool exists, never to conclude it's widely used or production-ready.
What's the single best signal if I only have time for one?
Live usage you can verify (a public leaderboard, download trend, or traffic graph). It's the closest thing to "are real people running this right now". Pair it with whether anyone pays, if you have a second minute.
How do I check live usage for an AI model or agent?
Public usage leaderboards like openrouter.ai/rankings show relative live traffic across models and apps. Treat the exact numbers as directional (they're third-party and not canonically audited), but the relative shape is informative.
Why does retention matter more than downloads?
Downloads (and stars) measure a one-time action. Retention measures whether the thing was good enough to come back to. A tool everyone tries once and abandons looks great on downloads and terrible on retention.
Isn't 'production volume' hard to see from the outside?
It's the hardest of the four, but not invisible: incident/status histories, integration write-ups, and Issues full of real production edge-cases (rather than install questions) all leak the fact that people depend on it.