{ "version": "https:\/\/jsonfeed.org\/version\/1.1", "title": "Ivan Baldin, Data Analyst: posts tagged Bayes", "_rss_description": "Data analytics, probability theory, and applied statistics — from the practitioner’s perspective. Ivan Baldin writes about sales funnels, call center performance, conversion analysis, confidence intervals, binomial distributions, Z-scores, and the math behind data-driven decisions. Years of hands-on experience turning raw CRM data into actionable insights for sales teams and business leaders.", "_rss_language": "en", "_itunes_email": "", "_itunes_categories_xml": "", "_itunes_image": "", "_itunes_explicit": "", "home_page_url": "https:\/\/ivanbaldin.blog\/tags\/bayes\/", "feed_url": "https:\/\/ivanbaldin.blog\/tags\/bayes\/json\/", "icon": "https:\/\/ivanbaldin.blog\/pictures\/userpic\/userpic@2x.jpg?1774085044", "authors": [ { "name": "Ivan Baldin", "url": "https:\/\/ivanbaldin.blog\/", "avatar": "https:\/\/ivanbaldin.blog\/pictures\/userpic\/userpic@2x.jpg?1774085044" } ], "items": [ { "id": "3", "url": "https:\/\/ivanbaldin.blog\/all\/a-smarter-way-to-calculate-ratings\/", "title": "A Smarter Way to Calculate Ratings", "content_html": "

Here’s a puzzle. Two employees get rated by their clients on a 1-to-5 scale. Their average scores are identical:<\/p>\n

Johnson — 5, 5, 5, 5, 5, 2, 2, 2, 2, 2. Average = 3.50.<\/b>
\nMiller — 4, 4, 3, 3. Average = 3.50.<\/b><\/p>\n

A tie. Except — it doesn’t feel like a tie, does it? Johnson has ten ratings, all over the map. Miller has four, tightly clustered. Shouldn’t that matter?<\/p>\n

It does. And a fascinating article on EvanMiller.org, “Ranking Items With Star Ratings,”<\/a> shows exactly how. (Judging by the author’s tone, the whole business of ranking things by ratings is far thornier than it looks<\/a>, and he keeps coming back to it.)<\/p>\n

The core problem is simple: a plain average ignores how many votes went into it. A 4.8 rating from 300 reviews is not the same as a 4.8 from 6 reviews. The first one you can trust. The second one — who knows.<\/p>\n

The solution comes from Bayesian statistics<\/a> (whatever that might mean in this context exactly), et voilà:<\/p>\n

Johnson — 2.72<\/b>
\nMiller — 2.63<\/b><\/p>\n

Somehow, either the tighter spread of Miller’s scores (SD of 0.58 vs. 1.58), or the smaller number of ratings (4 vs. 10), or both together refined<\/b> the average, giving Miller the edge by a few hundredths. The tie is broken.<\/p>\n

The formula behind the advanced rating<\/h2>\n

Brace yourself. It’s gonna sting a little.<\/p>\n

We assume<\/a> there are K<\/b><\/i> possible ratings, indexed by k,<\/b><\/i> where each rating is worth sk<\/sub><\/b><\/i> points (“1” = 1 point, “2” = 2 points, and so on). Given N<\/b><\/i> total ratings for each item, with nk<\/sub><\/b><\/i> ratings for each k,<\/b><\/i> we can compute the item’s score using the following formula:<\/p>\n

\n\"\"\n<\/div>\n

Where zα\/2<\/sub><\/b><\/i> is the 1−α\/2<\/b><\/i> quantile of the normal distribution. The resulting score is the lower bound of a normal approximation to the Bayesian confidence interval for the average rating. Setting, say, α=0.10 (z=1.65), the score S<\/b><\/i> means that in 95% of cases, the item’s true average rating will be above S<\/b><\/i>.<\/p>\n

In plain English: the “advanced” calculation gives you a conservative estimate of what the average rating would converge to if you had more data. And — as we saw with Johnson and Miller — it lets you rank items even when their plain averages are identical.<\/p>\n

The advanced rating in action<\/h2>\n

I took 2,000 ratings on a 1-to-5 scale across a set of regional sales offices and computed both the plain average and the “advanced” score for each one.<\/p>\n

\n
\n\"\"\n\"\"\n<\/div>\n
Average 1.0 — plain average rating; Average 2.0 — advanced average rating.<\/div>\n<\/div>\n

Flushing dropped from 2nd to 4th place. The likely reason: its 66 ratings don’t inspire enough confidence that the average is truly that high. In 90% of scenarios, its predicted rating is no better than 4.55 — roughly 4th place territory.<\/p>\n

Tribeca, on the other hand, was formally 13th. But thanks to a solid base of 249 ratings, the model predicts — with 90% confidence — an average of at least 4.4, which bumps it up to 10th.<\/p>\n

My impression is that the formula works most convincingly for short rating scales, like the 1-to-5 used here.<\/p>\n

In any case, here is a Google Sheets workbook<\/a> — it should handle rating scales up to 100, import up to 10,000 rows of individual ratings, and lets you adjust the confidence level (90% in our example).<\/p>\n

See also<\/h2>\n

https:\/\/www.evanmiller.org\/ranking-items-with-star-ratings.html<\/a><\/p>\n

Advanced rating calculator<\/a> in Google Sheets<\/p>\n", "summary": "Two employees, same average rating — but should we trust 10 scattered scores the same as 4 consistent ones? Inspired by Evan Miller's work on ranking items with star ratings, this post walks through a Bayesian approach that accounts for both spread and sample size.", "date_published": "2019-09-21T15:59:00+03:00", "date_modified": "2026-04-12T03:29:09+03:00", "tags": [ "Bayes", "equations", "Google Sheets", "ratings", "statistics" ], "image": "https:\/\/ivanbaldin.blog\/pictures\/ratings00.png", "_date_published_rfc2822": "Sat, 21 Sep 2019 15:59:00 +0300", "_rss_guid_is_permalink": "false", "_rss_guid": "3", "_e2_data": { "is_favourite": false, "links_required": [ "jquery\/jquery.js", "fotorama\/fotorama.css", "fotorama\/fotorama.js" ], "og_images": [ "https:\/\/ivanbaldin.blog\/pictures\/ratings00.png", "https:\/\/ivanbaldin.blog\/pictures\/ratings01.png", "https:\/\/ivanbaldin.blog\/pictures\/ratings02.png" ] } } ], "_e2_version": 4199, "_e2_ua_string": "Aegea 11.5 (v4199)" }