{"id":2499,"date":"2013-01-16T14:41:42","date_gmt":"2013-01-16T21:41:42","guid":{"rendered":"https:\/\/www.roughtype.com\/?p=2499"},"modified":"2013-01-19T16:25:00","modified_gmt":"2013-01-19T23:25:00","slug":"game-of-numbers","status":"publish","type":"post","link":"https:\/\/www.roughtype.com\/?p=2499","title":{"rendered":"Fixing Scrabble"},"content":{"rendered":"

\"4501060411_b78d13822f\"<\/a><\/p>\n

Joshua Lewis has run the numbers on letters, and he’s discovered that Scrabble’s point system is statistically suspect. Indeed, he suggests, it may be downright suboptimal. Lewis, a scientist and entrepreneur, developed a software program,\u00a0called\u00a0Valett<\/a>,\u00a0for “determining the appropriate letter valuations in word games.” The program’s algorithm “analyzes the corpus of a game’s legal plays and provides point values for the letters in the game based on a desired weighting of their frequency, frequency by length and the entropy of their transition probabilities.” He ran Scrabble’s tile values through the algorithm and discovered all sorts of anomalies, which he attributes to historical changes in the set of words that can be played legally in the game.<\/p>\n

The big problem seems to be that Q, which Lewis terms an “outlier” (I can attest, anecdotally, to the truth of that description), is substantially undervalued. To bring the game up to statistical snuff, the values of many other letters would, as a result, need to be “compressed.” As the BBC explains<\/a>:<\/p>\n

According to Lewis’s system, X (worth eight points in the current game) is worth only five points and Z (worth 10 points now) is worth six points.\u00a0Other letter values change too, but less radically. For example, U (one point currently) is worth two in the new version, G (two points) becomes three and M (three points) becomes two.<\/p><\/blockquote>\n

But Lewis’s plan is founded on a misunderstanding. By accounting for recent changes in word frequency and transition probability entropy, he seems to believe that he can return the Scrabble scoring system to a statistically pristine state. But that pristine state, that Eden where all the numbers line up, never existed. The points system was a kludge from the get-go, as the analytically minded have long known. The game’s inventor, Alfred Butts, “calculated a value for each tile by measuring how frequently each letter appeared on the front page of the New York Times.” Explains John Chew,\u00a0of the North American Scrabble Players Association,\u00a0“Butts had a selection bias in favour of printed newspaper English which many people have suggested ought to be rectified.” But changing the system at this point, Chew says, with considerable understatement, would inspire\u00a0“catastrophic outrage.”<\/p>\n

It would also make the game less fun, because it would make it more difficult for novices to occasionally beat veteran players. The scoring system’s lack of statistical rigor, it turns out, has the unintended but entirely welcome effect of adding a little extra dash of luck to the game, as Lewis himself points out. The apparent weakness is a hidden strength.<\/p>\n

Let the statistically impure thoughts of Alfred Butts serve as a lesson to us all about the dangers of our current fixation on the analysis of large data sets. Armed with a fast computer, a wonky algorithm, and whole lot of Big Data, a geek will begin to see problems everywhere in our messy human world. And by correcting every statistical anomaly or inefficiency, he’ll not only clean up the messiness, he’ll remove the fun.\u00a0To a statistician, a blank tile has no value. The rest of us know better.<\/p>\n

Photo by openfly<\/a>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Joshua Lewis has run the numbers on letters, and he’s discovered that Scrabble’s point system is statistically suspect. Indeed, he suggests, it may be downright suboptimal. Lewis, a scientist and entrepreneur, developed a software program,\u00a0called\u00a0Valett,\u00a0for “determining the appropriate letter valuations in word games.” The program’s algorithm “analyzes the corpus of a game’s legal plays and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-2499","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/2499","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2499"}],"version-history":[{"count":10,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/2499\/revisions"}],"predecessor-version":[{"id":2510,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=\/wp\/v2\/posts\/2499\/revisions\/2510"}],"wp:attachment":[{"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2499"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2499"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.roughtype.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2499"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}