Wednesday, August 23, 2017

'A primal experience: A note on the impossibility of “fairness”


A note on the impossibility of “fairness” – Thomas Miconi
“Various measures can be used to determine bias or unfairness in a predictor. Previous work has already established that some of these measures are incompatible with each other. Here we show that, when groups differ in prevalence of the predicted event, several intuitive measures of fairness (ratio of positive predictions to event occurrence, probability of positive prediction given actual occurrence or non-occurrence, and probability of occurrence given positive or negative prediction) are all mutually exclusive: if one of them is equal among groups, the other two must differ. The only exceptions are for perfect, or trivial (always-positive or always-negative) predictors. As a consequence, any non-perfect, non-trivial predictor can always be portrayed as biased or unfair under a certain perspective. The result applies to all predictors, algorithmic or human. We conclude with possible ways to handle this effect when assessing and designing prediction methods.”

Submitted 5 Jul 2017 to Applications [stat.AP]
Published 6 Jul 2017
Updated 1 Aug 2017



'A primal experience': Americans dazzled by solar eclipse


MEdia Dragon Connects the Dots of Industrial Democracy  which is so so overrated ;-)

The U.S. Spy Hub in the Heart of Australia The Intercept


Fear the rise of info-monopolies over America Fabius Maximus

CEO killing season puts boards to test | afr.com




The greatest sorrow is that which man himself has brought unto himself....



It hurts to read this. But it ok I am used to reading about the absurdity of human behaviours -  Havel said it all ;-)

Center for Data Innovation: “Artificial intelligence (AI) has the potential to dramatically transform huge swathes of the economy and society for the better, and as the technology continues to make headlines many countries are developing plans to ensure they can take full advantage of these benefits.

Senior politicians rush to disclose free Foxtel subscriptions amid uncertainty



opencorporates blog: “It is thanks to the persistence, effort, and hard-work from open data advocates both inside and outside government that French companies are now available as open data. Since January, France’s Institut National de la Statistique et des Études Économiques (INSEE) has published the official “SIRENE” company registration database as open data, and after extensive analysis and processing, we added this data to OpenCorporates. This added over 10 million entities, making it our 120th jurisdiction, and the 2nd largest just behind the UK. The INSEE dataset is very extensive, covering companies, associations, sole traders / individuals & state bodies, plus all their trading branches, from both mainland France as well as its overseas departments, territories and dependencies (such as French Guiana, Martinique, Reunion, etc). Given that OpenCorporates is primarily a database of legal entities, we have currently excluded the local trading branches as they are out of scope (although we may revisit this in the future, though it would be on a global, not local level). This first cut also (temporarily) excludes the  overseas jurisidictions, while we do further investigations and internal mapping, but we will  be adding them in due course. The INSEE dataset from January initially covered just active  companies, so we supplemented it with just over 500,000 dissolved companies for 2012-2017 from the open data sourced from Infogreffe (the grouping of chambers of commerce, who actually perform the registration of companies). Combining two such datasets has the potential for causing significant data issues, and so we have performed extensive research on the two, and how they may be combined. Based on this research we found, among other things, that where a company is found in both datasets, the INSEE data tended to be more accurate with regards to the company status, and therefore it has taken precedence over the Infogreffe data for that company…”