How NOT to build a reputation system ...

China's Government Shows the Way

"How good a citizen are you? China hopes to answer that question for every one of its citizens with a numerical rating system based on their financial standing, criminal record and social media behavior. A new translation of the government’s plans for a so-called social credit system sheds light on how China aims to utilize 'Big Data' to hold all citizens accountable for financial decisions as well as moral choices."

All Good Citizens...? 
That's the news from the International Business Times. And it's appalling. What's the core motivation they're addressing? Well, frankly, "good citizenship". That objective, in and of itself, isn't so horrible. We live as social beings.

We have civilization's structures, laws, and institutions to help create order out of what might otherwise be chaos. Encouraging people to bring their best selves to their daily engagements with the world is a noble goal. It is, in fact, one of the goals we hold here at The World Table.

But China's effort to create a social credit system is fundamentally flawed, and if successfully implemented, is terrible news for the Chinese people.


First, it's centralized. That means all the data is being collected and collated by a central authority. No surprise for the government of China, of course. But it's bad news because all that data shouldn't be in one place, under one authority. The World Table is explicitly choosing to use a distributed public ledger to make sure reputation data is NOT in one place.

Second, it's coercive. The article doesn't say so, but it's very easy for the Government to begin to use its authority to unilaterally declare that if you're not participating -- and worse, not participating with a certain moral rectitude as measured by your score -- you won't get access to certain privileges, benefits, or services. It's one thing for the market to create opportunities to participate, with rewards accruing to those who do. It's quite another for the government to compel participation in something as abstract as "social credit" while at the same time being the sole arbiter of "moral choices". The World Table operates on the premise that values (virtue sets) and reputation algorithms should be part of an open system (Open Reputation) that the community can examine, engage, extend, and critique. We all have useful perspectives to share on what elements make up a good reputation.

Third, it's Big Brother. Big Data serving a morally and socially oppressive regime is a modern 'panopticon' where the citizens are the prisoners and the government is the jailer.

If you want to do it right, every actor in the system must be subject to open scrutiny. Sunshine is the best disinfectant, but only if the jailers are being watched, too. The power relationship must be balanced or it's a recipe for enslavement by social manipulation.

This Chinese government program is as creepy as it gets, and serves as an excellent foil for the work we do at The World Table. We must be vigilant that our actions match the intentions of our words, and that we create a system that helps, not hurts. We hope you will join us in our endeavor to do so, and that you'll keep us honest when we err.



© 2014 The World Table | Legacy Forum | Terms of Service | Privacy Policy