South Koreans are worried about the surveillance state. At a recent conference that was supposed to be about protests — such as pro-democracy demonstrations and the #MeToo movement — journalists invariably asked me one question: how do we maintain democracy in an age when governments and companies are collecting ever more data on everything we do?
They’re right to be concerned. If people don’t recognise the danger, they might eventually be unable to protest at all.
Thanks to their proximity to China, South Koreans are witnessing up close that country’s efforts to deploy big data technology as a tool of social control. Omnipresent surveillance cameras, trained in facial recognition, record people’s whereabouts. The ubiquitous social credit score monitors how people behave on the train, whether they pay their bills on time and whether they participate in undesirable organisations.
This is a real threat. The potential accuracy and reach of government surveillance will only increase as the data piles up and the technology improves — particularly in places such as China, where there are few if any privacy protections. Worse, in many cases, it doesn’t even have to work as advertised. The mere knowledge that they are being watched will often be enough to keep people in line. And if mistakes are made — if, for example, the system misidentifies criminals or wrongly dings someone’s social score — repressive governments won’t necessarily be held to account.
China’s heavy-handed approach is spreading. Venezuela is reportedly using Chinese technology to create its own social credit score. The Saudi government has been monitoring Twitter to identify and punish political dissidents — an effort in which the consulting company McKinsey may have (perhaps inadvertently) played a role.
The next cold war
Governments will wage the next cold war with technology, in large part against their own citizens, but the nature and ideology of that conflict are still uncertain. So far, China’s version appears to be in the lead, in part because the world’s great superpower, the US, still hasn’t figured out what its strategy will be. Discussions about the use of big data and artificial intelligence in the US and China tend to frame it as a technology race, rather than a clash of values. From that perspective, privacy protection is merely a pesky constraint to the innovation that will bring about the brave new world.
Trying to compete with China in developing social control technology would be a terrible mistake. Europe’s General Data Protection Regulation explicitly disavows such an approach, choosing instead to defend human dignity and freedom. In the US, though, people are too distracted by the political circus to take on a creeping surveillance state, starting with big tech firms such as Facebook, which have demonstrated their incapacity to promote democratic values over profit.
We need to do better, and soon. — Written by Cathy O’Neil, (c) 2018 Bloomberg LP