You swipe your credit card at a shop, sit down at a restaurant after checking Foursquare and browse Facebook while waiting for your friend to arrive. En route to your car afterwards, you reply to an e-mail and then turn on your GPS to find the best route home.

At each step in this seemingly inconsequential chain of events, you — like millions of other people — are leaving a digital footprint, which, when analysed with the data of the faceless millions, shows clear patterns and daily routines.

What do you spend your money on? What economic bracket do you fit into? Are you one of hundreds of people taking the same route every day and should there be more bus stops on that road to encourage you to take public transport instead of your car? This is what your digital footprint tells anyone who looks, and this is what companies and governments want to know.

“Data is the new oil,” says IBM South Africa’s university relations manager Sean McLean. “Just like crude, it is very valuable. But if it hasn’t been refined, it cannot be used.”

This phenomenon is called big data — the explosion of data globally and the desire to analyse it — and with it come the challenges of how to analyse the huge quantities of data that global citizens are producing, as well as how to protect the privacy and rights of individuals as companies start mining their data to offer new and better services.

Though Europe is still in the process of drafting its private data legislation — a final draft is expected to be before the European parliament next year — South Africa is ahead of the curve. This week, parliament gave the Protection of Personal Information Bill the green light, and it is set to be signed into law.

Section 14 of the constitution provides for a right to privacy, and while many people see in the bill a way to finally be rid of irritating marketers and cold-callers, it has important implications for big data in the country.

First, your personal information — which ranges from your biometric DNA data to your sexual preference — cannot be collected without your knowledge. Personal information “must be collected directly from the data subject” and steps “must be taken … to ensure that the data subject is aware of the purpose of the collection of the information”. Also, it has to be destroyed once the responsible party is no longer authorised to retain it.

This does not relate to information that has been “de-identified”, which means that it is no longer possible to identify the individual. However, the ability to “de-identify” data is questionable.

Gert-Jan van Rooyen, a director at Stellenbosch University’s MIH electronic media laboratory, points to a high-profile court case against Netflix, a movie streaming site in the US.

In 2009, a woman sued Netflix for “outing” her after the company de-identified users’ data and challenged the public to create a better search engine for its site, based on users’ movie selections.

“Security researchers showed that it was possible to match up the ‘anonymous’ movie rating dataset with users’ reviews on another public resource… This revealed hidden information, such as users’ sexual orientation,” Van Rooyen says. “Not exactly what they had in mind when they made the dataset public with all good intentions.”

But on the other hand, big data offers huge opportunities both as a growing sector and to provide better services, especially in Africa.

IBM’s McLean says: “Governments across the continent want to become more e-enabled, capturing citizen information, and cross-border information about individuals.”

Big data has major applications for the continent, he says, although he would not say exactly what applications IBM is exploring.

“The applications for the African continent are about mobile and how to take advantage of that data. You can’t ignore communicable diseases [and how data around numbers and available treatment would help service delivery], plant genomics, drought resistance, weather patterns,” he says.

In its privacy and big data white paper published last month, the international nonprofit organisation Information Systems Audit Control Association writes: “Across all industries, including banking, government, healthcare, media, energy and education, the exponential automation of business processes is widening the landscape of data consumption and analytics.”

But throughout the report, while the organisation extols the myriad benefits of big data, or data mining, it reiterates: “Without proper governance, the same data that can be used to create value can be used to create intrusive and damaging outcomes and destructive decision making.” — (c) 2013 Mail & Guardian