But these tools are worrisome, too. Few people posting photos on Instagram are aware that they may be revealing their mental health status to anyone with the right computational power.
Computational inference can also be a tool of social control. The Chinese government, having gathered biometric data on its citizens, is trying to use big data and artificial intelligence to single out “threats” to Communist rule, including the country’s Uighurs, a mostly Muslim ethnic group.
[As technology advances, will it continue to blur the lines between public and private? Sign up for Charlie Warzel’s limited-run newsletter to explore what’s at stake and what you can do about it.]
Such tools are already being marketed for use in hiring employees, for detecting shoppers’ moods and predicting criminal behavior. Unless they are properly regulated, in the near future we could be hired, fired, granted or denied insurance, accepted to or rejected from college, rented housing and extended or denied credit based on facts that are inferred about us.
This is worrisome enough when it involves correct inferences. But because computational inference is a statistical technique, it also often gets things wrong — and it is hard, and perhaps impossible, to pinpoint the source of the error, for these algorithms offer little to no insights into how they operate. What happens when someone is denied a job on the basis of an inference that we aren’t even sure is correct?
Another troubling example of inference involves your phone number. It is increasingly an identifier that works like a Social Security number — it is unique to you. Even if you have stayed off Facebook and other social media, your phone number is almost certainly in many other people’s contact lists on their phones. If they use Facebook (or Instagram or WhatsApp), they have been prompted to upload their contacts to help find their “friends,” which many people do.
Once your number surfaces in a few uploads, Facebook can place you in a social network, which helps it infer things about you since we tend to resemble the people in our social set. (Facebook even keeps “shadow” profiles of nonusers and deploys “tracking pixels” situated all over the web — not just on Facebook — that transmit information about your behavior to the company.)