So how should we think about these apps? When does technology step over that line from being merely useful to becoming insidiously stereotype-enforcing?
Anyone can investigate a neighborhood by looking up local crime rates, median income, and demographics online – not to mention the information gleaned from word-of-mouth reports. To perform such research and then make a decision about traveling to a particular area involves critical thinking, which is hardly objectionable. The ethical problem occurs when your mobile device takes over that thinking for you.
Microsoft’s Pedestrian Route Production technology, patented in January 2012 – and immediately dubbed "the avoid-ghetto app" by many in the media -- was designed to one day let Windows Phone users filter walking routes according to "weather information, crime statistics, [and] demographic information." According to the language of the patent, such filtering is useful because "if it is relatively cold outside, then a pedestrian is far more likely to feel an impact then [sic] if a vehicle equipped with a heating system protected her. Moreover, it can be more dangerous for a pedestrian to enter an unsafe neighborhood then [sic] a person in a vehicle since a pedestrian is more exposed and it is more difficult for her to leave an unsafe neighborhood quickly." It makes sense to keep safety in mind while navigating an unfamiliar area on foot, but letting a computer algorithm divert you from a particular neighborhood on account of statistics is problematic.
There’s another feature mentioned in the Microsoft patent that deserves scrutiny: the ability to sell route directions. Corporations could pay to have the app send users through routes with carefully plotted advertising campaigns. If your GPS system directed you to turn down one street rather than a parallel one just so you’d encounter a specific poster, would it do so with your consent?
Jim Thatcher, a geographer at Clark University, says our increasing reliance on mobile spatial technologies opens the door for something he calls "teleological redlining," in which applications "make it very easy to malign certain areas" and can even "obliterate" the possibility of our encountering certain people, places, and events. What’s more, these applications are generally presented to us as "neutral." We often forget to consider the motivations and biases at work behind the scenes.
Our experiences have always been mediated by technology, Thatcher says, but these days that technology is increasingly opaque to its users – that is, few of us actually understand the mechanisms behind it. How many people know exactly how an email gets from one inbox to another? In contrast, the way the U.S. Postal Service transports letters is clear to us. Understanding the mechanisms is a key part of understanding the motivations driving these systems.