If this sounds invasive, it’s important to recognize that this is already happening, just online. Google and Facebook both record and analyze user behavior, use it to sort people into categories, and then target them with ads and other content. Facebook likely knows your race and religion, while Google uses your emails and search history to sort you into ad-ready brackets. Netflix infers all types of data on users based on what they watch, then serves back hyper-specific movie and TV categories. This patent simply expands the areas in which your behavior is already mined and recorded from your phone and laptop to your bedroom.
And your children’s bedrooms. The second patent proposes a smart-home system that would help run the household, using sensors and cameras to restrict kids’ behavior. Parents could program a device to note if it overhears “foul language” from children, scan internet usage for mature or objectionable content, or use “occupancy sensors” to determine if certain areas of the house are accessed while they’re gone— for example, the liquor cabinet. The system could be set to “change a smart lighting system color to red and flash the lights” as a warning to children or even power off lights and devices if they’re grounded.
Read: Alexa, should we trust you?
While people can set goals for their children or themselves, these policies could also be “based upon certain inputs from remote vendors/facilitators/regulators/etc.,” according to the patent. That opens the door for companies to offer rewards for behaviors in the home. A household may set the internal goal of “Spend less time on electronic devices,” or “Use 5 percent less energy each month for the next three months.” Google devices could then connect to anything “smart” in the home and send you, and potentially a vendor or third party, updates on usage and screen time.
Just this month, the insurance company United Healthcare began partnering with employers to offer free Apple Watches to those who hit certain fitness goals. Insurers might also offer benefits to residents whose homes prove their fitness or brand loyalty—and punish those who don’t. Health insurers could use data from the kitchen as a proxy for eating habits, and adjust their rates accordingly. Landlords could use occupancy sensors to see who comes and goes, or watch for photo evidence of pets. Life-insurance companies could penalize smokers caught on camera. Online and in person, consumers are often asked to weigh privacy against convenience and personalization: A kickback on utilities or insurance payments may thumb the scales in Google’s favor.
For reward systems created by either users or companies to be possible, the devices would have to know what you’re doing at all times. The language of these patents makes it clear that Google is acutely aware of the powers of inference it has already, even without cameras, by augmenting speakers to recognize the noises you make as you move around the house. The auditory inferences are startling: Google’s smart-home system can infer “if a household member is working” from “an audio signature of keyboard clicking, a desk chair moving, and/or papers shuffling.” Google can make inferences on your mood based on whether it hears raised voices or crying, on when you’re in the kitchen based on the sound of the fridge door opening, on your dental hygiene based on “the sounds and/or images of teeth brushing.”
Of course, patents aren’t products, but they do represent an important shift. For a long time, the foundational metaphor of surveillance studies has been the panopticon—unending, inescapable, unwanted surveillance. Now these patents seems to hint that the age of hyper-personalization will make people willing, enthusiastic participants in the panopticon, both as subjects and as architects.