Multiverse 2.357

Google filed a patent application for “Smart-home automation system that suggests or automatically implements selected household policies based on sensed observations”. This is a futuristic look at automated systems that monitor “temperature, humidity, lighting, water, power usage, sound signals, ultrasound signals, radio-frequency, other electromagnetic signals or fields, GPS, proximity, motion, light signals, fire, smoke, other gas, etc.”

Google’s smart home vision includes modules to allow setting household policies that are marketed as controls aimed at empowering parents to limit children’s activities; e.g. no television before doing homework. This is accomplished via a household manager module, that infers everything from occupants’ activities to their emotional states.

These monitoring systems would recognize certain activity signal patterns – including in the bathroom and bedroom, kitchen and entertainment room and some data from the outside of the house. According to the Consumer Watchdog Organization, these data can and probably will be used to infer users mood and sentiment from ambient speech and casual behaviors.

It can also be used for creepy product ad placement and suggestions based on behavior – recommending bedtime story apps a short while before the bedroom lights will probably be turned off in a child’s room. This home knows when you are likely to come home early, and when you tend to enjoy late night snacking.

Technology companies like Google and Amazon are eager to provide products and services powered by artificial intelligence to make your home aware and responsive. There seem to be important shortcomings in what has been shown to date in terms of privacy awareness from these companies, and the trend is worrisome because their business models have goals that are in conflict with our privacy concerns.

Digital privacy concerns surround devices like smart speakers, that listen to and understand human voice commands – and quite possibly record and store the data. Of course these devices that include microphones and cameras can be subject to warrants demanding the company turn over that data. This data is uploaded, analyzed, and stored; and can then be used, shared, lost or stolen.

This uploading of personal data is inevitable because it enables new services. For example, when the information is uploaded to the cloud based AI service, it can be combined with other data points to provide useful feedback like, “you forgot to lock the front door”, “you left the lights on” and so on.

These companies want to know every time a light is turned on and off, and every time the channel is changed on your TV. Google’s smart home patents show audio and video monitoring systems that recognize certain activity signal patterns – including bathroom and bedroom, kitchen and entertainment room.

These ever more intrusive systems can be downright creepy, especially if the company involved is not disclosing enough information about what dat is capture, how it is processed and what happens to it afterward. Google’s Nest Secure product secretly had an undocumented microphone built into it, only revealed much later, that ultimately enabled it to be used as a personal assistant.

According to a report from The Intercept the Ring team is accused of having some questionable usage practices with video data collected by it’s Ring devices. This report claims that employees in Ukraine and the US had unfettered access to video footage from all devices in all locations. Apparently this video treasure trove is searchable by user, and if a user email address is known live video feeds can be monitored. Employees annotate people and objects in some videos in order to train AI systems, according to this article.

These claims are disputed by company management, but it seems prudent to assume these things will happen. Even if access were carefully restricted accidents can happen and data may be exposed or hacks may occur. This affects not only household members and visitors to those households, but neighbors and those passing by.

This raises concerns about integration with other facial imagery collection systems as well, like police body cameras or commercial monitoring systems. The large scale systems can all be combined to paint a picture of where citizens are and what they are doing, in the same waythat Google maps knows about busy routes – they track lots of devices. This threatens a shared feeling among many that we have a right not be constantly surveilled.

Digital privacy concerns around Google and Amazon technology plans focus on sensors including cameras and microphones that monitor activity in the home. These systems that listen to and understand human voice commands quite possibly record and store data, for the purpose of maintaining a more detailed user profile in order to present advertisements more likely to be clicked on.

It is not clear what the effect on society will be when many children grow up in a surveillance household, where they can hide nothing form the all- sensing sensors and monitors. It is also unclear that families will be capable of protecting their privacy when the operators of these systems are incentivized to aggregate and share this detailed personal information.

One harbinger of this surveillance home future we face is the Roomba self operated vacuum, made by iRobot. The company’s latest models will store and share floor plan data over Wi-fi, but only when users grant permission. Still, this gives privacy advocates pause, noting that historically companies have frequently failed to secure sensitive user data.

Google and Amazon both have the same motivations for gathering as much activity data as possible and assigning them to user profiles. Knowing more about what happens in the home allows them to better target advertisements to users on their phones, televisions, computers and other devices. It also allows them to have an additional platform on which to show those targeted ads – your home.

If you recall, we posted last time about security implications of smart homes, which led us to talk about privacy concerns in this post. Stay tuned if you’re interested in this topic, because there is much more to say about it. After all, smart building are becoming commonplace, and let’s face it – smart homes do not exist in isolation, they will tend to get connected.