Amazon And Google File ‘Scary Smart Patents’ To Monitor Your Activities

  • Amazon and Google file patents that outline how audio and visual signals could be utilized to analyze users’ interest and behavior. 
  • They want to collect users’ data via smart household device(s), even when it’s not triggered by its ‘wakeword’. 

The giant tech companies are focusing on collecting as much users’ data as possible. They are literally trying to watch your every move. A couple of years back, Amazon and Google filed patent applications that allow them to monitor and record your activity through household devices (Amazon Echo, Google Home, etc.).

Currently, the voice assistant devices hear and process audio only after being triggered by certain phrases like ‘Hey Alexa’ or ‘OK Google’. But this might soon change.

These patents outline a wide range of possibilities for how smart household devices could be used to spy on users. They indicate that the always-on Amazon Echo and Google Home devices could know everything from user’s interest and mood to their medical conditions. Companies can use this data for financial gain by targeting ads and product recommendations.


Amazon has filed a patent titled “keyword determinations from conversational data” that can generate a personality profile of the user, while Google’s patent named “privacy-aware personalized content for the smart home” outlines how audio and visual signals could be utilized to analyze users’ mood.

Source: The New York Times

What’s Included In Patents?

1. Extracting behaviors and sentiments from ambient speech, even when the device is not triggered by its ‘wakeword’. The ‘voice sniffer algorithm’ can listen to human voice (including phone conversations), for keywords like “love”, “hate”, “interest”, and process the statements to target advertising.

Obtained from Amazon Patent 

2. Identify speakers in a conversation and generate interest profiles of each one. These profiles could be used to understand the behavior of individual members in the family (to target ads on individual level), and create acoustic voice profiles for voice-triggered devices.

Reference: Patent US20160260135A1 | Patent US20170323645A1

3. Recommend products and services observed by smart devices like security cameras. The algorithms can extract data from video feeds, for example, title of the book kept on the table, the face of celebrity on your shirt, and the presence of a football or guitar in the room. These data will help companies determine users’ interest and preferred activities.

Obtained from Facebook Patent 

4. The companies can share data with third parties, who can further make inferences based on users’ data. This includes your cooking, showering, entertainment and sleeping schedules.

5. Infer child mischief using movement sensors and audio. According to the patent, devices can monitor family members’ activity on individual level and report back to an entitled policy holder.

6. Insert advertisement into the responses provided by smart household devices, based on gathered data.

Google said that they don’t use raw sounds to extrapolate medical conditions, moods or demographic data. All their devices and services (Google Home and Google Assistant) are developed with customer privacy in mind.

Amazon said it takes users’ privacy seriously and they are looking for techniques that explore the full potential of new technology.

Read: Google Develops Voice AI That Is Indistinguishable From Humans | Tacotron 2

Moreover, companies file thousands of patents every year, but not all of them are granted and implemented into final products. However, this kind of technology definitely raises concerns about privacy intrusion on a whole different level.

Written by
Varun Kumar

I am a professional technology and business research analyst with more than a decade of experience in the field. My main areas of expertise include software technologies, business strategies, competitive analysis, and staying up-to-date with market trends.

I hold a Master's degree in computer science from GGSIPU University. If you'd like to learn more about my latest projects and insights, please don't hesitate to reach out to me via email at [email protected].

View all articles
Leave a reply