Amazon AI Shopping Feature Generates Bizarre and Insulting User Profiles

Amazon's new AI shopping feature 'About You' went viral as users discovered bizarre and insulting personal profiles generated by the algorithm, raising privacy concerns worldwide.

Source:Ynet
Amazon AI Shopping Feature Generates Bizarre and Insulting User Profiles
Illustration WireIsrael: AI Art Generated

Amazon recently attempted to gently nudge its customers into purchasing slightly more items. Instead, the retail giant found itself explaining to Americans why it believes they lack friends, possess unusually thin husbands, or require anatomical improvements to their rear ends. A new feature launched by the company, designed to provide personalized shopping recommendations through artificial intelligence, recently turned into a viral joke on social media after users discovered the surprising and sometimes insulting conclusions the company had drawn behind the scenes. It turns out that the algorithm recommending vacuum cleaners and grooming products has also developed some very strong opinions about the people buying them.

The social media storm began when American customer Megan Montgomery accidentally stumbled upon a page titled "About You" within her Amazon account. Alongside reasonably logical details—such as the fact that she likely owns a Shark vacuum cleaner and was planning a two-year-old birthday party themed around the children's television series Bluey—appeared a remarkably blunt statement: "Has a flat butt." Montgomery posted a screenshot to social media, noting that she was left completely speechless. "I mean, they aren't wrong, but did you really have to expose me like that?" she added, suggesting the insulting conclusion may have stemmed from purchasing leggings designed to accentuate her curves. Her post accumulated over 1.5 million views within hours, sparking an unofficial competition among users trying to uncover who won the most embarrassing personality analysis.

The results did not disappoint. One user discovered that Amazon had determined her husband has skinny legs. Another user was told she "curses when she is scared," while a third found the company believed she "does not trust people easily." Additional profiles featured conclusions like "has no friends,cannot keep real plants alive," and "treats her cats as if they were her biological children." In the latter case, Amazon even listed the cats first, followed by the maternal attitude toward them, and only finally mentioned that the customer has a husband—making it abundantly clear who tops the family priority list when opening their wallet.

Another user discovered Amazon believes he reads books on his mobile phone while in the shower. "I mean, yes, but what the hell?" he responded. Another user encountered the claim that he is "too short to reach the bottom of a 40-inch-deep bin," which is slightly more than a meter. Someone else found the company believes she has a "bad father," leaving her bursting with laughter and wondering how the algorithm figured that out. Meanwhile, another user was surprised to find Amazon determined he has a baby. "That is definitely news to me," he wrote.

The Launch of 'About You' and the Data Behind It

So what exactly happened here? Last May, Amazon launched "About You," a new feature designed to let customers view and edit the information the company uses to tailor shopping recommendations. In the United States, where Amazon has become an integral part of millions of households' daily routines, order history can reveal quite a bit about an account holder: what they eat, what clothes they wear, what medications and grooming products they purchase, how they clean their homes, and what family events they are planning.

The system analyzes previous purchases, site searches, items saved in lists, customer-written reviews, and conversations with Alexa for Shopping, Amazon's shopping assistant. The data is categorized into hobbies, product preferences, brands, home, and family. The business objective is clear: the more the company knows about a customer, the better it can suggest products they are likely to buy, potentially even before they realize they want them themselves.

Amazon presents far more innocent examples than those flooding social media. For instance, someone who regularly buys plant-based milk might receive recommendations for other dairy-free products, and someone talking to Alexa about their cat's favorite food will receive suggestions for matching treats. However, searching for a parenting book does not necessarily indicate parenthood, looking up an acne treatment product does not mean the buyer suffers from breakouts, and purchasing shaping leggings is certainly not an invitation for anatomical commentary on one's posture.

Privacy Concerns and Managing Your Profile

Nevertheless, it is important to note that not all the bizarre diagnoses circulating online were necessarily generated by artificial intelligence. Users can manually add details to their profile, making it impossible to verify whether every single screenshot reflects an independent system conclusion.

Beyond the jokes, the phenomenon raises significant questions about privacy. Amazon has been collecting data on customers' shopping habits for years, but presenting it as a list of personal conclusions illustrates just how many details can be inferred from everyday, simple actions. One customer shared that she was horrified to discover the system retained details she had previously written in product reviews, turning what she thought were recommendations for other buyers into a personal profile built about her.

Amazon explains that users can correct erroneous conclusions, add details, or remove information they do not want used for personalization. However, the feature cannot be entirely disabled, and removing a conclusion from the profile does not necessarily erase the underlying raw data. The company declined to comment on several of the specific reactions directed at it following the viral posts.

For those wishing to join the fun and discover what Amazon thinks of them—even if order history in Israel is far more modest than that of the average American customer—the process is relatively straightforward. In the Amazon Shopping app, navigate to the "Me" tab (the person icon at the bottom of the screen), tap your name, and select "About You." In some app versions, this can be reached via "Account," then "Shopping Preferences," and finally "About You."

On a desktop browser, hover over "Account & Lists" in the upper right corner, click your name or "Who's Shopping?", select the profile, and enter "About You." An alternative route is "Account," followed by "Your Shopping Preferences," and at the bottom of the page, "Manage Your Information." There, you can review conclusions, edit them, or remove them individually.

Users can also ask Alexa for Shopping in English: "What do you know about me?" Just be prepared for the possibility that the answer might turn out to be a bit more personal than intended.

Related News