Finding the truth is harder than ever — and online customer reviews are no exception. Take a look at the two reviews below. Do they sound like real commentary? Or something faked to sell more gear?
- “What can I say, it’s better than most and stand’s up to alot. I’ve got a bad back and I still use this frame, built right. Highly recommend,” reads one review of the ALPS OutdoorZ Pursuit backpack on Amazon.
- “I bought these for my grandson for Xmas. This is his favorite brand,” states another review of the Salomon Men’s X Ultra Pioneer Aero Hiking Shoes, also on Amazon.
If you’re feeling uncertain about these comments — whether because of the sentiments or misspelling — you’re not the only one. Both reviews are likely fake, according to AI software built to help consumers navigate the minefield of misinformation on e-commerce giants like Amazon, Walmart, and Best Buy. Called Fakespot, it tries to sort the questionable reviews from the reliable ones.
While the vast majority of consumers now look at online reviews before buying a product, research shows many of these reviews simply can’t be trusted. There’s lots of money to be made online, and it’s easy for companies to pay writers (sometimes using AI) to try to boost sales through virtual chicanery. While many sites try to foolproof their system against fraud, workarounds are still possible.
And humans haven’t excelled at spotting fake reviews, according to a 2022 study in the Journal of Retailing and Consumer Services. To stem the tide of lies, we’ll likely need “machines to fight machines,” the study’s authors wrote. That’s where AI-based fraud detectors like Fakespot step in to swiftly analyze thousands of reviews for us.
It remains to be seen if artificial intelligence can point consumers toward reliable products through data analysis. I spent 3 weeks testing out Fakespot to check online reviews of outdoor gear, and the service has much to offer consumers.
After all, if AI can’t solve this problem — it’s unlikely anything can.
Fake Reviews: The Scope of the Problem
Like so many things on the Internet, the scope of the problem is hard to comprehend.
Nine out of 10 online consumers peruse reviews before purchasing a product. At the same time, 67% of Americans believe that fake reviews are a growing problem, according to a survey from Uberall.
Working with The Transparency Company, Uberall analyzed 4 million reviews on Google, Facebook, Yelp, and Tripadvisor. Despite massive efforts from those companies to protect their sites from fraud, the Uberall study still found plenty of suspect reviews.
Of those four sites, Google had the most fake reviews overall (10.7%), followed by Yelp (7.1%), Tripadvisor (5.2%), and Facebook (4.9%).
“Unfortunately a lot of consumers take reviews at face value, and are being deceived,” said Curtis Boyd, founder and CEO of the Transparency Company. “Google removed 130 million fake reviews in 2019 and 2020 — it’s a multi-billion dollar problem.”
In October 2022, representatives from Yelp, Tripadvisor, Trustpilot, and Google, among other review sites, met for a one-day, closed-door conference in San Francisco, The New York Times reported. Their goal? To discuss how to prevent fake online reviews. (No word yet on what they decided.)
And a year earlier, in October 2021, the U.S. Federal Trade Commission announced it would begin cracking down on the practice. The federal agency notified more than 700 national advertisers that “misleading use of endorsements could lead to major financial penalties.”
Of course, enforcing those penalties means finding proof. With a new wave of PR companies aimed at supplying fake reviews while offering plausible deniability to their clients — holding companies accountable isn’t easy.
And while several tools now exist for businesses trying to curb the problem (ReviewTrackers and TruthEngine, for example), there’s little available for consumers. That’s where AI tools like Fakespot come in.
AI to the Rescue?
Sometimes, spotting fakery is easy for us humans — if you know what to look for.
Check out this review page for a pair of Bluetooth headphones from a company called PocBuds. The page is dominated by the same kinds of gushingly positive reviews, all long and — this is the kicker — all with several photos attached.
“This product is likely using promotional reviews,” Fakespot director Saoud Khalifah said in an interview. “These reviews are likely coming from a review farm or a Facebook group that offers these services to Amazon Sellers. A drill down into these reviews makes it clear they are paid for. They also all follow a similar review format with pictures. This seller most likely offered an Amazon gift card for reviews like this.”
That fraud is fairly obvious to the trained eye. (Fakespot, for example, gives the product and the seller F Ratings, the lowest possible.) But when both humans and robots can write believable commentary, catching every single fraudulent review might not be possible, even with AI.
Imagine a short review like: “Great shoes. Super comfy.” How could you know if that’s real? In short, by analyzing all the data around it. For Fakespot, that means using AI-driven tools to analyze all the reviews from the same user, along with product info, seller info, and other data.
For example, did a product get 100 five-star reviews in the same day? That’s suspicious. Did a positive review come from a user account that made 20 other glowing reviews the same day? Very fishy.
The AI tool is essentially conducting an investigation — just more quickly than a human ever could. According to its website, Fakespot makes money through paid ads, affiliate marketing, and offering paid detection services to private businesses.
For consumers, however, the platform remains a free, open-source method of checking the reliability of both the product’s seller and its reviews. Given that Mozilla just bought Fakespot a few weeks ago, it’s likely the AI platform is doing something right.
In the ongoing battle for a truthful Internet, AI can become a vital filter for misinformation, Khalifah said.
“We have trained our systems to catch fake reviews generated using different AI models and continue to train our systems to catch more advanced AI-generated fake reviews,” Khalifah said in an email. “User trust is our number one priority and it guides our model development.”
GearJunkie Tests Fakespot
After installing Fakespot’s free software, which works as a plugin for your web browser, I got to work.
I collected the AI’s analysis of over 200 camping and outdoor products on sites available through the software. In this case, that’s Amazon, Best Buy, and Walmart. (Fakespot can also analyze products and sellers on eBay, Sephora, and Shopify, with more sites likely on the way.)
I tried to analyze about the same number of products on each site (around 60-75) and skewed toward products with at least 50-100 reviews. (The more data, the more likely Fakespot can deliver something helpful.)
The software offers five ratings based on the reliability of the product’s reviews. In terms of ranking reviews, they break down like you’d expect from letter grades, as in at least 90% reliability for A Ratings, 80% for B ratings, 70% for C ratings, and so on.
For all 200+ tested products, nearly half (45%) received ratings of C or lower. That often means only 70% of their reviews could likely be trusted. In many cases, the percentage of reliable reviews falls below 40%.
When looking at the distribution of the five possible Fakespot ratings for all three sites, there’s a wide spread. While A ratings account for roughly a third of the tested products, the other four ratings each represent 13-20% of the rest.
However, those numbers change dramatically from site to site. Best Buy, with nearly 70% of its products receiving A ratings, heavily skews the overall picture. By comparison, Amazon products came out with just 22.5% A ratings, and Walmart with 22.2%.
Summary: The Bad News
Several products recommended here on GearJunkie received dismal test scores when put through the Fakespot review grinder. Many times, that comes from an analysis of the same Amazon product page linked from our own website.
For the Salomon Men’s X Ultra Pioneer Aero Hiking Shoes, Fakespot said just 39.4% of its 247 customer reviews were reliable. (Note that these percentages constantly change as new reviews are added, and Fakespot updates its analysis.)
Of the 1,600+ reviews for the GJ-recommended Moji Lantern from Black Diamond, just 32.5% were considered trustworthy by Fakespot.
For each analyzed product, Fakespot offers a summary page that compiles what it identifies as the most useful (and trustworthy) commentaries about the product, as well as the most unreliable.
It’s basically a vetted version of the summaries now common on product pages with hundreds or thousands of reviews.
On the Pros and Cons section of the analysis page, the AI tool noted one negative, but reliable, review for the Moji Lantern: “Save your money and don’t believe the hype. This is supposed to be rated for 100 lumens on high. Batteries are not included.”
Each page also comes with a section aimed at the Most Authentic and Least Authentic reviews. Often, the Least Authentic review sounds a little too glowing, as with the MPOWERD Luci String Lights on Amazon:
“I Love Biolite products (have the full camp stove kit & firepit too…LOVE!) and this is no exception. It is pricey & a bit complicated, but well made and fun. The battery lasts a long time on mine.”
What Can We Trust?
In case you haven’t noticed, we really like talking about outdoor stuff here at GearJunkie.
It’s part of our job to sort through the seemingly endless options of sleeping bags, tents, and hunting knives, and then tell you what we like and where you can buy it. That seems like a simple enough proposition, but the World Wide Web has a way of complicating things.
We can recommend products based on our personal experience. But we can’t vet the sellers or a thousand customer reviews for a single lantern on one website.
It’s enough to make us yearn for the days before e-commerce, when learning about gear meant talking to someone in person at your local gear store, or debating the merits of camping stoves with friends equally obsessed with the minutiae of weight, durability, and usefulness.
But, as we know too well — there’s always another angle. The online economy has made more options available to more people, and now includes the opinions of many marginalized communities long excluded from the conversation. AI-powered tools like Fakespot might help us make sense of all the noise, while still leveraging the immense power of the Internet in a way that benefits both weekend warriors and full-time athletes.
However, the AI arms race has just begun, and no one knows how it will end. Even Fakespot’s director, Saoud Khalifah, acknowledged that without regulation of the technology, AI could power the next tsunami of misinformation.
“There needs to be more regulation around generative-AI because it is possible for bad actors to flood the internet with AI-generated misinformation that will be hard to distinguish from human-generated content,” Khalifah said. “This will have profound and impactful effects for the world that goes beyond the concerns of fake reviews and eCommerce fraud.”
