Siri Shows Increasing Promise, But Accuracy Falls Short of Google Text Searches

In what should not be a surprise given Siri's beta status and its use of new technology for voice interaction with mobile devices, a study from Piper Jaffray analyst Gene Munster finds that Siri's abilities are still eclipsed by the traditional text-entry Google searches that have been in place for years.

The study included presenting Siri with 800 queries in each of two environments, a quiet room and a busy urban street, and then assessing Siri's ability to correctly interpret the queries and provide correct responses. Specifically addressing the outdoor testing, Munster notes:

While Google comprehends 100% of searches, Siri was only able to comprehend 83% of the prompted questions on a busy Minneapolis street. In terms of answer accuracy, Siri correctly answered 62% of queries on the street compared to Google's estimated 86% answer accuracy. Based on these results, we give Google an A+ for comprehension and a B+ for accuracy while we give Siri a B for comprehension and a D for accuracy.

But while Siri has some room for improvement, Munster acknowledges that the technology is only in its infancy and he remains "optimistic about its future" with an eye toward iOS 6 as bringing substantial improvements.

siri examples
Piper Jaffray's testing was conducted using the iPhone's built-in microphone in the quiet setting and Skullcandy microphone headphones for the outdoor setting, with Siri's ability to correctly comprehend queries falling from 89% to 83% when moving outdoors to the noisy urban setting. On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.

Piper Jaffray also analyzed the sources for Siri's results, finding that Google provided 60% of the answers, with Yelp and other sources filling in the remainder.

Of queries excluding commands (i.e. call Jay, send text) Google would provide 60% of the answers, Yelp 20%, WolframAlpha 14%, Yahoo 4%, and Wikipedia 2%. Breaking down Siri's reliance further, Google provides 100% of navigation results, 61% of information results, 48% of commerce results and 42% of local results. Among other result aggregators, Yelp provided the most local results (51%) and commerce results (51%), while WolframAlpha provided 34% of information results.

Munster estimates that Google's share of Siri results will drop to 48% with the launch of iOS 6 as navigation, sports, and movie queries shift over to Apple and its new partners.

Popular Stories

2024 iPhone Boxes Feature

Apple Changes Trade-In Values for iPhones, iPads, Macs, and More

Thursday January 16, 2025 6:45 am PST by
Apple today adjusted estimated trade-in values for select iPhone, iPad, Mac, and Apple Watch models in the U.S., according to its website. Some values increased, while others decreased. The changes were not too significant, with most values rising or dropping by $5 to $50. We have outlined some examples below: Device New Value Old Value iPhone 15 Pro Max Up to $630 U ...
Generic iOS 19 Feature Mock Light

iOS 19 Leak Reveals All-New Design

Friday January 17, 2025 2:42 pm PST by
iOS 19 is still around six months away from being announced, but a new leak has allegedly revealed a completely redesigned Camera app. Based on footage it obtained, YouTube channel Front Page Tech shared a video showing what the new Camera app will apparently look like, with the key change being translucent menus for camera controls. Overall, the design of these menus looks similar to...
Generic iOS 18

Everything New in iOS 18.3 Beta 3

Thursday January 16, 2025 12:39 pm PST by
Apple provided the third beta of iOS 18.3 to developers today, and while the betas have so far been light on new features, the third beta makes some major changes to Notification Summaries and also tweaks a few other features. Notification Summary Changes Apple made multiple changes to Notification Summaries in response to complaints about inaccurate summaries of news headlines. For...
iPhone 17 Slim Feature Single Camera 1 Redux

'iPhone 17 Air' Launching Later This Year With These 10 New Features

Wednesday January 15, 2025 7:16 am PST by
While the so-called "iPhone 17 Air" is not expected to launch until September, there are already plenty of rumors about the "ultra-thin" device. Overall, the "iPhone 17 Air" is shaping up to be a mixed bag. Due to its thinness, the device is expected to have some limited specifications compared to the iPhone 17 Pro models, including only a single rear camera, only a single speaker, no SIM...
2024 App Store Awards

Apple Explains Why It Removed TikTok From the App Store in the U.S.

Sunday January 19, 2025 6:58 am PST by
Apple on late Saturday removed TikTok from the App Store in the U.S., and it has now explained why it was required to take this action. Last year, the U.S. passed a law that required Chinese company ByteDance to divest its ownership of TikTok due to potential national security risks, or else the platform would be banned. That law went into effect today, and companies like Apple and Google...
iPhone 17 Pro Dual Tone Horizontal Single Feature

iPhone 17 Rumored to Feature Major Thermal Design Upgrade

Friday January 17, 2025 4:33 am PST by
The iPhone 17 lineup will feature a vapor chamber heatsink to improve thermal performance, according to a new report. The news comes from Chinese tech news site MyDrivers, which claims that the entire iPhone 17 lineup, consisting of the iPhone 17, iPhone 17 Air, iPhone 17 Pro, and iPhone 17 Pro Max, will adopt the improved thermal heat spreader. Vapor chamber technology is already used...
iPad Pro vs iPhone 17 Air Feature

Here's How Thin the iPhone 17 Air Might Be

Friday January 17, 2025 3:38 pm PST by
For the last several months, we've been hearing rumors about a redesigned version of the iPhone 17 that Apple might call the iPhone 17 "Air," or something along those lines. It's going to replace the iPhone 17 Plus as Apple's fourth iPhone option, and it will be offered alongside the iPhone 17, iPhone 17 Pro, and iPhone 17 Pro Max. We know the iPhone 17 Air is going to be super slim, but...
HomePod mini and Apple TV

Apple Expected to Launch 20+ Products This Year: Here's the Full List

Friday January 17, 2025 5:30 am PST by
2025 promises to be quite a big year for Apple, with the company rumored to be planning more than 20 product announcements this year. Apple's rumored smart home hub will be its second all-new product to launch in as many years, following the Apple Vision Pro headset last year. And of course, we will get several new iPhone and Apple Watch models, like every year. Beyond that, Apple could...

Top Rated Comments

HarryKeogh Avatar
164 months ago
I'm using Siri to post this message and, as you can see, it's 100% a cure writ and pancakes urethra.
Score: 115 Votes (Like | Disagree)
basesloaded190 Avatar
164 months ago
Siri is not on the same level as Google voice. Just saying.

What Google showed yesterday with offline support was very impressive, I agree.
Score: 18 Votes (Like | Disagree)
Ciclismo Avatar
164 months ago
I'm using Siri to post this message and, as you can see, it's 100% a cure writ and pancakes urethra.

Nominated for "Post of the Month" award.
Score: 15 Votes (Like | Disagree)
Bezetos Avatar
164 months ago

Google is just a dumb engine with no possible applications other than being a directory of the internet. They have never developed beyond coder style searches for specifics and their engineer backgrounds suggest they never will.
I think you've been living under a rock if you seriously believe in this...
Score: 13 Votes (Like | Disagree)
rockland Avatar
164 months ago
Siri is substandard

To me, there's no two ways about it. Uncharacteristically, Apple releases a "beta" product, repeatedly invoking that fact as if it were an excuse.

Which would be fine, if they charged a "beta-worthy" price for iPhone 4S.

Sorry, but it's been well over half a year, and Siri STILL underperforms. Badly.

In my experience and in all kinds of situations and environments, Siri works less than 50% of the time. More like a third of the time. I've pretty much given up on it.

We all love Apple, but to me it's hard not to state the obvious. Siri is (so far) a flop, in terms of living up to its profusely advertised promise. It is not even close to the usual high standards of Apple/SJ. And if he were alive now, I think he would be blowing a gasket about it.
Score: 13 Votes (Like | Disagree)
SleeplessChaos Avatar
164 months ago
Fine, if you want to embarrass yourself...

On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.

So this the point. Will you get information quicker using Siri or googling? So you've completely missed the point here.

It's not comparing two things at once. It is comparing how accurately you can get information using both tools. That's just one thing.

That would be biased. Secondly, read again: Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results. So if you ask Siri "how tall is the empire state building", you'll get a precise answer from Siri (hopefuly) and find that information on the first page of Google results (actually, if you google that you'll now get an answer too). So this is what they're comparing. So again, you're completely missing the point.

And again, just go and read the article.

----------


Another person who missed the whole point of this research...

I'm wondering if we should check your reading comprehension guys...
I don't think he is missing the point of the experiment, I think that he, like me, thinks the point is stupid.

Think about it for more than a millisecond and you'll realize that it's comparing apples and oranges. Google, like every other text based search engine on the web, is simply matching text keywords with a large indexed database built by years of crawling through the web. The search is basically brute force and simple.

Siri is receiving a digital waveform from the microphone, eliminating frequencies from that waveform that are not commonly found in the human voice, translating those remaining frequencies into readable text via a remote server and then interpreting that into a command which it then uses to either search the web using what it assumes to be the correct web search service or perform a function using a local app. That service list includes google search.

The fact of the matter is that the two services are not comparable whatsoever and that for all intents and purposes siri expands upon the abilities of google regardless of the situation.
Score: 8 Votes (Like | Disagree)