We Tested Google’s New AI Mode vs Traditional Search—Here’s What Contractors Need to Know | Energy Circle, LLC

We Tested Google’s New AI Mode vs Traditional Search—Here’s What Contractors Need to Know

By Emily Silverman |

AI is changing everything, including how your customers search for your services online. 

Google launched Bard (now Gemini) in 2023, followed by AI Overviews (AIO) in 2024. Now, they’re rolling out a brand-new feature, called AI Mode.

What Is AI Mode? 

AI Mode is a new feature from Google that completely alters the search experience. It allows you to ask complex, multi-step questions and receive a customized, AI-generated response instead of a list of links. You can even ask follow-up questions, turning search into a conversation. It’s very similar to ChatGPT, but lives right within the Google ecosystem.

Here’s what it looks like in action: 

How AI Mode Compares to Traditional Google Search

We ran a series of test searches related to insulation, HVAC, and solar to see how results in Google’s new AI Mode compare to traditional search results. 

Specifically, we looked at the Map Pack and first-page organic results in a standard Google search, then compared them to the AI-generated response and linked sources shown in AI Mode. 

Our goal was to understand whether—and how much—the results differ between the two search modes.

Two Test Groups: BOFU & TOFU 

We broke our test searches into two groups: bottom-of-the-funnel (BOFU) searches and top-of-the-funnel (TOFU) searches. 

  1. BOFU searches are specific and geo-targeted, signaling that the user is ready to convert. In a traditional Google search, these typically return sponsored ads (excluded from our analysis), a Map Pack, and organic results. In AI Mode, users see a generated response that usually includes a list of relevant companies with links to their websites, and a sidebar of related sources.
  2. TOFU searches are more educational, signaling that the user is still doing their research. These usually trigger an AI Overview followed by organic results in traditional search. In AI Mode, they generate a more detailed response with a similar sidebar of linked sources. Since these searches aren’t about trying to find specific companies, contractors are rarely mentioned in the response, but specific blog and service pages from contractor websites are often linked to as sources.

Here’s what we found. 

Top-of-the-Funnel Test Searches 

Test Search: “Insulation Companies Near Me” 

Test Search: “Insulation Companies Near Me”

Test Search: “AC Repair St Pete, Florida” 

Test Search: “AC Repair St Pete, Florida”

Test Search: “Residential Solar Companies” 

Test Search: “Residential Solar Companies”

BOFU Test Searches 

Test Search: “What Is the Heat Pump Tax Credit for 2025?” 

Test Search: “What Is the Heat Pump Tax Credit for 2025?”

 

Test Search: “Do I Need More Insulation in My Attic?”

 

Test Search: “Do I Need More Insulation in My Attic?”

 

While this was a purely anecdotal exercise, and things seem to be changing day to day, our findings point to a clear trend:

 

While this wasn’t a statistically significant study, our findings point to a clear trend: AI Mode consistently delivers different results than traditional Google search, even when using the exact same query under the same conditions. 

The results suggest: 

  1. Ranking well in organic search doesn’t guarantee visibility in AI Mode. Our test searches show that even if you are ranking organically, you have a pretty good chance of being overlooked (displaced) in AI Mode.
  2. Even success in AI Overviews doesn’t always carry over. 
  3. The one constant? Local Map Pack rankings held strong across every test, suggesting that local SEO will continue to play a key role in ranking well across all of Google’s products.
  4. Third-party directories and review sites (Yelp, Angi, Home Advisor, etc) are displaced at similar rates as contractor websites. We expected these sites to carry more weight in AI Mode and were surprised by this result. 

Some Context…

AI Mode is still brand new, and the marketing industry is in the early stages of understanding how to optimize for it. While the exact ranking factors remain unclear, early signs suggest they may differ from those that drive traditional SEO performance.

At Energy Circle, we’re actively testing, learning, and sharing insights alongside the broader industry. Experiments like this help us navigate the rapidly changing search landscape and adapt our marketing strategies as Google reshapes how customers find your services online. 

Want to see how your marketing efforts stack up? Get your free competitor analysis today to see what you and your competition are doing right (and wrong) to attract customers online.