How Behavioral Data Can Help Improve Search

Google has become the world’s most popular search engine for a good reason: You usually find what you’re looking for very quickly. Similarly, Amazon’s intelligent categorization of products has made it easy to find what you want to purchase.

Google and Amazon both invest tremendous amounts of money in search science – or, optimizing the algorithms and techniques used to produce relevant content. Not surprisingly, these algorithms go well beyond matching search terms by looking at your personal search history, purchasing history, and other behavioral factors.

In this article, we will look at how conventional search works, how behavioral techniques can help dramatically improve conversion rates, and how you can tap into these innovative search and ranking techniques in your eCommerce business.

How Conventional Search Works

The most basic search algorithms attempt to match a search parameter with some data.

For example, a basic Solr query may look something like this:

http://localhost:8983/solr/q=name:laptop

The query selects all product names from the products table where the name exactly matches the term “laptop”.

You can also use wildcards to match terms that are similar. For example, you can use this Solr query to match names starting with “lap”:

http://localhost:8983/solr/q=name:lap*

This query could return products like:

• Laptop
• Lapdog
• Lapd

More advanced search algorithms attempt to find better matches by incorporating other data or encourage better search behaviors. For example, autocomplete functionality can be used to recommend popular search terms and discourage a partial search (e.g. “Laptop” versus “Lap”). The results may also be ranked using various scoring algorithms, such as the product’s creation date, popularity, or other attributes.

These Solr queries may look more like this:

http://localhost:8983/solr/q=laptop&sort=inStock asc, score desc

Elastic Search (and Lucene) score search results using a Practical Scoring Function that looks at term frequency, inverse document frequency, and a vector space model for multi-term queries. In addition, the results may be filtered by index, document type, or other contextual logic to improve relevance for similar terms (e.g. showing apple fruit results versus apple computer results when a user is search for one or the other).

Example of Elastic Search with Suggested Results

These simplistic examples demonstrate how core search functionality works, but it’s just scratching the surface of what’s possible with modern search science.

How Behavioral Data Helps

Modern search algorithms incorporate behavioral data to optimize results. For example, behavioral algorithms may look at how many users clicked on a product, added it to their cart, or made a purchase after searching for a specific term.

The implementation of these search algorithms often involves the integration of search data and visitor data from sources like Google Analytics. Often times, eCommerce businesses turn to third-party services to help with this integration given the complexity.

Challenges with Behavioral Search

Behavioral search algorithms may seem pretty straightforward to implement, but there are many challenges that may not be immediately obvious. In fact, large companies, like eBay, didn’t discover these issues until years after optimizing their search.

Some examples of these issues include:

• Position bias is the tendency for top search results to be clicked and purchased more frequently than lower search results, which reinforces their position, even if there is a better product lower in the results. These issues can be resolved by normalizing each click based on its expected clicks given its position in the search results.

• Price bias is the tendency for shoppers to prefer cheaper products over more expensive products, which means that higher priced items may be ranked artificially lower due to their longer buying decision process. These issues can be resolved by normalizing the expected sales for products at a certain price point.

The solution to many behavioral search patterns, including the two biases mentioned above, is the normalization of data. By determining an expected result, you can rank products by their under- or over-performance of these expectations rather than ranking them linearly.

Incorporating Behavioral Data

Avatria Convert was created to improve conversions by showing customers what they’re most likely to purchase. By integrating Google Analytics customer behavior and shopping data with site search functionality, we provide a plug-and-play, customizable, machine learning-powered solution that integrates with eCommerce systems and search engines.

The first step is setting up Google Analytics Enhanced Ecommerce, which enables you to track eCommerce transactions within Google Analytics. While there are many different eCommerce analytics solutions, Google Analytics Enhanced Ecommerce is very popular due to its free or low cost and extensive feature set. See our previous article on setting up Google Analytics Enhanced Ecommerce for a detailed overview.

Once you’ve set up Google Analytics Enhanced Ecommerce, here’s how Avatria Convert leverages that data to produce insights:

1. Connect Your Account – Integrate with Google Analytics with a click of a button and we will immediately begin analyzing your data to generate recommendations for your website.
2. Generate Rankings – We use machine learning to understand your customers’ shopping patterns and predict the optimal product order to boost conversions. These rankings can be generated for any category page, search results, or other location.
3. Integrate the Results – We will account for changes in your customers’ behavior and automatically re-prioritize products accordingly to optimize conversion rates. As you continue to use the solution, we will deliver increasingly optimized results.

If you’re interested in learning more, sign up for a free trial and start improving your conversions today!

The Bottom Line

Search science has rapidly evolved over the past several years from simply matching terms to scoring the results to incorporating behavioral data. While the more advanced techniques were pioneered by Google and Amazon, these technologies are quickly becoming accessible to mid-sized eCommerce businesses using tools like Avatria Convert.