Wise blog graphic_Searching behaviour blog -08 (1).svg

Search

E-commerce

Changes in customer searching behavior and trends they bring

Wise blog graphic_Search performance-03.svg

Site searchers produce up to <a href="https://econsultancy.com/is-site-search-less-important-for-niche-retailers/#i.e6bb0iahterern" target="_blank">14% of overall revenue</a>, and site search optimization increases conversion rates by 43%. Having a site search is not enough though. There is a lot of work behind optimizing its performance to provide the fastest and most accurate match to search queries. There may be various challenges related to performance, such as increased traffic and workloads, search latency, downtime, irrelevant results, and more.

How to understand what component of your system may misbehave, and what would add an excess network hop? As an [Elasticsearch consulting company](https://wise-engineering.com/expertise/elasticsearch-integration/), we want to share some of our insights. In this article, we've put together some of the best tips from Wise software engineers on how to tune Elasticsearch search performance and what metrics to keep an eye on to know how things are going.

## How to improve Elasticsearch search performance

Configuration of internal site search is a part of [custom E-commerce development](https://wise-engineering.com/expertise/e-commerce/). One of the most common tools that can be used to configure site search is Elasticsearch. It is an incredibly powerful analytics engine that is ideal for huge databases with a large catalog and hundreds of real-time customers. It supports multi-tenancy and has the ability to store, search, and analyze document files in diverse formats. There are lots of benefits to configuring site search with Elasticsearch, but the scaling of its performance is a far more challenging task. 

Elasticsearch use cases include multiple situations when sites suffer from poor search performance, and often they remain overlooked. As for reasons why it might happen, there are several aspects to consider. Here are the tips that will help you to discover slow search faster and fix performance issues.

## #1 Keep track of metrics

This piece of advice is related to Elasticsearch monitoring. To ensure efficient work of the site searcg, we need to understand the current performance level. The Elasticsearch system unites multiple components, and each of them affects its overall performance. Here are some of the most crucial indicators to keep an eye on.

- **Cluster health.** This metric provides a quick and simple overview of how well your configuration is done. Cluster health may be used to assess the overall health of a cluster or a specific index or shard. Basically, there are three health statuses: red, yellow, and green: a red state indicates that the specific shard is not allocated in the cluster, a yellow status shows that the primary shard is allocated but not the replicas; and a green status indicates that all shards are allocated.
- **Indexing rate. **This metric represents a measure of how many documents are being added to the index per unit of time. The greater the indexing rate, the greater the chance your infrastructure can keep up with all your demands.
- **Query rate and latency.** The query rate reveals how many requests per second your Elasticsearch installation can process. The query latency, in turn, reveals all the delays in all launched requests. The perfect scenario is when you have a high query rate and a low latency rate.
- **Refresh time.** Time to refresh describes how long it takes for a page to load after being refreshed. Having a shorter refresh time is preferable. A major cause for concern is rises and very long refresh times.
- **CPU usage and disk space.** Over time, more and more information is created, necessitating the allocation of more and more storage space. You'll also need a more powerful computer to process the increased volume of data and requests. Disk space and CPU usage should be tracked to ensure sufficient resources. Additional hardware resources may be required if one of these is insufficient.

To check some of the metrics, set your E-commerce platform to developer mode and begin by analyzing any issues or warnings created by your code. Debug client-side JavaScript issues in your web browser's developer tools using the console, and utilize the network tab to see how quickly (and in what sequence) scripts, pictures, and files are loading on your site.

## #2 Hardware

The important place in search performance tuning is taken up by the choice of your hardware. Adding extra hardware isn't the only solution for scaling Elasticsearch. It goes beyond that. Even the most diligent optimization efforts are useless if you don't have the necessary hardware. When you are dealing with a massive quantity of data and want to provide your customers with speedy access to it, hardware is crucial. 

Here are some suggestions for expanding your current Elasticsearch installation.

- **Basic aspects.** Cache, storage space, CPUs, and RAM are the primary areas to optimize for great use of your hardware and achieve maximum Elasticsearch performance.
- **Scaling.** Scaling Elasticsearch isn’t just a matter of throwing higher-specced hardware at a problem. You should resolve performance issues first to ensure that resources are not wasted. From the technological perspective, success depends on whether you can quickly innovate, scale up your computing powers, and leverage new content delivery channels.
- **Performance testing.** Constantly conduct performance testing to check all of your hardware requirements. Performance testing allows you to understand the way your app will react to increased traffic volumes or slow server speeds or network issues. You can thus make the required changes to the hardware required for your app. This saves you a lot of embarrassment at a later stage if the app does not function properly.
- **Use AWS.** AWS provides a secure, reliable, high-performance hosting infrastructure that can scale with your website – to massive proportions if necessary. AWS charges by the minute for its <a href="https://aws.amazon.com/ec2/" target="_blank">Elastic Compute Cloud (EC2)</a> and <a href="https://aws.amazon.com/rds/" target="_blank">Relational Database Service (RDS)</a> instance, with higher fees for more powerful hardware. Spreading the load across multiple smaller instances can often be more cost-effective and result in better performance.

Make sure you can readily increase hardware resources, such as processor performance, memory capacity, and so on, to respond to unforeseen traffic spikes.

## #3 Load balancing

Every time you start an Elasticsearch instance, you start a node. A cluster is a group of nodes that are all connected to each other. By default, HTTP and transport traffic can be handled by every node in the cluster. As a rule, there may be multiple nodes running in production with a large number of requests being processed, which might add an excess network hop.

**Load balancing** is a straightforward technique to distribute the load coming to an endpoint across multiple nodes. More specifically, it will distribute requests, gather results after processing, and then merge these results to construct and provide a final result. You may also specify the number of load balancers and configure them accordingly. As a result, load balancing decreases the strain on a specific node, enhancing performance.

How to enable load balancing in Elasticsearch? The Elasticsearch cluster includes load balancers by default. When creating an *elasticsearch.url* that refers to all/any coordinating nodes, we also recommend employing a load balancer or round-robin DNS server. A load balancer allows you to switch, add, or delete data nodes and coordinating nodes without having to edit the main configuration file or restart the service, reducing downtime for any changes.

![Improving search performance via Elasticsearch. Part 1](https://wise-site.fra1.digitaloceanspaces.com/Improving_search_performance_via_Elasticsearch_1_707fead92d.svg)

## #4 Sharding

Data in an Elasticsearch index can grow to massive proportions. To keep it manageable, Elasticsearch supports the concept of a cluster in which multiple nodes run on one or more host machines, which can be grouped into a cluster with a unique name. These clustered nodes hold up the entire data in the form of documents and provide the functionality of indexing and searching those documents.

Sharding is the process of splitting up the data in an index into numerous smaller pieces for easier querying. All queries in Elasticsearch are processed on a single thread for each shard. However, numerous shards can be run simultaneously. Therefore, many shards would result in several concurrent threads. Having the right number of shards is important for performance.

- **Decrease the number of shards.** _shrink and _split are APIs that allow you to modify the number of shards. This method may be used to maximize the number of fragments produced by a split, and it can be reversed to produce smaller output. Elasticsearch sets up each index with 5 primary shards and 1 copy of each shard by default. There will be 5 main shards for each index, with each shard replicated twice for a total of 10 shards.
- At the same time, you'll need to strike **a balance between having too few or too many shards**. There is no one right answer when deciding how many shards to utilize. The appropriate value for your needs lies within your purview of calculation. Most implementations begin with a single shard and add more as needed to get optimal performance.
- **Replicas.** Making one or more copies of the index’s shards called replica shards or simple replicas. Optimize necessary index settings that play a crucial role in Elasticsearch performance, like the number of shards and replicas. Having additional copies often improves search performance. 
- **Take care of the shard size.** The appropriate number of shards should be determined by the quantity of data in an index. In general, an ideal shard should store 30-50GB of data. For example, if you plan to gather roughly 300GB of application logs every day, having around 10 shards in that index would be reasonable.
- **Eliminate a multitude of small shards.** Search performance may suffer if there are numerous little shards because of the high number of network requests and concurrent threads that may appear.

## #5 Indices
Elasticsearch organizes its data into indices. There is no hard and fast rule about the number of indices you need to save information, but here are some of our recommendations that may help you.

- **Use several indices.** All your data doesn't have to be contained in a single index. Indices can be configured to save information for a month, a day, or even an hour, depending on the requirements of the application. 
- Your **index buffer size should be large** if your nodes are busy indexing a lot of data. The size of the index buffer is the amount of information that may be temporarily kept in a memory location before being permanently recorded on a disk. The default for this value is 10% of the total heap size which may need to be increased if your use case relies heavily on indexes.
- Take into account the fact that **Elasticsearch data cannot be changed** once it is stored. A document is then filed away in an index. However, if you need to make changes to the values in this document, you cannot just edit the values there. When this occurs, Elasticsearch generates a new document that includes the most up-to-date information.
- Elasticsearch also uses version numbers to maintain tabs on the most up-to-date content. Therefore, the most recent version is always the one with the largest number. However, the index now contains both old and new documents, which causes the index size to grow. You can fix this by **reindexing the database**. Reindexing your data ensures that you are using the most up-to-date information while also reducing your data storage needs.

## #6 Interval for refreshing

Elasticsearch takes some time to index new data, so it can't be quickly accessed. In-memory storage is made available when indexing is complete. Whenever a refresh occurs, this information is saved to the segment. Reloading a website would achieve the same effect. The frequency of refreshes is controlled by the refresh interval. Results are automatically updated once per second with a preset refresh period of one second.

The default settings are to refresh the indexes every second for search queries that have been consecutive in the past 30 seconds. If your index is often searched, Elasticsearch will update the index every second. Increase the index if you can afford to increase the period between when a document is indexed and when it becomes visible. Setting the refresh interval to a higher number, such as 300 s (5 minutes), may assist in increasing the indexing speed.

The standard refresh rate should be sufficient in most situations. However, you should consider what is most suitable for you. There is a cost associated with maintaining and regularly updating indexes. A daily refresh is sufficient if you don't want data in near-real-time; for instance, if you only deal with data from the previous day.

![Improving search performance via Elasticsearch. Part 2](https://wise-site.fra1.digitaloceanspaces.com/Improving_search_performance_via_Elasticsearch_2_8de735abf0.svg)

## #7 Enhanced queries

- **Requesting in bulk is more efficient than making several separate requests**. Typically, Elasticsearch is put to use for massive data queries. The idea that querying on particular indices may improve speed during request execution is not far-fetched. But that's not the case. If you know that the information you need is only in one index, then you should restrict your search to only that index. If, however, you wish to simultaneously search across all indexes for a certain piece of information, you should do it simultaneously.
- **Use filters to narrow your search for a specific result.** You need specific information from indices or documents. In other words, there is no purpose in retrieving all the data and then using only a subset of it. Instead, you get what you need from a database by doing a simple search. To retrieve the necessary information, you may use the _source keyword, or you can do a term aggregation to obtain one-of-a-kind data. Limit your search to only the information you need by specifying specifics like date range, matching criteria, and search phrases. By limiting your results, filters speed up your workflow.

## #8 Well-formatted documents

Every index stores information in a set of documents. If documents are well-formatted, requests may be fulfilled more quickly. Queries are slowed down by parent-child and nested fields. If you want your queries to return results quickly, you should strive to make the documents as flat as possible. Here are several of our recommendations to do so.

- **Remove documents.** Having a significant number of removed documents in the Elasticsearch index creates search performance concerns. The Force merge API may be used to eliminate a large number of documents and optimize the shards.
- **Instead of always having to calculate the amount, you can just retrieve the sum field.** Raw data is often what makes it into Elasticsearch. Logs, whether they be system or application, are of particular interest. Data processing using ES queries is often unnecessary if data is preprocessed into the appropriate fields. You may also add more fields with commonly used values. If you find yourself regularly needing the sum of five integer values in a document, for instance, you may calculate the sum at document creation time and save it in a specific field.

## #9 Plugins
Some third-party plugins generate a large amount of useless data in your database. If your database transactions become slow, look for large tables and what data is stored in them. Determine the source of the slowness and delete any data that you no longer require.

MySQL is capable of detecting and logging your slowest database requests. Enable this logging to discover the queries that are slowing down your site the most; you may then cache these searches in an in-memory store like Redis or Memcached, or delete the data (and plugin) completely if it is no longer required. If you have influence over the architecture of your database, make sure that columns that are often fetched during site searches are indexed.

## #10 Caching strategy
Decide on a caching technique for your site early on. Caching pages and queries allows your site to fetch them faster, which improves speed. At the same time, it is recommended not to cache out-of-stock levels and other ephemeral data. Doing so would result in a confusing and irritating experience for your clients.

Keep track of the correct quantity of things in stock so that out-of-stock products aren't displayed and cached as available. If you sell through a lot of different channels, you might want to use a centralized stock management system to keep track of all the goods that come in and go out and to automatically clear your site's cache when this system changes. 

![Improving search performance via Elasticsearch_3.svg](https://wise-site.fra1.digitaloceanspaces.com/Improving_search_performance_via_Elasticsearch_3_d37c1e21f7.svg)

## Wrapping up

One of the most important aspects of a successful E-commerce project is having a highly-performing search functionality. But, customer expectations nowadays are high. Even if you have an effective site search solution in place, you will always find the need to update and improve search performance to make it flexible enough to meet ever-changing customer requirements. 

Here are key takeaways from our article that will help you to improve Elasticsearch search performance.

- **Keep track of metrics** such as cluster health, indexing rate, query rate, and more to understand the current state of performance.
- Use faster **hardware** that suits your requirements.
- Use **load balancing** to decrease the strain on a specific node and distribute all the load between different nodes.
- Take care of good **sharding and indices**.
- Use **well-formatted documents**.
- Decide on a **caching technique** for your site early on.

Elasticsearch search performance tuning is rather a complex task. So be cautious while adjusting any settings in the production environment. Book [Elasticsearch consulting](https://wise-engineering.com/expertise/elasticsearch-integration/) to figure out what’s best for your case or let us help you configure everything for you. 

Perhaps you've just decided to start using Elasticsearch, or maybe you're currently an Elasticsearch user who envisions a lot of scope for your company, we are ready to help you. The Wise team can assemble a dedicated team to ensure end-to-end Elasticsearch integration and handle all the technical challenges of your case. [Contact us](https://wise-engineering.com/contacts/) to discuss details.

Learn how to tune Elasticsearch search performance and what metrics to keep an eye on in the long run. Handy tips from our development team.

10 Tips to improve Elasticsearch search performance

Learn how to tune site search performance built on Elasticsearch and what metrics to keep an eye on in the long run.

10 tips to improve search performance with Elasticsearch

Wise blog_Site search personalization-51.svg

Personalization is a broad topic in E-commerce that includes a variety of components. One of the most critical ones is the configuration of internal site search and tailoring it as per the needs of each unique customer based on their profile data, purchase history, or preferences.

Site search is sometimes underestimated, yet poor or no site search has a direct impact on brand loyalty. Up to <a href="https://www.searchenginewatch.com/2016/05/03/how-to-use-site-search-to-improve-conversion/" target="_blank">43% of buyers</a> are using the site's search box. So customizing search results may be a key source for making information available throughout your site and, thus, closing sales more quickly. 

According to Google's research, on-site search rejection costs businesses more than <a href="https://cloud.google.com/blog/topics/retail/search-abandonment-impacts-retail-sales-brand-loyalty" target="_blank">$300 billion each year</a>. Another research shows that <a href="https://www.youtube.com/watch?v=6mC3JyIJ38M" target="_blank">80% of customers</a> will exit a website after having a bad search experience. When done right, personalization of site search helps minimize bounce rate, boost conversion rate, promote brand loyalty, and much more.

However, there are lots of details to figure out before personalization. In this article, we will explain how the personalization of site search works and how your business can benefit from it. Besides, we will take a look at practical ways to personalize the customer experience through site search and mainly analyze one of the leading search engines – via <a href="https://wise-engineering.com/expertise/elasticsearch-integration/" target="_blank">Elasticsearch integration</a>. 

## What is personalized site search, and why does it matter?

Before we cover best practices, let's define some terms and answer frequently asked questions about personalization.

### What is personalization?

Personalization entails adding user-specific signals on top of all other signals to increase the relevance of your product. It is a set of practices of showing different items to different customers based on demographics, intent, preferences, browsing history, past purchases, how they use their device, and other factors.

The purpose of personalization is to **enhance the customer experience** by anticipating and meeting customers' needs and preferences before they express them. Using data on each individual customer, you can make it simpler for customers to find what they want, based not just on the WHAT of previous purchases or browsing, but also by knowing **the WHY behind their preferences**.

**Relevance** is an important metric of personalization, which is determined by delivering personalization within the context of the digital experience. **Good personalization** is about using the right channel to deliver the right message to the right audience at the right time.

### Why is personalization important today?

Customers love personalization. By providing an advanced customer experience tailored to your client's demands, you make them feel more compelled by your brand and this directly impacts conversions. As a result, personalization is a current trend in many domains, particularly in site search E-commerce, where it has long been a rallying point, and one of the major [E-commerce site search best practices](https://wise-engineering.com/blog/post/6-e-commerce-site-search-best-practices-for-2022/) as of 2022.

Let's take a look at the statistics:
- <a href="https://www.accenture.com/_acnmedia/PDF-77/Accenture-Pulse-Survey.pdf" target="_blank">91% of customers</a> are more likely to shop with businesses that identify, recall, and provide relevant offers and suggestions.
- Marketers report an average increase in sales by <a href="https://instapage.com/blog/personalization-statistics#:~:text=B2B%20and%20B2C%20personalization%20statistics" target="_blank">20% </a>when implementing personalization. E-commerce enterprises that engage in sophisticated personalization regularly achieve additional <a href="https://www.bcg.com/publications/2019/next-level-personalization-retail" target="_blank">sales increases of 10%</a> or more.
- Businesses that use advanced personalization claim a <a href="https://www.clickz.com/roi-advanced-personalization/255692/" target="_blank">$20 return on investment for every $1 spent</a>.

### What is on-site search and why personalize it? 

The search feature is frequently the first point of contact for visitors to your site. Doing it correctly might be the difference between someone becoming a lifelong customer and leaving your site and going to a competitor. Other than that, customers frequently don't know what they're looking for. They occasionally misspell search queries and require a little nudge in the proper direction.

Personalized search results are more accurate and relevant, allowing customers to quickly obtain the information they require. Search personalization is also known as query understanding. It entails leveraging information about the searcher to either personalize query probability estimation or supplement the question as a signal for query interpretation.

### When to use site search personalization?

Personalizing site search won't magically improve your product, especially when user intent is clear. However, when user intent is broad, and you offer several options, personalizing site search helps connect customers with relevant information. Site search E-commerce allows organizations to look at each click, search, and every item added to a cart to better learn what each consumer appreciates and predict what they’re most likely to buy.

### Known vs unknown visitors

All customers were once unknown visitors to your site. Even if a customer is not logged in or is visiting the site for the first time, a search may be tailored by browser type, IP location, time of day or year, mobile versus desktop, and other criteria. You can advertise items based on their IP location even if you don't know their gender, age, or purchasing history. For example, in Miami, you may promote different winter clothes than in Minnesota.

Customers who log in to their accounts and have a purchasing history, such as pages viewed, items purchased, gender, age, and so on, are considered known visitors. You may tailor results depending on their profile and site behavior. In general, the more information you know about someone, the more customizable results the system can provide.

## Top 5 site search personalization practices

Once you have customers' data, you can evaluate which practical method is the best suited to leverage personalization for your website. There are various approaches you may take to implement intelligent site search and customize search queries for your customers.

![Key approaches to site search personalization](https://wise-site.fra1.digitaloceanspaces.com/Key_approaches_to_site_search_personalization_cea6e18c6b.svg)

- **Behavior-based ranking.** According to this model, we build the search that is constantly learning from your customer user experience on your website. In other words, you analyze the behavior of your customers on the website and predict the possible actions they want to make. The user can affect the results with their own choices and browsing history. This model also refers to in-session personalization that seeks to offer relevant experiences from the moment a consumer enters your site. This approach helps us to take into account every action the user performs on the site to learn about their individual search experience. The idea is to determine the user's purpose fast and utilize that knowledge to direct them to the most appropriate product suggestions.

- **Rule-based ranking.** This approach is built by the principle that we first define the audience based on their data such as demographic information, devices, browsers, and similar, and based on such specific parameters we set up rules according to which we show particular products to particular groups of customers that drive search personalization. As a result, customers get results offered based on their profile automatically. This method is also related to history-based personalization which is based on the knowledge of who the customers are and what actions they previously have done on the website. With every new interaction, this model collects information about user intent building its knowledge base. Often, users are added to engagement groups on the guidance or recommendation of the personalization algorithm.

- **Did you mean & You might also like.** This approach allows automated search assistance for each unique customer. First, we analyze user profiles, then make suggestions on what’s better to recommend them, based on the current trends and their profiles. Besides, we can also take into account what other similar user personas are browsing, or purchasing while interacting with the website.

- **Personalized query probabilities.** When the visitors are known, you can use this approach to predict what they will search for. Searchers often repeat queries, and sometimes even within the same session, which then is saved in historical query logs. Using this information, we can analyze a searcher’s interests and personalize query probabilities.

- **Personalization as an intent signal.** Personalization might be used for query rewriting or result ranking. Searchers on a clothing website, for example, are most likely seeking things that they can wear. If we know, or can guess, the gender and measurements of the searcher, we may alter the query to filter or enhance results that fit those criteria.

## How Elasticsearch enhances personalized site search

[Custom E-commerce development](https://wise-engineering.com/expertise/e-commerce/) includes a variety of options for implementing site search personalization. Elasticsearch, a well-known and powerful technology for personalizing site search, allows you to add scalable, relevant search experiences to all of your apps and websites. It offers a wide range of search result customizing possibilities right out of the box.

**Technical details behind Elasticsearch**

It is a RESTful search and analytics engine. Released in 2010, Elasticsearch is a Java-based API built on Apache Lucene. Elasticsearch personalization indexes your data using keywords. This makes search queries go quickly because Elasticsearch searches through the keywords instead of the full text (this is called *an inverted index*).

**What Elasticsearch offers for site search personalization**

- Personalized search, superior relevance
- Accurate results
- Leverage machine learning
- Promote your best results
- Refine results during the process

**What Elasticsearch offers for site search personalization**

- **Personalized search, superior relevance.** With excellent out-of-the-box search relevance, Elastic gives all of the tools you need to create compelling search experiences that help people discover precisely what they need. Rich relevance tuning tools and cutting-edge machine learning assist you in further implementation of rule-based and behavior-based ranking.
- **Accurate results.** Elastic's search relevance, as well as plenty of customization options, allows you to curate results based on real-time corporate data. As a result, you better understand customers' demands by providing relevant search results that help them get from where they are to where they want to go. 
- **Leverage machine learning.** Machine learning may complement search and business insights to improve your search applications and customer experience, whether you're introducing new ideas to extend the effect of your search or looking for new methods to increase search accuracy. Improve semantic relevance with vector search, NLP model support, and model management.
- **Promote your best results.** With automatic, data-driven suggestions based on search metrics, you can prioritize your best-performing results. Accept, reject, or implement curation suggestions automatically. Your automatic suggestions grow increasingly relevant over time as a result of machine learning.
- **Refine results during the process.** Precision tuning allows you to easily change the recall of your findings, as well as give weight or boosts to certain data fields. Elasticsearch personalization allows you to add synonyms for frequent search phrases to tailor the search to the needs of the company and the user. There's no need to redeploy because the modifications to search relevancy are live in production as soon as you're ready.

## Best cases of personalized site search in E-commerce
Here are some of the best examples of site search personalization to learn from.

### <a href="https://www.franshalsmuseum.nl/nl/" target="_blank">Frans Hals Museum</a>

This website reflects the duality of the classic and contemporary, showcasing a collection of 17th-century art alongside modern pieces and colorful design.

**Challenge**

The website offers plenty of tickets to reserve and buy for exhibition programs in the art museum. The goal is to make it easy for customers to use the website and give them useful recommendations. 

**Solution**

The *Discover* section provides some interesting and unexpected methods to view the museum's collection. Visitors can explore artworks by mood, color, medium, or artist, or they can choose *Random* to be provided with a group of items that are produced casually. A navigation bar on the site helps visitors to find key information, such as the museum's location, opening hours, and daily events, without having to go through the *Visit us* pages.

It is not necessary to register on the site because the Frans Hals Museum uses an in-session personalization model. The site actively analyzes all requests made by each specific user and outputs not only all relevant results but also those that are similar in style to artists, exhibitions, locations, and more. By opening each of these results, the visitor sees another portion of recommendations that match their interests.

### <a href="https://www.zennioptical.com/" target="_blank">Zenni</a>

Zenni, as a pioneer in the eyewear market, provides a one-of-a-kind experience to its clients. Augmented reality (AR) is used in this technology, which lets people try on the glasses on their own devices before buying them. 

**Challenge**

Zenni wanted to create a more engaging customer experience by providing personalized product recommendations through its digital store.

**Solution**

Customers can register and enter their prescription information, such as pupillary distance (single or dual PD) and preferred lens type (single vision, bifocal, progressive, or readers). All search results will be personalized according to the provided parameters.

Other than that, the website offers autocomplete recommendations with visuals by making the search box visible. For example, when users type *kids*, product photos from the Kids category are displayed.

### <a href="https://www.ryanair.com/ua/uk" target="_blank">Ryanair</a>

Ryanair is based in Dublin, and its flights link 37 countries and 225 locations. Ryanair has positioned itself as Europe's greenest airline in recent years.

**Challenge**

Ryanair intends to show customized search results to customers to preserve its competitive edge long-term and to continue providing higher value for customers.

**Solution**

The great feature on the website is pre-flll search options based on past searches, purchases, and locations. When customers use the search function again, Ryanair includes a dropdown of suggestions of appropriate terms and categories. This practice works well both for customers that are browsing and those that have arrived on the site with a specific repeat purchase in mind. Browsing visitors will be able to select from a range of relevant categories while returned customers can quickly select the options they require.

### <a href="https://www.patagonia.com/home/" target="_blank">Patagonia</a>

Patagonia is a manufacturer of outdoor apparel and equipment for the silent sports of climbing, surfing, skiing and snowboarding, fly fishing, and trail running.

**Challenge**

The aim of Patagonia was to offer query predictions for customers that are tailored to the site’s specific content, so users can be confident they’re finding their desired content.

**Solution**

Patagonia provides a great example of personalized autocomplete on E-commerce site search. When a user starts typing, autocomplete displays options in a menu beneath the search. Users can click the recommendation or use the arrow keys to go up and down the suggestion list and choose an alternative.

## Key takeaways
The following are the important conclusions from the article:

- **Personalized customer experiences** are integral to business growth. 
- **Personalization** is the process of understanding your online users' preferences using historical information and prior interactions with your site. Your customers tell you which products they like to buy with every search, click, bounce, add-to-cart, and purchase.
- Customers who use your **on-site search capabilities** are telling you exactly what they're searching for, and it’s important to help them find it promptly before they leave.
- **Site search personalization** is often overlooked but this component is a powerful way to offer better recommendations and help customers find what they’re looking for. The key to success in site search personalization is to use the right algorithms at the right touch point of the customer journey.
- Among **primary methods to personalize site search** are behavior-based and rule-based ranking. Best practices to implement site search include autocompleting, autocorrect, personalized query probabilities, personalization as an intent signal, *did you mean & you might also like*, and more.
- **Elasticsearch personalization** helps to configure all of the mentioned methods of personalization and even more. It puts search power at the heart of the fastest, most reliable, and most scalable E-commerce experiences.

If you are looking for additional guidance on the implementation of site search functionality, the Wise team can provide you with [Elasticsearch consulting](https://wise-engineering.com/expertise/elasticsearch-integration/) or launch an audit of your existing search engine. We can also assemble a dedicated team to ensure end-to-end Elasticsearch integration and handle all tech challenges of your specific case.

At Wise Engineering, we implement advanced personalization that helps deliver real-time recommendations to users based on multiple factors, including purchase history, card additions, and more. [Contact us](https://wise-engineering.com/contacts/) to discuss details.


Check out how you can provide your customers with personalized online shopping experiences with site search personalization.

Site search personalization: How to configure & use cases

Check out how you can provide your customers with personalized online shopping experiences with the help of an internal site search.

Solr_vs_Elasticsearch_preview.svg

Learn about Solr vs Elasticsearch use cases, features, and other critical differences for E-commerce projects offered by both search engines. 

Solr vs Elasticsearch: In-depth comparison for E-commerce [2022]

Learn about use cases, features, and other critical differences for E-commerce projects offered by both search engines.

Solr vs Elasticsearch: In-depth comparison for E-commerce

search budgeting (1).svg

E-commerce search is an extensive domain, and the computing efforts to on-site search tuning can be rather intensive. But do all the search functions require the same amount of resources to implement? Where should you start when improving the search experience on your website, and what will bring you the most value at different steps? 

At Wise Engineering, we've been working with E-commerce site search implementation and tuning for 15 years, and here's a general roadmap we recommend to our customers. It will help you approach search budgeting more strategically, see where you're now, and where you're going in the long run.

## How to approach search budgeting strategically

As E-commerce companies begin to realize the benefits of improved search experience to their customers, it is now always clear how to start the optimization process and how much resources will be needed for the configuration of each particular feature or selected optimization. 

We’ve divided the whole process of enterprise search tuning into levels depending on the engineering efforts they require. Though we do not provide the exact numbers, you can clearly see what will cost you the most. The higher on the pyramid you go, the more advanced the task is, and the more budget you will need to allocate.

For us, the pyramid also illustrates the logic roadmap of the improved E-commerce search. You can easily start with the first levels, measure the results and move up the pyramid towards the advanced search experience for your E-commerce platform.

This roadmap is applicable to all existing search engine solutions you might currently use – <a href="https://www.elastic.co/" target="_blank">Elastic search</a>, <a href="https://www.algolia.com/" target="_blank">Algolia</a>, <a href="https://solr.apache.org/" target="_blank">Solr</a>, and others. Some features require a few configurations of the search engine and they are on the initial layers of our pyramid. Some search capabilities will be engineered specifically for your business and are located on the top. 

![E-commerce search budgeting](https://wise-site.fra1.digitaloceanspaces.com/Investing_steps_for_internal_site_search_799e3253a6.svg)

Let's analyze each of the levels in detail.

## Level 1 – Basic search

The first level usually starts with UX improvements and functionality available out-of-the-box by most search engines and requires a few simple configurations that don’t take much developer’s time. Before starting any significant and time-consuming configurations, try enabling the autocomplete feature, compose separate thesaurus specific to your E-commerce segment, configure a fuzzy search and stopwords filtering. Measure the performance and decide on further investments.

- **UX improvements**. They can, for example, include: creating a visible search box, changing the search box location, highlighting search results, using microcopy for a search bar, showing the number of results, keeping recent user’s queries, and more. 
- **Autocomplete**. Do not consider this a simple feature that reduces typing efforts for customers. Despite the simplicity in the configuration, autocomplete allows your customers to express their search intent aligned with the query understanding of your online store. This helps you increase the search quality with minimum investments.
- **Synonyms**. For every industry you’re selling products or services to, there is a specific synonym range your on-site search should be able to interpret. You can compose a thesaurus specific to your business and add it to the search engine configurations.
- **Fuzzy search**. This configuration helps to ignore typos, plural forms, case forms and minimize the failed search results.
- **Stopwords filtering.** Stopwords are the common words that do not bring any valuable context in most search cases and only expand the search results. At first, the stopwords list can include articles, prepositions, auxiliary verbs. Then it can be filled with the words you would define as non-important as you analyze the queries further.

## Level 2 – Search refinement

When the basic search configurations are made, you can proceed with improving how users browse the search results and refine their searches. People hate any extra work, and your customers are no exception. It is always more pleasurable to click on refinement options than browse through multiple paginated search results.

- **Faceted search** helps users refine the search results once they have a listing in response to their query. This could be done based on numerous additional parameters like size, color, brand, price, and more. With facets configured, search results change between the searches depending on the selected parameters.
- **Filters** do not work this flexibly and help only to eliminate the search results based on initial criteria. For example, users can select either browsing shoes for adults or children, but they can not select both and add more parameters to get the search results changed. 
- **Filter suggestion** feature enabled allows users to search directly within a selected category. This also helps searchers avoid information overload and find what they are looking for faster.
- **Sorting** helps to order search records based on specific attributes like price (from lowest to highest or vice versa), popularity, and other parameters.

If it is relevant to your business, at this step you can also configure a **location-based search** and provide results relevant to your customers from different locations.

## Level 3 – Search ranking

We approached the level that would require some more implementation time, and, thus, resources. We included fine-tuning **textual relevance**, **user-based** and **attribute-based ranking** in this step. All of them are related to the search ranking and aim to show at the top those results that address your business goals.

At this point, the search engineering team should closely collaborate with the product or business development teams. The sample questions they would define at this step are the following:

1. What results should be shown at the top?
2. Does merchants' rating influence the relevance?
3. What role does the price play? Do we show the least expensive goods first?
4. What fields are included in the searching process and what should be added or removed?
5. What fields in a product are the most important?
6. How to evaluate the relevance of each product for different customer personas?

Working with your development team at this level, you can set up rules that drive search relevance based on specific parameters.

- **Attribute-based ranking** helps to match shopper queries with not only product names, but also brand names, product types, descriptions, categories, colors, or several of these at the same time.
- **User-based ranking** will take into account demographics, location, user preferences. All the criteria known about the particular user.  
- **Textual relevance** tuning is especially important for the cases when the complexity of catalogs is combined with user-generated product listings. 

## Level 4 – Relevance tuning

Custom ranking surely matters in search optimization and helps address some business objectives, but it is a search relevance model in its broader sense that we want to achieve. 

We recommend starting this step with an **analysis of zero-result queries** that are usually the best source of insights. Why does your system fail to provide what users are searching for in particular cases? At this point, we help our clients compare successful and failed sessions, look more precisely at searches that return zero results, and find commonalities between them. Ideally, we would divide zero-searches into classes and find solutions to turn them into successful ones.

During the previous step, we defined the ranking rules that can be now neglected for some specific use cases. By defining **query-dependent ranking factors**, you can achieve dynamic ranking. For example, you have the rule to show more expensive products on top of the search results, but if a person types the word free in the search bar this rule should be neglected. At this step, with the right tech team, you can fine-tune your search engine to provide the most relevant search results specific to your business use cases.

Furthermore, to make the search system understand which results are relevant for broad or ambiguous queries, you would need to configure **query expansion**. This is a technique when the original user’s query is expanded by adding additional words that help to narrow the query semantics. Query expansion helps determine the searcher’s intent and route users towards a more refined search query.

## Level 5 – Advanced search

We have to admit, this level is the most complex and resource-consuming, yet once you reach it, there are, really, no limits. At this step, you will have the capabilities to rethink how your search performs on the highest level and optimize the enterprise search engine with the most advanced technology and methods. Instead of focusing on the efficient ranking algorithm, you will have the ability to create an efficient query understanding mechanism, an advanced system that will learn along the way and focus on query performance as a key success metric. 

- **Query refinement analysis** helps to look at the process of refining (narrowing down) the initial user searching item and analyze user behavior after the results were provided. This data will help to provide narrowed down paths to results that your users await.
- **Query understanding** is the process of establishing the searcher's intent before providing any results. Your search engine will be able to automatically determine when a search query is broad or ambiguous and expand it to provide relevant results.
- **AI-based search personalization** allows you to serve users personalized search results based on their previous searches, activities, and purchases. The tech team will need to track and collect users' signals, interpret them, and build relevant user profiles. The search results will be modified based on these profiles but the system won't neglect the textual relevance you've built during the previous steps. 

## Configurations running in parallel

As you move along the steps of E-commerce site search optimization, two aspects should be considered no matter where you are in this roadmap. They are internal search monitoring and performance tuning. You can also learn more [E-commerce site search best practices](https://wise-engineering.com/blog/post/6-e-commerce-site-search-best-practices-for-2022/) from our recent blog post.

### Analytics for internal search

Some queries lead to great performance. Some queries deliver zero results. Without measuring this data, you can not move to optimization at any step. Search analytics provides you with important information on what your customers want, how they describe the products or services they are looking for, and gives you hints on how you can optimize your offers and overall search experience.

For example, you can track:

- Queries users enter
- Most popular results
- Most-bought products after search 
- Most-used filters
- Average position of search result click
- How deep do users go in pagination (do they find good results on the first page?)

Though analytics is a topic that never gets underestimated and is often included in the project scope, setting it up properly is a daunting task for the tech team. Engineers will need to build a data tracking pipeline, normalize queries (taking into account synonyms, typos, word forms), make the analytics privacy compliant, and visualize the results for stakeholders. Analytics also opens a possibility to run A/B tests and evaluate introduced optimizations during and after each level we’ve described above.

### Performance tuning

Because search experience is often the primary path to purchase, making the search for enterprise perform faster even for milliseconds can lead to increased revenue. From day one as you introduced search to your E-commerce store, monitor search for downtimes, speed, and vulnerabilities for the sensitive data you might collect.

When our team deals with on-site performance tuning, we start with tech resources analysis that includes analysis of the computing, memory storage, and network you use to run the search on your platform. After this analysis, we proceed with cluster configuration optimization and index configuration to help the search engine perform faster. All the additional optimizations depend on the business goals set for the internal search.

## Reach us for E-commerce search consulting 

Enterprise search is not something you set and forget. There is always room for investment, and now you have a clear vision of where you might start. You can hire the Wise Engineering team if you're looking for site search experts that can ensure quality, best performance, and scalability for your on-site search.

Learn more about our [Elastic site search expertise](https://wise-engineering.com/expertise/elasticsearch-integration/) and contact us to book a consultation to discuss your specific business context.
 

Approach E-commerce search budgeting more strategically. Check out a roadmap for E-commerce site search optimization and get more search conversions. See blog>>

E-commerce search budgeting: Where to invest first?

Check out a roadmap for E-commerce site search optimization and approach search budgeting more strategically.

site_search_main.svg

Discover customer behavior trends of 2022 and E-commerce site search best practices that will help you meet customer expectations and boost sales. Read more >>

6 E-commerce Site Search Best Practices for 2022

Discover customer behavior trends of 2022 and E-commerce site search best practices that will help you meet customer expectations.

6 E-commerce site search best practices for 2022

search.png

An E-commerce store without an in-house search is like a grocery store without labels. You'd be left wandering down the aisles of the store with the hope of finding what you are "searching" for by chance. For the store owner, this is a bad practice as some customers might get jaded before finding anything, and the outcome of this means no or little business. This case can also be applicable to an E-commerce store if we replace the offline attendants with the online visitors who use the search widget.
 
The efficiency of the internal search has surpassed the need for navigation on the website by users and has also become a great tool for the store owners. It helps to drive sales and increase conversions. 
 
The tech surrounding the functionality of the search widget is beyond the generic search present on most websites as this only gives results based exclusively on the search keyword. For example, if the word "shirt" is typed into the search bar, the search results are not expected to go beyond the word "shirt", and we could say this type of search is based strictly on the search keyword and not influenced by the user experience and the other factors.
 
![Copy of cases-image-1.png](https://wise-site.fra1.digitaloceanspaces.com/Copy_of_cases_image_1_57331e4918.png)
 
Currently, there are two main options that we will describe at a high level in this post. For most CTOs this sounds like "build vs buy" and in many cases. The decision depends not only on what is better but what kind of company you are in: are you a tech-first company or a business-first company.
 
## Building an in-house search for E-commerce

Building an E-commerce search yourself involves the configuration of a search engine, like <a href="https://wise-engineering.com/expertise/elasticsearch-integration/">Elastic site search</a>, ManticoreSearch, or Solr. There are a handful of search features to consider and scope when building an E-commerce search, for example, search types, autosuggest, typo tolerance, and many other built-in features.
 
These solutions provide a very flexible search platform. Setting it up can take a month at most, and this relies upon certain factors, for example, the number of products in the catalog. The need to structure and optimize data for search cannot be left out as it is crucial to the solution functionality. The data should be prepared for the search integration. This might include:
- Normalizing attribute values.
- Configuration of basic and advanced features that would improve user experience.
- Structuring and optimization of data then the ranking of search results based on their relevancy.

Pros and cons of using self-hosted search

![Pros and cons of self-hosted search.png](https://wise-site.fra1.digitaloceanspaces.com/building_in_house_search_c2cdd352f3.svg)

## Adopting 3rd-party vendor
 
SaaS companies specialize in specific niches and always optimize their products to get excellent results based on the niche the product specializes in. They have focused on creating an efficient site search that possesses features like text auto-complete, spell checker, natural language processing (NLP), advanced stemming, and more.

Pros and cons of using SaaS

![Adopting 3rd-party vendor](https://wise-site.fra1.digitaloceanspaces.com/adopting_3rdparty_vendor_b9503a7048.svg)

## Where to start?

If you are wondering where you should start while doing your research on how to improve search on your site there are a few questions to answer first. Do you know what problem you are solving or what goal you pursue? If so – great. If not, you might want to start by analyzing a few basic metrics to understand how your search performs and how it affects sales.

**Conversion Rate**

Conversion rate is arguably the most important metric, it has a significant impact on your site's profitability. When your website search functions as well as it should, it becomes a salesman that leads users to their desired destination pages. This way, search helps to engage and convert potential customers.

**Bounce Rate**

The bounce rate is the rate at which new visitors open your site and immediately click away without interacting with it. The lower the bounce rate, the better. A high bounce rate can mean several things and common problems include poor design, low usability, or high search time. This metric is a great indicator for improvement.

**Search Page Speed**

Page Speed is the amount of time that it takes for a web page to load. A page's loading speed is determined by several different factors. This issue can be solved by site search providers or the in-house development team.

**How Deep Search Users Go**

A lot of users get jaded after searching the first and second pages, so it's up to the search provider to optimize the search tool to bring results relevant to the keywords up on the search page. This data is very important to analyze while working on improving internal search.

**Relevancy Ranking**

The task of the search engine is to ensure that the best-matched results are shown up first. Search irrelevance turns into poor user experience and this affects sales directly. Constantly improving your data tagging, assessing site search performance, and tweaking your relevance algorithms will be key to search relevancy.

**A/B Testing**

This is an important tool that involves a performance comparison between two versions of software (a search results page, in this case). A/B testing can be used to determine which search changes to launch after monitoring certain metrics like the ones listed above. A/B testing approach will have significant implications for what kinds of changes you need to create, test, and then launch.

**Team Cooperation**

The last but not least significant thing to start with is building an in-house team for E-commerce development or hiring an outsourcing partner. Experienced engineers, data analysts, and marketers are required to work in an across-functional team. Look for people with an expertise-based approach that have experience with internal search integration and optimization specifically.

*** 
To sum up, the pros and cons of both options have to be considered in terms of how it fits the E-commerce store and the budget, among other factors. Choosing how to develop your E-commerce search, shouldn't be a decision to jump into. Start with defining what goals you want to achieve by working on search and evaluating all options carefully.
 
Wise Engineering experts provide consulting services on [custom E-commerce development](https://wise-engineering.com/expertise/e-commerce/) and will gladly discuss your tech challenges and business goals via call or email. 

Get handy tips on how to integrate in-house search to your E-commerce platform and steps to start.

Building an in-house search for E-commerce | Wise Engineering Blog

Get handy tips on how to integrate internal search to your E-commerce platform and steps to start.

Building an in-house search for E-commerce

05_cases-01.svg

## Problem

Nowadays search becomes a primary source of resource discovery. So having a modern search engine that powers it is a vital part of a marketplace or other E-commerce site. E-commerce platforms contain gigabytes of data, usually not well structured and being input by different sellers so searching within it is not a trivial thing.

There are a variety of site search engines and services built on top of them so it is not an easy thing to choose the one that will perfectly fit requirements and be a good platform for further search results optimization and improvements.


## Goals and challenges

- Choose between SaaS and self-hosted engines
- Flexible search results tuning mechanisms
- Relatively low infrastructure maintenance
- Reach search experience features implemented


## How Wise Engineering helped
We have created a list of available search engines and services on a market that fit the required goals. Helped to create an evaluation matrix with a list of required search features and tuning capabilities. Filtered candidates to 2-3 vendors and created a plan for further evaluation of search results, integration efforts and further infrastructure maintenance.


## Results
- Created a finalized list of 2-3 potential vendors to evaluate.
- Listed search features to be added with new search vendor.
- Planned further evaluation process, tools and KPIs.
- List regressions and workarounds if current search implementation has some specific use cases.


## Implementation process

### Evaluation matrix
First, we created a long list of search engines and services and combined them into a matrix where we’ve put features and capabilities we want to have in a new search engine or provider. Each feature has its weight so it sets priorities and helps to evaluate a final result better. Some engines/providers may have features implemented in a different way and not always meet requirements. In this case, additionally to feature weights we created another feature evaluation dimension with three simple states as “doesn’t meet requirements”, “par” and “exceeds requirements”. So if the feature we want to have in search is important (which is defined in its weight) but has “doesn’t meet requirements” value for it we exclude this search engine/provider from the further evaluation process. At this point, we had an evaluation matrix with only 3 engines and services left that met all the requirements we’ve set on the first step. It was a mix of services and engines to be self-hosted so we needed to create different evaluation and implementation plans for them.

### Sales calls and estimation
For services, we scheduled calls with the sales team and went over features we wanted to have, implementation concerns, etc. At this step, it was important to estimate an annual cost of search service according to the features we want, search rates, potential index size, etc. The final price was an important factor in final vendor decision making. For self-hosted engines, we calculated an estimated hardware quantity required to support the search. Also, estimated a potential ops time required to support search infrastructure which also leads to additional cost.

### Demo search results evaluation
There are a few more additional steps we took to help. We created a step by step plan for demo search evaluation. This plan included a mockup of possible search results evaluation tool to run side by side comparison of current search results and demo search setup with new vendors and services. We also created a regression list of features in current search implementation and possible workarounds in new ones. There were some domain-specific search filters and search query expansions that weren’t possible to port to every new search candidate, so we designed a workaround for them to make sure new search candidates didn’t get rid of some search features users may have been used to.

## Results

Evaluation matrix and further vendors evaluation process have been created. We planned tools to be built and KPIs to measure and prepare for a final search engine decision recommendation, regressions, and possible workarounds. A list of new and modern search features has been created, they will be implemented after a new search vendor is chosen and integrated.


Check out the main steps of the site search implementation and how the right tech team can help.

Wise Engineering Blog

Changes in customer searching behavior and trends they bring

10 tips to improve search performance with Elasticsearch

Site search personalization: How to configure & use cases

Solr vs Elasticsearch: In-depth comparison for E-commerce

E-commerce search budgeting: Where to invest first?

6 E-commerce site search best practices for 2022

Building an in-house search for E-commerce

Choosing search vendor

Get more product
development insights from
our experts

Business

Join our team

Wise Engineering Blog

Changes in customer searching behavior and trends they bring

10 tips to improve search performance with Elasticsearch

Site search personalization: How to configure & use cases

Solr vs Elasticsearch: In-depth comparison for E-commerce

E-commerce search budgeting: Where to invest first?

6 E-commerce site search best practices for 2022

Building an in-house search for E-commerce

Choosing search vendor

Get more product development insights from our experts

Get more product
development insights from
our experts