Choosing Search Vendor

CHOOSING SEARCH VENDOR

PROBLEM

Nowadays search becomes a primary source of resource discovery. So having a modern search engine that powers it is a vital part of a marketplace or other e-commerce site. E-commerce platforms contain gigabytes of data, usually not well structured and being input by different sellers so searching within it is not a trivial thing.

There are a variety of site search engines and services built on top of them so it is not an easy thing to choose the one that will perfectly fit requirements and be a good platform for further search results optimization and improvements.

GOALS AND CHALLENGES

  • Choose between SaaS and self-hosted engines
  • Flexible search results tuning mechanisms
  • Relatively low infrastructure maintenance
  • Reach search experience features implemented

HOW WISE ENGINEERING HELPED

We have created a list of available search engines and services on a market that fit the required goals. Helped to create an evaluation matrix with a list of required search features and tuning capabilities. Filtered candidates to 2-3 vendors and created a plan for further evaluation of search results, integration efforts and further infrastructure maintenance.

RESULTS

  • Created a finalized list of 2-3 potential vendors to evaluate.
  • Listed search features to be added with new search vendor.
  • Planned further evaluation process, tools and KPIs.
  • List regressions and workarounds if current search implementation has some specific use cases.

IMPLEMENTATION PROCESS

Evaluation Matrix

First, we created a long list of search engines and services and combined them into a matrix where we've put features and capabilities we want to have in a new search engine or provider. Each feature has its weight so it sets priorities and helps to evaluate a final result better. Some engines/providers may have features implemented in a different way and not always meet requirements. In this case, additionally to feature weights we created another feature evaluation dimension with three simple states as "doesn't meet requirements", "par" and "exceeds requirements". So if the feature we want to have in search is important (which is defined in its weight) but has "doesn't meet requirements" value for it we exclude this search engine/provider from the further evaluation process. At this point, we had an evaluation matrix with only 3 engines and services left that met all the requirements we've set on the first step. It was a mix of services and engines to be self-hosted so we needed to create different evaluation and implementation plans for them.

Sales Calls and Estimation

For services, we scheduled calls with the sales team and went over features we wanted to have, implementation concerns, etc. At this step, it was important to estimate an annual cost of search service according to the features we want, search rates, potential index size, etc. The final price was an important factor in final vendor decision making. For self-hosted engines, we calculated an estimated hardware quantity required to support the search. Also, estimated a potential ops time required to support search infrastructure which also leads to additional cost.

Demo Search Results Evaluation

There are a few more additional steps we took to help. We created a step by step plan for demo search evaluation. This plan included a mockup of possible search results evaluation tool to run side by side comparison of current search results and demo search setup with new vendors and services. We also created a regression list of features in current search implementation and possible workarounds in new ones. There were some domain-specific search filters and search query expansions that weren't possible to port to every new search candidate, so we designed a workaround for them to make sure new search candidates didn't get rid of some search features users may have been used to.

RESULTS

Evaluation matrix and further vendors evaluation process have been created. We planned tools to be built and KPIs to measure and prepare for a final search engine decision recommendation, regressions, and possible workarounds. A list of new and modern search features has been created, they will be implemented after a new search vendor is chosen and integrated.

Wise Engineering © 2020 All Rights Reserved.Privacy policy