72 - Semantic Search Indexing
Increasing search accuracy by adding semantic information about a piece of text to the index, in addition to keyword search.
Search Engines became famous for their keyword-based information retrieval. Adding semantic information about a piece of text can increase search accuracy. Adding not only the text, but also it’s vector will allow to search for the intent and semantic meaning of the search terms, in addition to keyword search.
NMSLIB (Non-Metric Space Library) is a fast similarity search library that can find objects with a minimal (cosine) distance to other objects. When handling a question, you calculate its vector and then find closest embedding vector from the NMSLIB-index. Calculating vectors can, for example, be done with the Universal Sentence Encoder. NMSLIB has become a part of Amazon Elasticsearch Service.
This article is part of the project Periodic Table of NLP Tasks. Click to read more about the making of the Periodic Table and the project to systemize NLP tasks.