Semantic Search API for Retrieval Augmented Generation (RAG)

What is Semantic Search?

Semantic search is about searching content using natural language, exactly the way Google does. When using semantic search, you don't need to search for exact keywords (also known as keyword search) as the AI is able to understand your request and interpret it.

Let's say that you are an HP printers reseller and that you have have thousands of documents like technical descriptions about printers, prices, terms of service... Maybe you want to make it easy to search these documents on your e-shopping website? See these 3 short documents for example:

HP® LaserJets have unmatched printing speed, performance and reliability that you can trust. Enjoy Low Prices and Free Shipping when you buy now online.
Every HP LaserJet comes with a one-year HP commercial warranty (or HP Limited Warranty).
HP LaserJet ; Lowest cost per page on mono laser printing. · $319.99 ; Wireless options available. · $109.00 ; Essential management features. · $209.00.

Now, imagine that one of your customers asks the following question on your e-shopping website:

How long is the warranty on the HP Color LaserJet Pro?

The semantic search AI model will return the following in the blink of an eye:

Every HP LaserJet comes with a one-year HP commercial warranty (or HP Limited Warranty).

Maybe your customer did not ask a properly formed question? No problem, a query like this would work too:

period warranty HP Color LaserJet Pro

So as you can see, semantic search is much more advanced than the traditional keyword search, as you can ask questions in natural language like you would do with a human. Additionally, semantic search AI is very good at performing disambiguation (understanding the meaning of a word thanks to its context).

Semantic search is a very good solution when it comes to searching and performing question answering on your own data, because it is both blazing fast and accurate.

If you want to answer questions about a large corpus of internal domain knowledge, you might want to set up a Retrieval Augmented Generation (RAG) system. In that case please read our dedicated article about RAG: read it here.

Semantic search can be achieved by populating a vector database with embeddings, which is the approach that vector database providers like Pinecone or Milvus use. But for the most advanced response times, you will want to create your own semantic search model and deploy it on a GPU, which is what we do at NLP Cloud.

Noun Chunks

Why Use Semantic Search?

Semantic search has made dramatic progress these last few years, both in terms of speed and accuracy. Here are some use case examples:

Website Search

It is now very common to see search bars on online websites, like e-shopping websites, technical documentation, etc. Thanks to semantic search, you can greatly improve this search feature in order to make it more relevant and accurate.

Customer Support

Support chatbots are more and more advanced. You can now ask a support AI advanced questions about your contract, product features, refund policies, etc.

Internal Knowledge Base

Employees sometimes have a hard time retrieving the right information, which makes their day-to-day job harder and slows down their productivity. A good solution is to propose an internal knowledge base that is accessible with semantic search.

Search Legal And Financial Documents

Parsing complex legal and financial documents can be a challenge. A solution here is to add these documents to the AI engine, and easily apply semantic search in order to retrieve results.

NLP Cloud's Semantic Search API

NLP Cloud proposes a semantic search API that allows you to create your own semantic search engine out of your own business data, and then perform semantic search out of the box, based on the best Sentence Transformers models.
The response time (latency) is very good for these models!

For more details, see our documentation about semantic search here.

Testing semantic search locally is one thing, but using it reliably in production is another thing. With NLP Cloud you can just do both!

Frequently Asked Questions

What is semantic search?

Semantic search is a data searching technique that aims to improve search accuracy by understanding the searcher's intent and the contextual meaning of the search query. It goes beyond keyword matching to consider various factors such as user location, search history, and synonyms of words, to provide more relevant results.

Is a semantic search API a good alternative to vector databases like Pinecone or Milvus?

Yes, creating your own semantich search model will give you state-of-the-art performances, especially when deployed on a GPU like we do on NLP Cloud

How does semantic search differ from traditional keyword-based search?

Semantic search understands the context and intent behind a query, leveraging natural language processing to improve search accuracy. In contrast, traditional keyword-based search relies on matching exact phrases or keywords in the query to content, without considering the broader context or synonyms.

How do search engines like Google use semantic search?

Search engines like Google use semantic search to understand the intent and contextual meaning of a query by analyzing the relationship between words in the search phrase. This allows them to return more relevant and personalized search results to the user.

How does semantic search impact SEO?

Semantic search improves SEO by enabling search engines to understand the context and intent behind users' queries, thus allowing webpages to rank better if they closely match the intended meaning. This emphasizes the importance of creating content that not only includes keywords but is also rich in relevant topics and concepts that fulfill users' informational needs.

How to evaluate the accuracy of semantic search?

To evaluate the accuracy of semantic search, precision and recall metrics are commonly used, comparing the relevance of retrieved documents or answers to a set of queries against a manually curated ground truth. Additionally, user satisfaction and relevance feedback in practical implementations can provide insights into the effectiveness and accuracy of the semantic search algorithms.

What languages does your AI API support for semantic search?

We support semantic search in 50 languages

Can I try your semantic search API for free?

Yes, like all the models on NLP Cloud, the semantic search API endpoint can be tested for free

How does your AI API handle data privacy and security during the semantic search process?

NLP Cloud is focused on data privacy by design: we do not log or store the content of the requests you make on our API. NLP Cloud is both HIPAA and GDPR compliant.