Semantic similarity is about detecting whether 2 pieces of text have the same meaning or not.
For example, you might want to know whether the 2 following blocks of text are talking about the same thing:
Batch inference is very powerful because it will take almost the same time for your model to address several requests as it takes to address 1 request. Under the hood some operations will be factorized, so that instead of doing everything n times, the model only has to do it once.
Batch inference is a good way for your model to address more requests faster. Some operations are actually factorized in order to do things only once.
They clearly DO talk about the same thing and pretty much have the same meaning.
Sending these 2 blocks of text to a semantic similarity model would return a score like 0.90, meaning that, according to the model, the 2 inputs have the same meaning. On the other hand, a low score would indicate that the inputs don't have the same meaning.
The quality of semantic similarity has recently dramatically improved and has led to many interesting applications. Here are some examples:
Thanks to semantic similarity, you can automatically detect whether a piece of text is a paraphrase of another piece of text.
Modern search engines must be able to detect the intent behind a search request and then match that intent against a high volume of text samples. This is a great application for semantic similarity.
Thanks to semantic similarity, it is possible analyze a huge volume of Tweets, conversations, comments... and then detect some trends out of them.
Sentence Transformers is an amazing library that has been recently released in order to dramatically improve embeddings calculation between many pieces of text. This is useful for semantic similarity, semantic search, or paraphrase mining. It is much faster than standard Transformer-based models like Bert, Distilbert or Roberta because it uses the so called cosine-similarity technique.
Paraphrase Multilingual Mpnet Base v2 is a very good Natural Language Processing model based on Sentence Transformers, that is able to perform embeddings calculation and then detect semantic similarity in more than 50 languages with excellent performances.You can find it here.
Building an inference API for semantic similarity is interesting as soon a you want to use semantic similarity in production. But keep in mind that building such an API is not necessarily easy. First because you need to code the API (easy part) but also because you need to build a highly available, fast, and scalable infrastructure to serve your models behind the hood (hardest part). Machine learning models consume a lot of resources (memory, disk space, CPU, GPU...) which makes it hard to achieve high-availability and low latency at the same time.
Leveraging such an API is very interesting because it is completely decoupled from the rest of your stack (microservice architecture), so you can easily scale it independently and ensure high-availability of your models through redundancy. But an API is also the way to go in terms of language interoperability. Most machine learning frameworks are developed in Python, but it's likely that you want to access them from other languages like Javascript, Go, Ruby... In such situation, an API is a great solution.
NLP Cloud proposes a semantic similarity API that gives you the opportunity to perform semantic similarity
out of the box, based on Paraphrase Multilingual Mpnet Base v2
The response time (latency) is very good for this model. You can either use the pre-trained
model, train your own model, or upload your own custom model!
For more details, see our documentation about semantic similarity here.
Testing semantic similarity locally is one thing, but using it reliably in production is another thing. With NLP Cloud you can just do both!