Semantic Similarity
Semantic Similarity
What is Semantic Similarity?
Semantic similarity is about detecting whether 2 pieces of text have the same meaning or not.
For example, you might want to know whether the 2 following blocks of text are talking about the same thing:
Batch inference is very powerful because it will take almost the same time for your model to address several requests as it takes to address 1 request. Under the hood some operations will be factorized, so that instead of doing everything n times, the model only has to do it once.
Batch inference is a good way for your model to address more requests faster. Some operations are actually factorized in order to do things only once.
They clearly DO talk about the same thing and pretty much have the same meaning.
Sending these 2 blocks of text to a semantic similarity model would return a score like 0.90, meaning that, according to the model, the 2 inputs have the same meaning. On the other hand, a low score would indicate that the inputs don't have the same meaning.
Why Use Semantic Similarity?
The quality of semantic similarity has recently dramatically improved and has led to many interesting applications. Here are some examples:
Plagiarism Checking
Thanks to semantic similarity, you can automatically detect whether a piece of text is a paraphrase of another piece of text.
Semantic Search
Modern search engines must be able to detect the intent behind a search request and then match that intent against a high volume of text samples. This is a great application for semantic similarity.
Opinions Analysis
Thanks to semantic similarity, it is possible analyze a huge volume of Tweets, conversations, comments... and then detect some trends out of them.
Python Code
import nlpcloud
client = nlpcloud.Client("paraphrase-multilingual-mpnet-base-v2", "", gpu=False, lang="en")
client.semantic_similarity(["Adding natural language processing to my project is a game changer, it really enhances the product.","Integrating natural language processing to my project does make a difference, it really makes the product better."])Language
NLP has a critical weakness: it doesn't work well with non-English languages.
We do our best to add non-English models when it's possible. See for example XLM Roberta Large XNLI, TF Allociné, German Sentiment Bert... Unfortunately few models are available so it's not possible to cover all the NLP use cases with that method.
In order to solve this challenge, we developed a multilingual AI that automatically translates your input into English, performs the actual NLP operation, and then translates the result back to your original language. It makes your requests a bit slower but returns impressive results.
Simply select your language in the list, and from now on you can write the input text in your own language!
This multilingual add-on is a paid feature. Please contact the support team so they can upgrade your plan.