Semantic Similarity
Semantic Similarity
What is Semantic Similarity?
Semantic similarity is about detecting whether 2 pieces of text have the same meaning or not.
For example, you might want to know whether the 2 following blocks of text are talking about the same thing:
Batch inference is very powerful because it will take almost the same time for your model to address several requests as it takes to address 1 request. Under the hood some operations will be factorized, so that instead of doing everything n times, the model only has to do it once.
Batch inference is a good way for your model to address more requests faster. Some operations are actually factorized in order to do things only once.
They clearly DO talk about the same thing and pretty much have the same meaning.
Sending these 2 blocks of text to a semantic similarity model would return a score like 0.90, meaning that, according to the model, the 2 inputs have the same meaning. On the other hand, a low score would indicate that the inputs don't have the same meaning.
Why Use Semantic Similarity?
The quality of semantic similarity has recently dramatically improved and has led to many interesting applications. Here are some examples:
Plagiarism Checking
Thanks to semantic similarity, you can automatically detect whether a piece of text is a paraphrase of another piece of text.
Semantic Search
Modern search engines must be able to detect the intent behind a search request and then match that intent against a high volume of text samples. This is a great application for semantic similarity.
Opinions Analysis
Thanks to semantic similarity, it is possible analyze a huge volume of Tweets, conversations, comments... and then detect some trends out of them.
Language
AI models don't always work well with non-English languages.
We do our best to add non-English models when it's possible. See for example Fine-tuned GPT-NeoX 20B, Dolphin, ChatDolphin, XLM Roberta Large XNLI, Paraphrase Multilingual Mpnet Base V2, or spaCy. Unfortunately not all the models are good at handling non-English languages.
In order to solve this challenge, we developed a multilingual module that automatically translates your input into English, performs the actual NLP operation, and then translates the result back to your original language. It makes your requests a bit slower but often returns very good results.
Even for models that natively understand non-English languages, they actually sometimes work even better with the multilingual addon.
Simply select your language in the list, and from now on you can write the input text in your own language!
This multilingual add-on is a free feature.