Batch inference is very powerful because it will take almost the same time for your model to address several requests as it takes to address 1 request. Under the hood some operations will be factorized, so that instead of doing everything n times, the model only has to do it once.
什么是语义相似性？
语义相似性是指检测两个文本是否具有相同的含义。
例如，您可能想知道下面的两个文本块是否在谈论同一件事：
Batch inference is a good way for your model to address more requests faster. Some operations are actually factorized in order to do things only once.
显然，它们说的是同一件事，意思也基本相同。
将这两个文本块发送给语义相似性模型，会得到 0.90 这样的分数，也就是说，根据该模型，这两个输入具有相同的含义。反之，如果得分较低，则表示这两个输入不具有相同的含义。
