Batch inference is very powerful because it will take almost the same time for your model to address several requests as it takes to address 1 request. Under the hood some operations will be factorized, so that instead of doing everything n times, the model only has to do it once.
Kaj je semantična podobnost?
Semantična podobnost je ugotavljanje, ali imata dva dela besedila enak pomen ali ne.
Morda želite na primer vedeti, ali naslednja dva bloka besedila govorita o isti stvari:
Batch inference is a good way for your model to address more requests faster. Some operations are actually factorized in order to do things only once.
Očitno govorita o isti stvari in imata skoraj enak pomen.
Če bi ta dva bloka besedila poslali modelu semantične podobnosti, bi dobili rezultat 0,90, kar pomeni, da imata po mnenju modela oba vhoda enak pomen. Po drugi strani pa bi nizka ocena pomenila, da vnosa nimata enakega pomena.
