Creating Your Own Semantic Search Model
Creating your own semantic search model is a great way to get accurate results while
ensuring a very low latency. This is even more true if you deploy your own semantic search model
on a GPU.
First, let's make a small dataset containing our data. Create a 1-column CSV file (called
"dataset.csv")
containing the following
technical documentation about HP printers (in a real life scenario you will want to include many more
examples of course):
"HP® LaserJets have unmatched printing speed, performance and reliability that you can trust. Enjoy Low Prices and Free Shipping when you buy now online."
"Every HP LaserJet comes with a one-year HP commercial warranty (or HP Limited Warranty)."
"HP LaserJet ; Lowest cost per page on mono laser printing. · $319.99 ; Wireless options available. · $109.00 ; Essential management features. · $209.00."
Each row can contain up to 512 tokens (roughly equivalent to 400 words), and in order to maximise
accuracy it is recommended to stay below 128 tokens (roughly equivalent to 100 words). Now that we have
our 3 pieces of documentation in our dataset, we can encode the data using our model with
Sentence Transformers. Create a Python script with the following (make sure that PyTorch and Sentence
Transformers are installed).
from sentence_transformers import SentenceTransformer
import csv
import torch
model_name = 'paraphrase-multilingual-mpnet-base-v2'
encoded_model_path = 'semantic_search_model.pt'
dataset_path = 'dataset.csv'
bi_encoder = SentenceTransformer(model_name)
passages = []
with open(dataset_path) as csv_file:
csv_reader = csv.reader(csv_file)
for row in csv_reader:
passages.append(row[0])
corpus_embeddings = bi_encoder.encode(
passages, batch_size=32, convert_to_tensor=True, show_progress_bar=True)
torch.save(corpus_embeddings, encoded_model_path)
This script downloads and uses paraphrase-multilingual-mpnet-base-v2 as a base model and uses it to
encode our data. You can choose among many available pre-trained models, depending
on your requirements (model size, use case, supported languages, ...). Depending on your hardware you
will want to adapt the "batch_size" parameter in order to speed up the encoding process.
Once created you can use your model for inference with the following Python script:
import csv
from sentence_transformers import SentenceTransformer, util
import torch
bi_encoder = SentenceTransformer('paraphrase-multilingual-mpnet-base-v2')
semantic_search_model = torch.load('semantic_search_model.pt')
passages = []
with open('app/custom_model/dataset.csv') as csv_file:
csv_reader = csv.reader(csv_file)
for row in csv_reader:
passages.append(row[0])
question_embedding = bi_encoder.encode(
"How long is the warranty on the HP Color LaserJet Pro?", convert_to_tensor=True)
hits = util.semantic_search(
question_embedding, semantic_search_model, top_k=3)
hits = hits[0]
result = {"search_results": [
{"score": hit['score'], "text": passages[hit['corpus_id']]} for hit in hits]}
The above inference script returns the following results:
{
"search_results": [
{
"score": 0.99,
"text": "Every HP LaserJet comes with a one-year HP commercial warranty (or HP Limited Warranty)."
},
{
"score": 0.74,
"text": "All consumer PCs and printers come with a standard one-year warranty. Care packs provide an enhanced level of support and/or an extended period of coverage for your HP hardware. All commercial PCs and printers come with either a one-year or three-year warranty."
},
{
"score": 0.68,
"text": "In-warranty plan · Available in 2-, 3-, or 4-year extension plans · Includes remote problem diagnosis support and Next Business Day Exchange Service."
},
]
}
In our inference script, the "top_k" parameter determines how many results we want to return. In the
result, we show the matching text from the dataset with a confidence score. This score is important
because it helps us decide whether we want to accept the response or not.
