Named Entity Recognition (NER) API With spaCy Alternatives

What is NER?

NER is the most common technique used in production by projects that want to leverage Natural Language Processing today.
It solves a very interesting challenge: extracting structured entities from an unstructured block of text.

Let's say you have the following sentence:

John Doe is a web developer at Google.

You would like to automatically detect that "John Doe" is a name, "web developer" is a job title, and "Google" is a company. And this is exactly what NER is going to do.

You have 2 ways to perform entity extraction: let the AI process your sentence and extract all the possible entities it can find, or ask it to look for a specific entity.

Why Use NER?

The world is full of unstructured data, especially the web. Being able to extract structured information from it can give access to a lot of valuable information. Here are a couple of examples.

Sort Customer Requests

When dealing with lots of customer requests (support, sales, ...) it definitely helps to apply NER in order to automatically sort these incoming requests. For example you could automatically extract the type of product mentioned in the request and route this to the right service accordingly.

Extract Financial Data

Extracting and consolidating financial data can be long and tedious. NER can definitely boost your productivity here by helping you extract the right data in a second.



Pre-process Resumes/Applications

HR services are sometimes having a hard time reading all these applications. It can be interesting for them to automatically highlight interesting entities like company names, skills,... in order to save time.

Extract Leads

Many B2B leads can be found on public websites or company brochures, but extracting them manually can sometimes be a pain. Thanks to NER you can automatically extract a person, with her jobtitle, and company, if they exist.

NER With The SpaCy Alternatives: Generative Models

SpaCy is an excellent Natural Language Processing framework that can be usefully leveraged for Named Entity Recognition (see their website here). NER with spaCy is both fast and accurate. The spaCy pre-trained models can natively recognize entities like name, company, country... The Ginza model based on spaCy, released by Megagon Labs, is performing extremely well in Japanese (see the project here). Entities like job titles cannot be extracted natively though. If you want to extract non-native entities like job titles, you will need to train your own model using annotation. This is when generative models become interesting.

GPT-4, ChatGPT, GPT-J, GPT-NeoX 20B, LLaMA 2, Dolphin, and ChatDolphin are huge AI models that can basically detect any sort of entities, even if they haven't been trained for it. For example, even if a generative AI model hasn't specifically been trained to detect job titles, you can still ask it to do so and get amazing results.

See more in this short video where we quickly compare spaCy and GPT-NeoX 20B for advanced NER: click here

Annotation illustration

SpaCy offers small, medium, and large pre-trained models that can handle NER out of the box without needing to train a model by yourself. These models are available in the following languages: Chinese, Danish, Dutch, English, French, German, Greek, Italian, Japanese, Lithuanian, Norwegian Bokmål, Polish, Portuguese, Romanian, Russian, Spanish

NER Inference API

Building an inference API for NER is a necessary step as soon a you want to use NER in production. But building such an API is hard... First because you need to code the API (easy part) but also because you need to build a highly available, fast, and scalable infrastructure to serve your models behind the hood (hardest part). It is especially hard for machine learning models as they consume a lot of resources (memory, disk space, CPU, GPU...).

Such an API is interesting because it is completely decoupled from the rest of your stack (microservice architecture), so you can easily scale it independently, and you can access it using any programming language. Most machine learning frameworks are developed in Python, but it's likely that you want to access them from other languages like Javascript, Go, Ruby...

NLP Cloud's NER API

NLP Cloud proposes an NER API that gives you the opportunity to perform Named Entity Recognition out of the box, based on spaCy, Ginza, or generative models, with excellent performances. NER is not very resource intensive, so the response time (latency), when performing NER from the NLP Cloud API, is very good when using spaCy. It is also quite reasonable when using generative models. You can either use all the large spaCy pre-trained models in all languages, or GPT-J, GPT-NeoX 20B, LLaMA 2, Dolphin, and ChatDolphin, or fine-tune your own models!

For more details, see our documentation about NER here. For advanced usage, see the text generation API endpoint here. And easily test entity extraction on our playground.

Testing NER locally is one thing, but using it reliably in production is another thing. With NLP Cloud you can just do both!

Try NER for free
for free