Chatbot and Conversational AI API, With Generative Models

What are Chatbots and Conversational AI and Why Use Generative AI?

Conversational AI is a central sub-field of Natural Language Processing that makes it possible for a human to have a conversation with a machine. Everytime the human says or asks something to the AI, the whole conversation history is sent too, so the AI can have the context in memory and make relevant responses. Modern chabots leverage conversational AI and can do more than simply having a conversation. For example they can detect customer intents, search documents, understand the customer tone and adapt their own tone (anger, joy, sarcasm...).

Until recently, chatbots have been very limited. But with the creation of large generative models like GPT-4, ChatGPT, GPT-J, GPT-NeoX, LLaMA 2, Dolphin, and ChatDolphin, it is now possible to easily create advanced chatbots that are both fluent and relevant.

GPT-J, GPT-NeoX, LLaMA 2, Dolphin, and ChatDolphin are the most advanced alternatives to GPT-4 and ChatGPT. These models are so big that they can adapt to many situations, and perfectly sounds like a human. For advanced use cases, it is possible to fine-tune GPT-J and Dolphin (train them with your own data), which is a great way to get a chatbot that is perfectly tailored to your company/product/industry.

Generative models have no "memory". So you should help them by re-sending the conversation history in every request you're making. We actually wrote a dedicated blog article about how to build a chatbot with a generative model, feel free to read it!

Building an Advanced Chatbot With Generative AI

Building an inference API for conversational AI with generative models is a necessary step as soon a you want to use a chatbot in production. But building such an API is hard... First because you need to code the API (easy part) but also because you need to build a highly available, fast, and scalable infrastructure to serve your models behind the hood (hardest part). It is especially hard for machine learning models as they consume a lot of resources (memory, disk space, CPU, GPU...).

Such an API is interesting because it is completely decoupled from the rest of your stack (microservice architecture), so you can easily scale it independently, and you can access it using any programming language. Most machine learning frameworks are developed in Python, but it's likely that you want to access them from other languages like Javascript, Go, Ruby...