Text Generation
Using Text Generation
This example uses text generation. Text generation can be very powerful, but using it correctly takes a bit of practice.
For most use cases based on text generation, it is crucial to understand the few-shot learning concept. Basically, in order for GPT to understand what you want, you should give it a couple of examples in your input request. In the playground, we are illustrating with some random examples, but for better results you should create your own few-shot examples that better suit your own use case.
If you're still not getting good results even when using few-shot learning, it might be a sign that you need to fine-tune your own GPT model. You can easily do it on NLP Cloud.
Feel free to play with the text parameters like top p, temperature, repetition penalty... and see the documentation about how to use these parameters if needed.
If you need advice, please contact us!
Use GPU
Control whether you want to use the model on a GPU. Machine learning models run much faster on GPUs.
Length No Input
Whether min_length and max_length should not include the length of the input text. If false, min_length and max_length include the length of the input text. If true, min_length and max_length don't include the length of the input text. Defaults to false.
Remove Input
Whether you want to remove the input text form the result. Defaults to false.
Do Sample
Whether or not to use sampling; use greedy decoding otherwise. Defaults to true. This is an advanced parameter, so you should change it only if you know what you're doing. Not supported yet by Fast GPT-J.
Early Stopping
Whether to stop the beam search when at least num_beams sentences are finished per batch or not. Defaults to false. This is an advanced parameter, so you should change it only if you know what you're doing. Not supported yet by Fast GPT-J.
Min Length
The minimum number of tokens that the generated text should contain. The size of the generated text should not exceed 256 tokens on a CPU plan and 1024 tokens on GPU plan (please contact us if you need more generated tokens). The input size + the size of the generated text should not exceed 2048 tokens. If length_no_input is false, the size of the generated text is the difference between min_length and the length of your input text. If length_no_input is true, the size of the generated text simply is min_length. Defaults to 10. Not supported yet by Fast GPT-J.
Max Length
The maximum number of tokens that the generated text should contain. The size of the generated text should not exceed 256 tokens on a CPU plan and 1024 tokens on GPU plan (please contact us if you need more generated tokens). The input size + the size of the generated text should not exceed 2048 tokens. If length_no_input is false, the size of the generated text is the difference between max_length and the length of your input text. If length_no_input is true, the size of the generated text simply is max_length. Defaults to 50.
End Sequence
A specific token that should be the end of the generated sequence. For example if could be `.` or `###` or anything else below 10 characters (`\n` is not supported for the moment).
Remove End Sequence
Whether you want to remove the end sequence form the result. Defaults to false.
Bad Words
List of tokens that are not allowed to be generated. Defaults to null. Not supported yet by Fast GPT-J.
No Repeat Ngram Size
If set to int > 0, all ngrams of that size can only occur once. Defaults to 0. This is an advanced parameter, so you should change it only if you know what you're doing. Not supported yet by Fast GPT-J.
Num Returned Sequences
The number of independently computed returned sequences. Defaults to 1. Not supported yet by Fast GPT-J.
Repetition Penalty
Prevents the same word to be repeated too many times. 1.0 means no penalty. Defaults to 1.0. Not supported yet by Fast GPT-J.
Length Penalty
Exponential penalty to the length. 1.0 means no penalty. Set to values < 1.0 in order to encourage the model to generate shorter sequences, or to a value > 1.0 in order to encourage the model to produce longer sequences. Defaults to 1.0. Not supported yet by Fast GPT-J.
Num Beams
Number of beams for beam search. 1 means no beam search. If num_beams > 1, the size of the input text should not exceed 40 tokens on GPU (please contact us if you need a bigger input length with num_beams > 1). Defaults to 1. This is an advanced parameter, so you should change it only if you know what you're doing. Not supported yet by Fast GPT-J.
Top K
The number of highest probability vocabulary tokens to keep for top-k-filtering. Maximum 1000 tokens. The lower this value, the less likely GPT-J is going to generate off-topic text. Defaults to 0 on base GPT-J and defaults to 50 on fast GPT-J.
Top P
If set to float < 1, only the most probable tokens with probabilities that add up to top_p or higher are kept for generation. The higher this value, the less deterministic the result will be. It's recommended to play with top_p if you want to produce original content for applications that require accurate results, while you should use temperature if you want to generate more funny results. You should not use both at the same time. Should be between 0 and 1. Defaults to 0.7.
Temperature
The value used to module the next token probabilities. The higher this value, the less deterministic the result will be. For example if temperature=0 the output will always be the same, while if temperature=1 each new request will produce very different results. It's recommended to play with top_p if you want to produce original content for applications that require accurate results, while you should use temperature if you want to generate more funny results. You should not use both at the same time. Should be between 0 and 1. Defaults to 1.
Language
NLP has a critical weakness: it doesn't work well with non-English languages.
We do our best to add non-English models when it's possible. See for example XLM Roberta Large XNLI, TF Allociné, German Sentiment Bert... Unfortunately few models are available so it's not possible to cover all the NLP use cases with that method.
In order to solve this challenge, we developed a multilingual AI that automatically translates your input into English, performs the actual NLP operation, and then translates the result back to your original language. It makes your requests a bit slower but returns impressive results.
Simply select your language in the list, and from now on you can write the input text in your own language!
This multilingual add-on is a paid feature. Please contact the support team so they can upgrade your plan.
Text Generation
What is Text Generation?
Text generation is about starting a text, and then letting the NLP model create the rest of the text for you, in the spirit of your initial input. It is up to you to decide how large you want the generated text to be.
Let's say you have the following piece of text:
GPT Neo is a powerful NLP model
Now, let's say you want to generate about 250 words from the above text. Just give your text to the model and it will generate the rest:
GPT Neo is a powerful NLP model that does what every model should do: predict your future behavior. It's very effective at detecting patterns in your real-time input, predicting upcoming events and responses that you may not even consider. GPT's power consists of a set of internal neural processes that are extremely accurate. But at the same time that results are good, they need to be fast. So GPT Neo is designed to run on the server. To speed up your results, you may need to use the cloud. Here's why: you may be storing your business data in the cloud. And using the cloud may mean that you can't get your data into the NLP process because you have to transfer everything over the cloud.
Pretty smart isn't it? As you can see, the text generated by the model says interesting things about GPT Neo. Most text generation models are not deterministic, meaning that, every time you send the same text to the model, you will get different results.
You can achieve almost any NLP use case thanks to text generation, as long as you are leveraging a huge and versatile model: sentiment analysis, grammar and spelling correction, question answering, code generation, machine translation, semantic similarity, intent classification, paraphrasing... and more!
Why Use Text Generation?
Text generation is a great way to automatically create content. Here are a couple of examples.
Marketing Content Generation
Content creation is crucial for SEO today, but it's also a tedious job. Why not leave it to a dedicated NLP model, and then focus on something more important?
Chatbot
An interesting way of making chatbots sound more human is to add non-essential "chit-chats" to the core discussion. Text generation can help in this situation.
Fuzzy testing
Fuzzy testing is a technique used by programmers in order to test their applications with random content. Generating new content for every test is a convenient way to perform fuzzy testing.
Mock-up Creation
Before releasing a new application, it is often necessary to create mock-ups in order to get user feedbacks. Filling the blanks of these mock-ups with generated text is a good way to make it look as real as possible.
Python Code
import nlpcloud
client = nlpcloud.Client("fast-gpt-j", "", gpu=True, lang="en")
client.generation("""We are currently integrating Natural Language Processing because""",
min_length=10,
max_length=50,
length_no_input=False,
remove_input=False,
end_sequence=,
top_p=1,
temperature=0.8,
top_k=50,
repetition_penalty=1,
length_penalty=1,
do_sample=True,
early_stopping=False,
num_beams=1,
no_repeat_ngram_size=0,
num_return_sequences=1,
bad_words=None,
remove_end_sequence=False
)