Text Generation

This example uses text generation. Text generation can be very powerful, but using it correctly takes a bit of practice.

For most use cases based on text generation, it is crucial to understand the few-shot learning concept. Basically, in order for GPT to understand what you want, you should give it a couple of examples in your input request. In the playground, we are illustrating with some random examples, but for better results you should create your own few-shot examples that better suit your own use case.

If you're still not getting good results even when using few-shot learning, it might be a sign that you need to fine-tune your own GPT model. You can easily do it on NLP Cloud.

Feel free to play with the text parameters like top p, temperature, repetition penalty... and see the documentation about how to use these parameters if needed.

If you need advice, please contact us!

Control whether you want to use the model on a GPU. Machine learning models run much faster on GPUs.

Whether min_length and max_length should not include the length of the input text. If false, min_length and max_length include the length of the input text. If true, min_length and max_length don't include the length of the input text. Defaults to false.

Whether you want to remove the input text form the result. Defaults to false.

Whether or not to use sampling; use greedy decoding otherwise. Defaults to true. This is an advanced parameter, so you should change it only if you know what you're doing. Not supported yet by Fast GPT-J.

Whether to stop the beam search when at least num_beams sentences are finished per batch or not. Defaults to false. This is an advanced parameter, so you should change it only if you know what you're doing. Not supported yet by Fast GPT-J.

The minimum number of tokens that the generated text should contain. The size of the generated text should not exceed 256 tokens on a CPU plan and 1024 tokens on GPU plan (please contact us if you need more generated tokens). The input size + the size of the generated text should not exceed 2048 tokens. If length_no_input is false, the size of the generated text is the difference between min_length and the length of your input text. If length_no_input is true, the size of the generated text simply is min_length. Defaults to 10. Not supported yet by Fast GPT-J.

The maximum number of tokens that the generated text should contain. The size of the generated text should not exceed 256 tokens on a CPU plan and 1024 tokens on GPU plan (please contact us if you need more generated tokens). The input size + the size of the generated text should not exceed 2048 tokens. If length_no_input is false, the size of the generated text is the difference between max_length and the length of your input text. If length_no_input is true, the size of the generated text simply is max_length. Defaults to 50.

A specific token that should be the end of the generated sequence. For example if could be `.` or `###` or anything else below 10 characters (`\n` is not supported for the moment).

Whether you want to remove the end sequence form the result. Defaults to false.

List of tokens that are not allowed to be generated. Defaults to null. Not supported yet by Fast GPT-J.

If set to int > 0, all ngrams of that size can only occur once. Defaults to 0. This is an advanced parameter, so you should change it only if you know what you're doing. Not supported yet by Fast GPT-J.

The number of independently computed returned sequences. Defaults to 1. Not supported yet by Fast GPT-J.

Prevents the same word to be repeated too many times. 1.0 means no penalty. Defaults to 1.0. Not supported yet by Fast GPT-J.

Exponential penalty to the length. 1.0 means no penalty. Set to values < 1.0 in order to encourage the model to generate shorter sequences, or to a value > 1.0 in order to encourage the model to produce longer sequences. Defaults to 1.0. Not supported yet by Fast GPT-J.

Number of beams for beam search. 1 means no beam search. If num_beams > 1, the size of the input text should not exceed 40 tokens on GPU (please contact us if you need a bigger input length with num_beams > 1). Defaults to 1. This is an advanced parameter, so you should change it only if you know what you're doing. Not supported yet by Fast GPT-J.

The number of highest probability vocabulary tokens to keep for top-k-filtering. Maximum 1000 tokens. The lower this value, the less likely GPT-J is going to generate off-topic text. Defaults to 0 on base GPT-J and defaults to 50 on fast GPT-J.

If set to float < 1, only the most probable tokens with probabilities that add up to top_p or higher are kept for generation. The higher this value, the less deterministic the result will be. It's recommended to play with top_p if you want to produce original content for applications that require accurate results, while you should use temperature if you want to generate more funny results. You should not use both at the same time. Should be between 0 and 1. Defaults to 0.7.

The value used to module the next token probabilities. The higher this value, the less deterministic the result will be. For example if temperature=0 the output will always be the same, while if temperature=1 each new request will produce very different results. It's recommended to play with top_p if you want to produce original content for applications that require accurate results, while you should use temperature if you want to generate more funny results. You should not use both at the same time. Should be between 0 and 1. Defaults to 1.

NLP has a critical weakness: it doesn't work well with non-English languages.

We do our best to add non-English models when it's possible. See for example Fine-tuned GPT-NeoX 20B, XLM Roberta Large XNLI, Paraphrase Multilingual Mpnet Base V2, or spaCy. Unfortunately few models are available so it's not possible to cover all the NLP use cases with that method.

In order to solve this challenge, we developed a multilingual AI that automatically translates your input into English, performs the actual NLP operation, and then translates the result back to your original language. It makes your requests a bit slower but returns impressive results.

Simply select your language in the list, and from now on you can write the input text in your own language!

This multilingual add-on is a paid feature. It is included by default in the pay-as-you-go plan. If you are on another plan, please contact the support team so they can upgrade your plan.

What is Text Generation?

Text generation is about starting a text, and then letting the NLP model create the rest of the text for you, in the spirit of your initial input. It is up to you to decide how large you want the generated text to be.

Let's say you have the following piece of text:

Now, let's say you want to generate about 250 words from the above text. Just give your text to the model and it will generate the rest:

Pretty smart isn't it? As you can see, the text generated by the model says interesting things about GPT Neo. Most text generation models are not deterministic, meaning that, every time you send the same text to the model, you will get different results.

You can achieve almost any NLP use case thanks to text generation, as long as you are leveraging a huge and versatile model: sentiment analysis, grammar and spelling correction, question answering, code generation, machine translation, semantic similarity, intent classification, paraphrasing... and more!