How to fine-tune a GPT-3 model using Python with your own data for improved performance

I'm constantly looking for ways to automate the work with support requests. An idea has been to fine-tune a GPT-3 model to answer common support-related questions.

Here's how you can use Python to fine-tune a GPT-3 model with your own data for improved performance.

---

Prefer video over text?

Check out my free video course on fine-tuning a GPT-3 model

If you prefer learning through videos, I've got you covered!

I've created a free video course that teaches you how to fine-tune a GPT-3 model with your own data. The course covers everything in this blog post and more, with step-by-step instructions and examples.

You can access the course for free: https://norahsakal.gumroad.com/l/qwhwa

Happy learning!

---

Here's what we'll use:

1. OpenAI API 🤖
2. Python 🐍

---

Here are the steps:

1. Get OpenAI API key
2. Create training data
3. Check the training data
4. Upload training data
5. Fine-tune model
6. Test the new model on a new prompt

---

Disclaimer

This guide walks you through fine-tuning a GPT-3 model in Python, shown in a Jupyter notebook.
If you're looking for the steps of fine-tuning right in a terminal, OpenAI has a great guide for fine-tuning in your terminal.

---

1. Get OpenAI API key

Before we go ahead and fine-tune a model, let's get the OpenAI credentials needed for the API calls.

Go to https://beta.openai.com/, log in and click on your avatar and View API keys:

Then create a new secret key and save it for the request:

Now we have all the credentials needed to make an API request.

---

2. Create training data

The next step is to create training data to teach GPT-3 what you'd like to say. The data need to be a JSONL document with a new prompt and the ideal generated text:

{"prompt": "<question>", "completion": "<ideal answer>"}
{"prompt": "<question>", "completion": "<ideal answer>"}
{"prompt": "<question>", "completion": "<ideal answer>"}

Let's start by importing the libraries needed:

import json
import openai

Then add your API key from the previous step:

api_key ="YOUR_OPENAI_API_KEY"
openai.api_key = api_key

Now create a regular dict with the training data. For this guide, I'll add some support questions I've recently received:

training_data = [{
    "prompt": "Where is the billing ->",
    "completion": " You find the billing in the left-hand side menu.\n"
},{
    "prompt":"How do I upgrade my account ->",
    "completion": " Visit you user settings in the left-hand side menu, then click 'upgrade account' button at the top.\n"
}]

Make sure to end each prompt with a suffix. According to the OpenAI API reference, you can use ->.

Also, make sure to end each completion with a suffix as well; I'm using .\n.

The next step is to convert the dict to a proper JSONL file. JSONL file is a newline-delimited JSON file, so we'll add a \n at the end of each object:

file_name = "training_data.jsonl"

with open(file_name, "w") as output_file:
 for entry in training_data:
  json.dump(entry, output_file)
  output_file.write("\n")

Now you have the training data as a JSONL file, let's check the training data before starting the fine-tuning.

---

3. Check the training data

We can check the training data using a CLI data preparation tool provided by OpenAI. It gives you suggestions about how you can reformat the training data.

Let's try it out with our training data. Run this line in Jupyter notebook:

!openai tools fine_tunes.prepare_data -f training_data.jsonl

You'll get suggestions similar to this:

I'm only using a couple of prompts for this guide, but the suggestions mention that the recommendations are to have at least a few hundred examples.

You'll also see the approximate time depending on how many jobs are ahead of you in the queue:

We're ready to upload the training data in the next section.

---

4. Upload training data

Now that you checked the improvement suggestions, let's upload the training data to OpenAI:

upload_response = openai.File.create(
  file=open(file_name, "rb"),
  purpose='fine-tune'
)
file_id = upload_response.id
upload_response

If you check the response, you'll see the file id which we'll need in the next step when we're training the model:

Use this file id in the next step, where we'll fine-tune a model.

---

5. Fine-tune model

Alrighty, we have the prepared training data, uploaded it, and now we're finally ready to fine-tune the model.

Start the fine-tuning by running this command:

fine_tune_response = openai.FineTune.create(training_file=file_id)
fine_tune_response

The default model is Curie. But if you'd like to use DaVinci instead, then add it as a base model to fine-tune like this:

openai.FineTune.create(training_file=file_id, model="davinci")

The first response will look something like this:

---

6. Check fine-tuning progress

You can use two openai functions to check the progress of your fine-tuning.

Option 1: List events

You can use openai.FineTune.list_events() and pass in the fine_tune_response id to list all the current events:

fine_tune_events = openai.FineTune.list_events(id=fine_tune_response.id)

This will give you a list of all the events related to the fine-tuning job you're running:

Option 2: Retrieve fine-tuning job

You can also use openai.FineTune.retrieve(id=fine_tune_response.id) to retrieve a specific fine-tune:

retrieve_response = openai.FineTune.retrieve(id=fine_tune_response.id)

This will give you a JSON object with information about a specific fine-tune job:

When your fine-tuning job is first in line, your fine-tuning event starts:

---

7. Save fine-tuned model

Troubleshooting fine_tuned_model as null
During the fine-tuning process, the fine_tuned_model key may not be immediately available in the fine_tune_response object returned by openai.FineTune.create().

To check the status of your fine-tuning process, you can call the openai.FineTune.retrieve() function and pass in the fine_tune_response.id. This function will return a JSON object with information about the training status, such as the current epoch, the current batch, the training loss, and the validation loss.

After the fine-tuning process is complete, you can check the status of all your fine-tuned models by calling openai.FineTune.list(). This will list all of your fine-tunes and their current status.

Once the fine-tuning process is complete, you can retrieve the fine_tuned_model key by calling the openai.FineTune.retrieve() function again and passing in the fine_tune_response.id. This will return a JSON object with the key fine_tuned_model and the ID of the fine-tuned model that you can use for further completions.

Once the fine-tuning is finished, go ahead and save the name of the fine-tuned model:

# Option 1 | if response.fine_tuned_model != null
fine_tuned_model = response.fine_tuned_model
fine_tuned_model

# Option 2 | if response.fine_tuned_model == null
retrieve_response = openai.FineTune.retrieve(response.id)
fine_tuned_model = retrieve_response.fine_tuned_model
fine_tuned_model

It will look something like this:

We now have a fine-tuned model. Let's try it out on a new prompt in the next section.

---

8. Test the new model on a new prompt

We're finally at the last step, where we'll try our fine-tuned model on a new prompt.

I only ran my fine-tuning on 2 prompts, so I'm not expecting a super-accurate completion.

Start by creating a new prompt.

Remember to end the prompt with the same suffix as we used in the training data; ->:

new_prompt = "How do I find my billing? ->"

Next, run the completion with your fine-tuned model:

answer = openai.Completion.create(
  model=fine_tuned_model,
  prompt=new_prompt,
  max_tokens=100,
  temperature=0
)
answer['choices'][0]['text']

Here's what I got:

Great! Now you have a fine-tuned GPT-3 model ready for future prompts.

---

Here's a summary of what we did

1. Got our OpenAI API key

2. Created training data

3. Checked the training data

4. Uploaded training data

5. Fine-tuned a GPT-3 model

6. Tested the new model on a new prompt

---

7. Improvements

Expand the training data
The training data analysis by OpenAI in step 2 of this guide suggests expanding the training data amount. I only used 2 prompts in this guide. The suggestion says at least a few hundred examples/prompts.

FAQ might not be the best use case
FAQ-related questions might not be the best use case for fine-tuning. If you'd like to automate your support questions, a question-answering approach might be better suited than fine-tuning a GPT-3 model.

---

Next steps

1. Repo with source code
Here is the repo with a Jupyter notebook with all the source code if you'd like to implement this on your own ⬇️
https://github.com/norahsakal/fine-tune-gpt3-model

2. Do you need help with fine-tuning your own GPT-3 model? Or do you have other questions?
I'm happy to help, don't hesitate to reach out ➡️ [email protected]

Or shoot me a DM on Twitter @norahsakal

---

Originally published at https://norahsakal.com/blog/fine-tune-gpt3-model