21. Fine-Tuning

Alpha Version: Work in progress.

Fine-tuning is the process of adapting a pre-trained GPT-1 model to a specific downstream task. After the language model has been pre-trained, GPT-1 is further trained using task-specific input examples together with their corresponding labels.

The original GPT-1 paper evaluated the model on the following four downstream tasks:

• Textual Entailment (Natural Language Inference, NLI)
• Question Answering (QA)
• Semantic Similarity
• Commonsense Reasoning

In this document, we implement the Natural Language Inference (NLI) and Question Answering (QA) tasks.

As an example, consider the Natural Language Inference task.

Given a Premise and a Hypothesis, the objective is to determine whether their relationship is entailment, neutral, or contradiction.

For example, given the premise

• Premise: “A man is playing guitar on stage.”

the model should infer that the hypothesis

• Hypothesis: “A man is playing a musical instrument.”

is an ’entailment’, as illustrated in Figure 21.1 [1].

To learn this task, the pre-trained GPT-1 model is fine-tuned using a large collection of Premise–Hypothesis pairs together with their corresponding labels, as illustrated in Figure 21.1 [2].

Examples of the training data are shown below.

• Example 1
  • Premise: “Two dogs are running in the park.”
  • Hypothesis: “There are animals outside.”
  • Label: entailment
• Example 2
  • Premise: “A man is playing guitar on stage.”
  • Hypothesis: “The guy playing guitar is playing badly.”
  • Label: neutral
• Example 3
  • Premise: “A woman is cooking dinner in the kitchen.”
  • Hypothesis: “No one is in the kitchen.”
  • Label: contradiction

Section 21.1 presents the mathematical formulation of GPT-1 fine-tuning. Section 21.2 and Section 21.3 then describe the implementation of the Natural Language Inference and Question Answering tasks, respectively.