How LLMs work

Unblock all features

Sign in to track progress, get AI help and save conversations.

Implementation

Helpers

Let's start with helper functions that will be reused in multiple places.

Mathematical functions

First we need to introduce three mathematical functions: sigmoid, softmax, and cross_entropy. The implementation follows the formulas we defined earlier.

  • sigmoid takes an array of numbers and squashes each one into a value between 0 and 1.
  • softmax takes an array of scores and returns a probability distribution for each row.
  • cross_entropy takes the predicted probabilities and the correct indices, and returns an average loss.
python

When the model scales up, some numbers can get too large or too small, so the formulas might need to be adjusted to handle these cases. This is not an issue for our small example, so we keep the raw formulas for simplicity.

Vocabulary

The vocabulary creates a list of the most common tokens in the given text and stores them with assigned ids.

  • tokenize splits the text into tokens (words and punctuation marks) using a regular expression.
  • encode turns the text into a sequence of token ids using the stoi (string-to-integer) lookup.
  • decode turns the sequence of token ids back into text using the itos (integer-to-string) lookup.
python

Training examples

For training we need to split the whole corpus into input/target pairs of size block that we will pass to our model.

  • Each row of X is a sequence of tokens in one example.
  • Each row of Y is a sequence of next tokens for each position in X row.
python