Skip to content

Fundamentals · Part 0, post 1 of 4

Vectors and Matrices in Machine Learning

How models turn words and images into numbers

By Jeremy Greer

November 17, 2026

Share
AI summary
Markdown

Everything as a Number

Type a word into a chatbot, upload a photo, or hand a model (a program that learns from data) one row of a spreadsheet, and it's able to use them, but not the same way people process information. How does machine learning represent words and images as numbers? Everything a model sees is a list of numbers, called a vector.

Each item the model looks at is an example: a word, an apartment, a photo (the spreadsheet row is one too). This post covers vectors and matrices in machine learning, the first topic in our Fundamentals series on the math behind it. It shows how words, batches of examples and images each become numbers in their own way, and how the model then compares them.

Words as Vectors

Under the hood, machine learning models are typically a lot of math. As a result, these models need numbers, not the letters c, a, t. Each word is swapped for a vector, looked up in a table of vectors. The model adjusted those numbers while it was being built from lots of text. Our post on decision models explains that lookup. (Models really look up tokens, which are pieces of words, but we'll say words here.)

A vector with two numbers is easy to draw on a map (a flat plane with two directions, across and up). The arrow and the point in the demo below are two views of the same vector.

Word Vector
cat4433upacross43catdogcartruckcat4433upacross43catdogcartruck

Step 1 of 5

Toy numbers. The model starts with a word: "cat".

Step 1 of 5. Toy numbers. The model starts with a word: "cat".

In this series a vector is written in bold lowercase. Call the vector for "cat" x\mathbf{x} (the letter is arbitrary):

x=[43]\mathbf{x} = \begin{bmatrix} 4 \\ 3 \end{bmatrix}

Real models learn vectors from text, and they hold far more than two numbers. Words used alike tend to end up with similar vectors, so words like "cat" and "dog" tend to land near each other. Our example uses a small vector (two numbers), while many modern language models use much larger vectors (often thousands of numbers per token). Vectors that point the same way are considered similar, and we use math to calculate that. We'll go into what this looks like a bit later.

Batches as Matrices

Any example, not just a word, can be a vector. Say we describe an apartment by its area, number of rooms and age. Each measurement, one column of the grid below, is called a feature, so one apartment is a vector of three numbers.

Models usually handle a batch of examples (a group handled together) at a time. Stack four apartments, one under the other, and each row is one example, written as a vector. Together the rows form a grid of numbers: a matrix. One matrix lets the computer work on many rows at once, which is what GPUs (graphics processing units, the chips large models commonly run on) are built to do.

Batch Matrix
area (m²)roomsage (years)apartment 16565224545apartment 28080331212apartment 35050113030apartment 412012044554 rows × 3 columnsarea (m²)roomsage (years)apartment 16565224545apartment 28080331212apartment 35050113030apartment 412012044554 rows × 3 columns

Step 1 of 5

Toy numbers. One example, an apartment, is a vector: its area, rooms and age.

Step 1 of 5. Toy numbers. One example, an apartment, is a vector: its area, rooms and age.

Matrices are written in uppercase, so call this one XX. An entry of a matrix is written with its lowercase letter and two subscripts, row then column.

X=[x11x12x13x21x22x23x31x32x33x41x42x43]X = \begin{bmatrix} x_{11} & x_{12} & x_{13} \\ x_{21} & x_{22} & x_{23} \\ x_{31} & x_{32} & x_{33} \\ x_{41} & x_{42} & x_{43} \end{bmatrix}

So x23x_{23} is the age of apartment 2: row 2, column 3.

Images as Grids

A grayscale (black-and-white) image is a matrix: one number per pixel (one tiny square of the picture) for its brightness, usually from 0 (black) to 255 (white). A color image is three such matrices, one each for red, green and blue.

One way to feed a grid to a model is to read the pixels off row by row, giving one vector. That's called flattening. (Some models keep the grid shape instead.) The demo below flattens a toy image of the letter "T". A full-size color photo flattens the same way, into millions of numbers.

Image Grid
25025025525524524530302402402020101025525535352502502552552452453030240240202010102552553535One vector of 9 numbers25025025525524524530302402402020101025525535352502502552552452453030240240202010102552553535One vector of 9 numbers

Step 1 of 4

Toy numbers. A picture is a grid of tiny squares called pixels. This image is 3 × 3 and shows a T.

Step 1 of 4. Toy numbers. A picture is a grid of tiny squares called pixels. This image is 3 × 3 and shows a T.

Those nine numbers can be compared with any other nine, which is where the dot product comes in.

The Dot Product

A model often needs a similarity: a score for how similar two vectors are, mostly in which direction they point. The dot product is one such score, and it also grows with length (an arrow's length: how far its tip is from the center). To get it, multiply each pair of numbers in the same position, then add up the results. The total is the dot product. For two vectors a\mathbf{a} and b\mathbf{b} with nn numbers each:

a⋅b=a1b1+a2b2+⋯+anbn\mathbf{a} \cdot \mathbf{b} = a_1 b_1 + a_2 b_2 + \dots + a_n b_n

For a=[4,3]\mathbf{a} = [4, 3] and b=[3,4]\mathbf{b} = [3, 4] (the same vectors written sideways), that is 4×3+3×4=244 \times 3 + 3 \times 4 = 24.

Why does a big total suggest the vectors point the same way? A product is large and positive when both numbers are large and share a sign, and negative when their signs differ. When two vectors agree on most positions, the products pile up. When they disagree, they cancel or turn negative.

In the demo below, b keeps its length and turns, and the dot product follows.

Dot Product
abproducts4433−3−3−4−41212−12−12−16−16334444−3−312121212−9−9+a · b =242400−25−25ababproducts4433−3−3−4−41212−12−12−16−16334444−3−312121212−9−9+a · b =242400−25−25ab

Step 1 of 6

Toy numbers. Take two vectors, a and b, with the same number of entries. Each is written as numbers and drawn as an arrow.

Step 1 of 6. Toy numbers. Take two vectors, a and b, with the same number of entries. Each is written as numbers and drawn as an arrow.

The dot product can also be written with the angle θ\theta between the vectors:

a⋅b=∥a∥ ∥b∥cos⁡θ\mathbf{a} \cdot \mathbf{b} = \lVert\mathbf{a}\rVert \, \lVert\mathbf{b}\rVert \cos\theta

The double bars mean that length. The cosine is a number from 1 at angle 0, through 0 at a right angle, to −1 when the arrows are opposite; you don't need its formula here. That makes the dot product a similarity score for direction.

It's only one way to score similarity, because it grows with length too. Double a vector's length and its dot product with any other vector doubles, though it points the same way. Divide by both lengths and what's left is the cosine, called cosine similarity.

LLMs and GPUs

A large language model (LLM), the kind of model behind a chatbot, is built from these pieces. It splits your prompt into tokens and looks each one up as a vector, as in words as vectors. The prompt's vectors stack into a matrix, one row per token, like a batch. A model that also reads pictures turns an image into vectors too, starting from its grid of pixels. From there, layer after layer, the model reworks each row with dot products: against the other rows, to work out which words matter to each other, and against learned numbers called weights.

The last step is the easiest to see. To choose the next word, the model takes the vector it has built for the end of the prompt and works out its dot product with a vector for every word it knows. The higher the score, the likelier that word comes next.

Next Word
the cat sat on the ___the cat sat on the matvectorscoreprompt2, 12, 1mat4, 24, 21010sofa2, 32, 377car−3, 2−3, 2−4−4moon−2, −4−2, −4−8−8matsofacarmoonthe cat sat on the ___the cat sat on the matvectorscoreprompt2, 12, 1mat4, 24, 21010sofa2, 32, 377car−3, 2−3, 2−4−4moon−2, −4−2, −4−8−8matsofacarmoon

Step 1 of 4

Toy numbers. After reading "the cat sat on the", the model has built one vector for what comes next.

Step 1 of 4. Toy numbers. After reading "the cat sat on the", the model has built one vector for what comes next.

That adds up to a lot of dot products. Meta's Llama 3 8B model, for example, knows 128,000 tokens and uses 4,096 numbers per vector, according to Meta's paper. So that last step alone is 128,000 dot products of 4,096 pairs each: over 500 million multiplications for every word it writes, before counting the layers.

This is where GPUs come in. None of those dot products needs another's answer, so they can all be worked out at the same time. NVIDIA, which makes GPUs, describes a CPU (the general-purpose processor in a computer) as built to run a few tens of tasks at once, each as fast as possible, and a GPU as built to run thousands at once. A GPU's cores are simpler, but there are far more of them, and that is what this math needs.

CPU and GPU
CPU4 fast cores−6−6−1−14499−5−50011661111−3−3227788−6−6−1−14499−5−5−4−411661111−3−322GPU24 simpler cores−6−6−1−14499−5−50011661111−3−3227788−6−6−1−14499−5−5−4−411661111−3−322CPU4 fast cores−6−6−1−14499−5−50011661111−3−3227788−6−6−1−14499−5−5−4−411661111−3−322GPU24 simpler cores−6−6−1−14499−5−50011661111−3−3227788−6−6−1−14499−5−5−4−411661111−3−322

Step 1 of 4

Toy numbers. One layer's work for a small batch: 24 numbers, each its own dot product. None waits for another.

Step 1 of 4. Toy numbers. One layer's work for a small batch: 24 numbers, each its own dot product. None waits for another.

What Comes Next

Words, batches and images are now numbers, and the dot product compares them. That's the math under every LLM.

Next up is neural networks, where we'll go into more depth on the layers this post kept pointing to. A layer keeps its weights in a matrix and turns each input vector into a new one, with one dot product per row of weights.