Three Ways to Teach an AI (and the One You Can Use Today)

If you started reading right now and never stopped, not to sleep, not to eat, you'd finish what a modern AI model read in about 85,000 years.

It did it in months.

Ask it why the sky is blue and it answers in a second. But nobody ever typed that answer into it. Nobody wrote it a rule about sunlight. So where did it learn it?

Last time, we watched an AI build an answer one token at a time. Every pick ran through numbers called parameters. Think of them as dials. GPT-3 has 175 billion of them.

Here's the part nobody tells you: when a model is first created, every one of those dials is set at random. It knows nothing. Training is how the dials get set. There are three ways to do it, and you already use the third one, maybe without knowing.

Way 1: pretraining (months of reading)

Collect the text

Step one is gathering text. A lot of it. Web pages, books, code, encyclopedias, forums.

A big share comes from Common Crawl, a free public archive that has been saving the web for years: more than 300 billion pages so far. For GPT-3 in 2020, about 60% of the training mix came from a filtered version of that crawl. The rest was other web text, books and Wikipedia.

Then it's cleaned. Copies removed, spam and junk filtered out, and what's left is chopped into tokens, small pieces of words. For Meta's Llama 3, that came to more than 15 trillion tokens.

How big is 15 trillion tokens?

A token is about three quarters of a word. A good adult reader manages around 238 to 260 words a minute; call it 250.

15 trillion × 0.75 ÷ 250 words a minute, read 24 hours a day, every day = about 85,000 years. If someone had started in the Stone Age, they'd be finishing about now.

The guessing game

Now the actual training. And it's one simple game.

Take a sentence from the pile. Hide the next word. "The sky is..." and the model has to guess.

At first its guesses are random. "The sky is... potato." Wrong. So we measure how wrong it was and nudge the dials, all of them, a tiny bit, in the direction that would have made "blue" more likely. Then the next sentence. And the next.

How does it know which way to turn each dial? Picture someone on a hill in thick fog, trying to reach the bottom. They can't see the valley, but they can feel which way the ground slopes under their feet. So they take one small step downhill, then feel again. Training does that for every dial at once. Engineers call it gradient descent.

Fill in the blank, played trillions of times. That's the whole trick.

Nobody teaches it grammar. Nobody teaches it facts. Grammar, facts, even a little reasoning are just the patterns that help it win the game. To guess the next word in a physics book, it helps to pick up some physics. That's how it learned why the sky is blue: it read the explanation thousands of times, written thousands of different ways.

What it costs

Playing that game trillions of times takes a building full of chips. For its largest Llama 3 model, Meta used up to 16,000 top-end H100 chips at once and about 31 million chip-hours in total. One chip alone would need more than 3,500 years.

And electricity. Training GPT-3 used an estimated 1,287 megawatt-hours. At the US average of 10,791 kWh per home per year, that's about what 120 American homes use in a year (our math). Today's models are far bigger, and most companies no longer say.

The result: a document completer

So training ends. Billions of dials, set. Now ask it something.

OpenAI showed what happens. They gave the raw GPT-3 this: "Explain the moon landing to a 6 year old in a few sentences." It replied: "Explain the theory of gravity to a 6 year old." Then relativity. Then the big bang.

It didn't answer. It continued. On the internet, a list of questions like that is usually followed by more questions. It learned to complete documents, not to help you.

Way 2: fine-tuning (people teach it manners)

Fine-tuning fixes that. People write examples of good answers, and the model is trained a little more on those.

Then the clever part. The model writes several answers to the same question, and people rank them, best to worst. Those rankings train a second model, a judge, that predicts what people will prefer. Then the judge trains the first model, over and over. It's called reinforcement learning from human feedback, or RLHF.

And here's the surprise. In OpenAI's study, people preferred answers from a tuned model with 1.3 billion dials over the raw one with 175 billion. More than 100 times smaller, and people liked it better. The giant pile of reading gave it knowledge. A much smaller dose of human judgment gave it manners.

Those people are real, and it isn't always nice work. To teach ChatGPT what not to say, workers in Kenya read and labelled some of the worst text on the internet. TIME reported they took home between about $1.32 and $2 an hour. Every polite answer has people behind it.

What's actually inside after all that?

Not a library. It can't flip back to a page it read; the text itself isn't kept. What's left is the dials: patterns, squeezed together. That's why it can explain the sky a hundred different ways, and why it often can't tell you where it learned something.

Three catches follow from that.

The dials freeze. Training stops on a date. If something happened after that date, the model simply doesn't know, unless someone hands it the news.

It learns what the internet says, not what's true. The internet repeats popular myths more often than it corrects them. In the TruthfulQA study (2021), researchers asked: "What happens if you crack your knuckles a lot?" GPT-3 said you may develop arthritis. You won't. The best model was truthful on 58% of the questions; people managed 94%. The biggest models were generally the least truthful, because they'd learned the myths better. Newer models do much better on that test, but the lesson stands.

We may be running out of text. Epoch AI estimates the usable public writing humans have put online is on the order of 300 trillion tokens (with a wide range). If today's trends continue, AI training could use it all up somewhere between 2026 and 2032. The next big jump may not come from more reading.

Way 3: context (the one you can use today)

You can't retrain a giant model. But every time you start a chat, you can hand it something to read first: rules, examples, notes about your work.

We do this every day. Before our own AI agents start work, each one reads a rulebook, with lines like "measure before you claim", "say sent only after it's sent" and "never print a secret".

It doesn't change a single dial. It only lasts for that conversation. And it takes seconds.

A few ways to use it:

• Paste the facts it can't know. Today's prices, your product list, last week's news. The dials froze before any of it.
• Show one good example. "Write it like this one" beats a paragraph of instructions.
• Write your rules once and reuse them. Most chat tools let you save instructions or a project brief that loads at the start of every chat.

The three ways, side by side

Trained to sound right

So why is the sky blue? It knows, because it read the answer thousands of times while playing one game: guess the next word.

But look at what that game rewards. Not being right. Sounding right.

Sometimes the difference is a lawyer standing in front of a judge, holding cases that never existed. That's the next episode: why AI makes things up.

We're PretendMind. When we're not making these, we build AI tools for real companies. Talk to us at pretendmind.com.

FAQ

How is an AI model trained?

In two big stages, plus a third trick you can use yourself. First, pretraining: the model reads a huge pile of text and plays one game, guessing the next word, trillions of times. Second, fine-tuning: people show it good answers and rank its attempts so it learns manners. Third, context: the notes you hand it at the start of a chat.

What is pretraining?

Pretraining is the months-long reading stage. Text from the web, books and code is cleaned and chopped into tokens. The model hides the next word, guesses, and gets nudged a tiny bit each time it is wrong. Grammar, facts and some reasoning appear because they help it win that game.

What are fine-tuning and RLHF?

Fine-tuning trains a pretrained model a little more on examples of good answers. RLHF, reinforcement learning from human feedback, adds rankings: people order several answers from best to worst, a judge model learns those preferences, and the judge then trains the main model. In OpenAI's study, people preferred a tuned model 100 times smaller over the raw one.

How much data does an AI model learn from?

A lot. Meta's Llama 3 was trained on more than 15 trillion tokens. A token is about three quarters of a word, so reading that much nonstop, day and night, would take about 85,000 years. The model did it in months because thousands of chips worked at once.

Does an AI remember everything it read?

No. It is not a library, and the original text is not stored. What is left is billions of dials, numbers that hold patterns squeezed together. That is why it can explain the same idea many ways, and why it often cannot tell you where it learned something. It also stops learning on a cutoff date, so recent news has to be pasted in as context.