# 第五章：预训练：为什么“预测下一个词”能学到知识

预训练（Pre-training）是现代大语言模型的基石。一个看似简单的任务——“预测下一个词”——为什么能让模型学会语法、语义、推理乃至世界知识？不同的预训练范式（自回归、掩码语言模型、编码器-解码器）的本质区别是什么？预训练数据的规模和质量如何影响模型能力？

本章将系统回答这些问题，揭示预训练策略背后的深层逻辑。


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://yeasy.gitbook.io/llm_internals/di-er-bu-fen-xun-lian-pian/05_pretraining.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.