# 本章小结

**BERT** 开创了双向预训练范式，在理解型任务上取得了突破性成果。**RoBERTa** 证明了充分训练的重要性，**ALBERT** 展示了参数效率优化的可行性，**ELECTRA** 通过替换检测任务大幅提升了训练效率。**Longformer** 和 **BigBird** 通过稀疏注意力使编码器能够高效处理长文本。

虽然编码器模型在特定理解任务上仍有优势，但仅解码器的大语言模型已在大多数场景中展现出同等甚至更强的能力。下一章将深入讨论解码器系列模型。

***

> 📝 **发现错误或有改进建议？** 欢迎提交 [Issue](https://github.com/yeasy/llm_internals/issues) 或 [PR](https://github.com/yeasy/llm_internals/pulls)。


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://yeasy.gitbook.io/llm_internals/di-si-bu-fen-qian-yan-yu-shi-jian-pian/12_encoder_models/summary.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.