Vocabulary Glossary

Browse 2 vocabulary terms defined in plain English, from the cultural dictionary of computing.

2 Vocabulary Terms

Byte Pair Encoding: A tokenization algorithm that repeatedly merges common symbol pairs to build a useful subword vocabulary. It influences how models are trained, evaluated, or...
Vocabulary Size: The number of unique tokens available in a model's vocabulary. Vocabulary size affects tokenization efficiency, memory usage, and how text in different...