Word Piece
Noun · AI & Machine Learning
Definitions
A tokenization unit smaller than a whole word but larger than a character, commonly used to represent text efficiently in language models. Word pieces help models handle rare words and morphological variation.
In plain English: A sub-word token used in tokenization.
Example: "The tokenizer broke technical terms into word pieces so unseen compounds could still be represented effectively."