05 · AI & Search Infrastructure

Token

The basic unit of text a model processes.

Technical Term

Full explanation

A token is a unit of text, often a word or part of a word, that an AI language model processes as its basic building block.

Why this matters

It is the unit behind context limits and pricing, and useful to know whenever anyone claims a system has read an entire website.

Where this sits

Token belongs to AI & Search Infrastructure, one of the seven layers of the Aingworth Glossary. The technical vocabulary behind the interfaces. Useful for understanding how the systems work, without becoming the center of the conversation.

Related terms

Context Window How much text a model can hold in view at once. Large Language Model (LLM) A model trained on large amounts of text to understand and generate language. Embeddings Numerical representations that place similar meanings close together.
Back to AI & Search Infrastructure Back to Full Glossary

Ready to see where your brand stands?

Start with the free diagnostic to see which condition of the Recognition Triangle is leaking, or read the frameworks these terms belong to.

Run the Free Diagnostic Read the Frameworks