Token

aiintermediate

A token is the basic unit of a Large Language Model's vocabulary. It's essentially how a model translates our language – words, characters, and punctuation – into computer language. Tokenization is the process of actually doing that conversion.

Mentioned in

Authentication

Authentication is how the apps you use know that you are who you say you are: through something as simple as a password, or as complex as biometrics.

Universeweb-apps

Tokenization

Tokenization is one of the ways that backends protect sensitive information, like credit cards or social security numbers.

Universecybersecurity

Why Technically Exists

Learn

Explore knowledge bases

Archives

Recent Posts

Token

Mentioned in

Authentication

Tokenization

Related terms

RAG

Machine Learning

ChatGPT