The AI Reference

The guide to AI that you've been looking for. Click on any term for an in depth explanation and FAQ (plus diagrams).

Concepts

Context Window

A context window is how much data an AI model can hold in memory at once.

Embeddings

Embeddings are how AI models turn words, images, or other data into mathematical coordinates that computers can actually work with.

Instructional Fine-tuning

Instructional fine-tuning is how you turn a pre-trained AI model – knowledgeable, but useless – into a helpful assistant that actually answers your questions.

Token

A token is the basic unit of a Large Language Model's vocabulary.

Training

Fine Tuning

Fine tuning is the process of taking a pre-trained AI model and specializing it for your specific use case.

Loss Function

A loss function is something you design that tells the model when its answers are right and when they're wrong.

Post-Training

Post-training turns a model from a knowledgeable blob that produces rambling answers, into a helpful assistant.

Pre-Training

Pre-training is the "undergrad degree" phase where the model builds its foundational knowledge and world model.

Reinforcement Learning from Human Feedback (RLHF)

RLHF is the final training step that turns a knowledgeable but rambling AI into the helpful assistant you know and love.

Training

Training is the process of creating an AI model and teaching it how to actually do something useful.

Training Dataset

Training datasets are the examples you show an AI model so it can learn to recognize patterns and make predictions.

Inference

Hallucination

AI hallucination is when AI models confidently generate information that's completely made up or wrong.

Inference

Inference is a fancy term that just means using an ML model that has already been trained.

Prompt Engineering

Prompt engineering is the art of talking to AI models in a way that gets you the results you actually want.

RAG

Retrieval Augmented Generation (RAG) is a way to make LLMs like GPT-4 more accurate and personalized to your specific data

Reasoning

AI reasoning is how artificial intelligence systems solve problems, think through complex situations, and draw conclusions from available information.

Model Architecture

Neural Network

Neural networks are the mathematical brains behind modern AI—think of them as simplified versions of how your actual brain processes information.

Parameters

Parameters are the learned "knowledge" stored inside AI models—the numerical values that determine how the model responds to inputs.

Transformers

Transformers are the neural network architecture that powers modern AI like ChatGPT, revolutionizing how models process language and other sequential data.

Infrastructure

GPU (Graphics Processing Unit)

GPUs are specialized chips that do thousands of simple calculations simultaneously, making them perfect for AI training

Impress your engineers

70K+ product managers, marketers, bankers, and other -ers read Technically to understand software and work better with developers.

Learn

Explore knowledge bases

Meet Technically

Solutions for Teams

The AI Reference

Concepts

Context Window

Embeddings

Instructional Fine-tuning

Token

Training

Fine Tuning

Loss Function

Post-Training

Pre-Training

Reinforcement Learning from Human Feedback (RLHF)

Training

Training Dataset

Inference

Hallucination

Inference

Prompt Engineering

RAG

Reasoning

Model Architecture

Neural Network

Parameters

Transformers

Infrastructure

GPU (Graphics Processing Unit)

Impress your engineers

Learn

Explore knowledge bases

Meet Technically

Solutions for Teams

The AI Reference

Concepts

Context Window

Embeddings

Instructional Fine-tuning

Token

Training

Fine Tuning

Loss Function

Post-Training

Pre-Training

Reinforcement Learning from Human Feedback (RLHF)

Training

Training Dataset

Inference

Hallucination

Inference

Prompt Engineering

RAG

Reasoning

Model Architecture

Neural Network

Parameters

Transformers

Infrastructure

GPU (Graphics Processing Unit)

Impress your engineers