Purpose
This article explains the most important LLM terms that appear repeatedly in documentation, discussions, and Giotto configuration files.
Overview
LLM systems come with a lot of vocabulary. Some of it is model theory, some of it is runtime terminology, and some of it appears directly in configuration files.
This article introduces the essential terms you need before moving to generation, training, and Giotto YAML.
Token
A token is a small unit of text processed by the model.
A token is not always the same as a word. Depending on the tokenizer, a token may be:
- a whole word
- part of a word
- punctuation
- a symbol
- a special control token
When a model generates text, it is really generating tokens one step at a time.
Tokenizer
A tokenizer is the component that converts text into tokens and tokens back into text.
The model does not directly read raw text. It reads token IDs produced by the tokenizer. This is why tokenizer choice matters and why configuration files may include fields such as tokenizer_id.
Prompt
A prompt is the text input given to the model.
This may include:
- a question
- instructions
- system behavior rules
- prior conversation
- examples
In many systems, prompt design has a strong effect on output behavior.
Context window
The context window is the amount of text the model can consider at once.
This matters because the model only sees a limited amount of recent input. If the conversation or document is longer than the context window, some information may be truncated or omitted.
Inference
Inference is the act of running the model to produce outputs.
In training, the model learns. In inference, the trained model is used to answer a question, generate text, classify input, or perform another runtime task.
Training
Training is the process of updating the model’s parameters so that it learns from data.
In practice, training is how a base model becomes better suited for a task, dataset, or domain.
Fine-tuning
Fine-tuning is training performed on top of an existing pretrained model.
Instead of learning from scratch, the model starts from a pretrained base and is adapted to a narrower task or domain.
Dataset
A dataset is the collection of examples used for training, evaluation, decoding, or benchmarking.
In Giotto, datasets are declared in configuration files and later used by stages such as data preparation, training, and decoding.
Epoch
An epoch is one full pass through the training dataset.
If num_train_epochs is 10, the training process goes through the dataset ten times.
Batch size
A batch size is the number of examples processed together in one training or inference step.
Batch size affects:
- speed
- memory usage
- optimization behavior
In configs, you may see values such as per_device_batch_size.
Learning rate
The learning rate controls how strongly the model updates its parameters during training.
A learning rate that is too high may make training unstable. A learning rate that is too low may make training very slow or ineffective.
Gradient accumulation
Gradient accumulation allows the system to simulate a larger effective batch size by accumulating gradients over several smaller steps before updating the model.
This is useful when GPU memory is limited.
Decoding
Decoding is the process of turning model probabilities into actual output text.
This is where terms such as greedy, beam_search, temperature, and max_new_tokens become important.
Artifact
An artifact is any stored output or input resource used in the workflow.
Examples include:
- models
- checkpoints
- datasets
- decoded outputs
- scored outputs
In Giotto, artifacts are often stored in object storage and referenced through YAML.
Serving
Serving means exposing a model or system so it can handle requests at runtime.
Training produces models. Serving makes them available to applications or users.
Agent
An agent is a runtime component that uses an LLM under a defined role, often with instructions, tools, and delegation rules.
In Giotto, online systems are often made of multiple agents.
Tool
A tool is an external callable capability that an agent can use.
Examples include:
- Python execution
- memory search
- memory storage
- API calls
Orchestration
Orchestration is the control layer that coordinates how stages, services, agents, and tools interact.
This is one of the core ideas behind Giotto.
Key takeaways
- Most LLM terminology falls into input, training, inference, or system-runtime concepts.
- Many configuration parameters in Giotto directly correspond to these terms.
- Understanding these words makes Giotto configuration easier to understand.
Comments
0 comments
Article is closed for comments.