Transformer
Definition
A Transformer is a neural-network architecture built around attention mechanisms that process relationships between elements in a sequence or other structured input. Transformers underpin many modern language and multimodal foundation models.
Why it matters
Transformers made it practical to train large models that capture long-range contextual relationships and scale across broad datasets. They are central to modern LLMs, embeddings and many generative AI systems.
How it works
Token representations
- Input is represented as tokens or other discrete units before being processed by the model.
- Position information is incorporated so the model can distinguish sequence order.
Attention
- Attention mechanisms allow the model to weight relationships between different input elements.
- Multiple attention heads can learn different patterns of interaction.
Layered transformation
- Transformer blocks repeatedly transform contextual representations across layers.
- The resulting representations can support generation, classification, embeddings and other tasks.
How Netsleek uses the term
Netsleek uses Transformer as a foundational architecture term rather than as an AI search optimisation concept. It helps explain why attention, tokenisation, context and embeddings are connected, while keeping those model mechanics separate from retrieval and selection systems.
Comparisons
- Transformer vs LLM: Transformer is an architecture. An LLM is a trained language model that may use that architecture.
- Transformer vs Attention: Attention is a core mechanism used within Transformers, not a synonym for the whole architecture.
- Transformer vs Embedding Model: A Transformer can be used to build embedding models, but embedding generation is one application of the architecture.
How Transformer Connects Across AI Systems
Transformer does not operate in isolation. It connects to other established concepts across the AI search and retrieval process:
- Uses tokenisation: Language inputs are converted into tokens before model processing.
- Uses attention: Attention Mechanisms are a defining component of Transformer architectures.
- Supports LLMs: Many modern Large Language Models are Transformer-based.
- Produces contextual representations: Transformer representations can be used for generation or embeddings.
Related glossary concepts
- Attention Mechanisms
- Tokenization
- Large Language Model (LLM)
- Foundation Model
- Embedding Models
- Context Window
Common misinterpretations
- Transformer is not a synonym for AI.
- Transformers do not themselves perform web retrieval.
- Attention weights should not automatically be treated as human-readable explanations of model decisions.
Summary
The Transformer is a foundational neural architecture that uses attention to build contextual representations. It underpins many modern LLMs, embedding models and generative systems but remains distinct from retrieval, ranking and search infrastructure.