SEARCH LAVACS

Find a topic across LavaCS

Search textbooks, study guides, six encyclopedias, experiments and comparisons.

10 results · page 1 of 1

Transformers and Large Language Models · Comparisons and explanations

Encoder versus decoder models

Compare encoder-only, decoder-only, and encoder-decoder models by visibility, output construction, and task rather than assuming exclusive capabilities.

Intermediate

Transformers and Large Language Models · Concepts and theorems

Attention masks

Separate causal and padding constraints, place masking before softmax, and account for position offsets when a KV cache is present.

Intermediate

Transformers and Large Language Models · Concepts and theorems

Context windows

Calculate input-output sequence budgets and distinguish configured length, truncation, retrieval, and effective use of long context.

Intermediate

Transformers and Large Language Models · Concepts and theorems

Embedding spaces

Use small vectors to inspect dimensions, distance, and normalization, separating retrieval similarity from factual correctness.

Intermediate

Transformers and Large Language Models · Concepts and theorems

Positional encodings

Use sinusoidal coordinates and two-dimensional rotations to explain sequence order, separating positional representation from guaranteed long-context generalization.

Intermediate

Transformers and Large Language Models · Concepts and theorems

Tokens

Distinguish tokens from characters, bytes, and words, and inspect how IDs, normalization, and special tokens affect length and decoding.

Intermediate

Transformers and Large Language Models · Experiments and implementations

Implementing and checking attention masks

Implement allowed-position attention and verify zero forbidden weights, row normalization, cached offsets, and explicit empty-row handling.

Intermediate

Transformers and Large Language Models · Experiments and implementations

Exploring tokenizer, character, and byte boundaries

Use UTF-8, a combining character, and two explicit merge rules to reproduce differences between code points, bytes, and toy tokens.

Intermediate