Foundational Glossary

Browse 23 foundational terms defined in plain English, from the cultural dictionary of computing.

23 Foundational Terms

Attention Mechanism
Attention Mechanism is a component in neural networks that allows a model to dynamically focus on the most relevant parts of its input when producing each...
Big-O Notation
A mathematical notation describing the upper bound of an algorithm's time or space complexity as input size grows. O(1) is constant, O(log n) is logarithmic,...
Boolean Algebra
A branch of algebra dealing with variables that have two possible values: true/false (1/0). Operations include AND, OR, NOT, XOR, NAND, and NOR. The...
C
The foundational systems programming language that has shaped virtually all modern computing. C was created to rewrite Unix, and its direct hardware access,...
Church-Turing Thesis
The hypothesis that any function computable by an effective mechanical procedure can be computed by a Turing machine (or equivalently, by lambda calculus). Not...
Floating Point
A representation of real numbers using a mantissa and exponent (like scientific notation), defined by IEEE 754. Provides wide range but limited precision --...
GPT
Generative Pre-trained Transformer — OpenAI's series of large language models. GPT-3 (2020) demonstrated emergent abilities; GPT-3.5 powered the original...
Graph Theory
The mathematical study of graphs — structures consisting of vertices (nodes) connected by edges. Undirected or directed, weighted or unweighted, cyclic or...
Hacker
Hacker is a term with a rich and contested history in computing culture. In its original sense, dating to the 1960s at MIT, a hacker is someone who explores...
Halting Problem
The proven impossibility of writing a general algorithm that determines whether any given program will eventually stop (halt) or run forever. Alan Turing...
Lambda Calculus
A formal system for expressing computation using function abstraction (λx.body) and application (f x). Equivalent in power to Turing machines but models...
Linux
Linux is an open-source, Unix-like operating system kernel created by Linus Torvalds in 1991. Combined with the GNU userland and other software, it forms...
Machine Learning
Machine learning is a subset of artificial intelligence in which systems learn patterns from data rather than being explicitly programmed with rules. Instead...
Neural Network
A computing system loosely inspired by biological neural networks, consisting of layers of interconnected nodes (neurons) that process information by adjusting...
NP-Completeness
A class of problems where: (1) a proposed solution can be verified quickly (in polynomial time), but (2) no known algorithm can find a solution quickly. All...
Open Source
Open Source refers to software whose source code is publicly available, allowing anyone to view, modify, distribute, and use it, typically under a license that...
Public Key Cryptography
An encryption system using mathematically linked key pairs: a public key (shared openly) for encryption/verification and a private key (kept secret) for...
Regex
Regular Expression -- a pattern language for matching, searching, and manipulating text. Supported by virtually every programming language and tool (grep, sed,...
Shannon Entropy
A measure of the average information content (or unpredictability) in a message, measured in bits. Higher entropy means more randomness and less...
TCP/IP
The foundational protocol suite of the internet, consisting of TCP (reliable, ordered delivery) and IP (addressing and routing). The four-layer TCP/IP model...
Transformer
A neural network architecture that uses self-attention mechanisms to process sequential data in parallel rather than sequentially. Introduced in the 2017 paper...
Turing Completeness
The property of a computational system that can simulate any Turing machine — meaning it can compute anything that is computable, given enough time and memory....
Von Neumann Bottleneck
The performance limitation caused by the shared bus between CPU and memory in Von Neumann architecture — the processor spends significant time waiting for data...

Related Topics