Embedding

Definitions

1. A dense vector representation of data (text, images, etc.) in a continuous vector space where semantic similarity corresponds to geometric proximity. The insight that king - man + woman ≈ queen (Word2Vec, 2013) was a watershed moment.

   In plain English: Converting words, sentences, or images into lists of numbers where similar things end up close together — allowing computers to understand that 'king' and 'queen' are related concepts.

2. In practical applications, embeddings power semantic search, recommendation systems, and RAG (retrieval-augmented generation). Instead of keyword matching, you compare the vector similarity of queries and documents — 'comfortable shoes' matches 'cozy footwear' even without shared words.

   Example: 'We replaced keyword search with embedding-based semantic search and relevance improved 40%. Users searching for 'how to undo a commit' now find the 'git reset' documentation.'

   Source: practical applications

Etymology

2003

Yoshua Bengio publishes 'A Neural Probabilistic Language Model,' learning word representations (embeddings) in continuous vector space

2013

Google's Word2Vec makes embeddings mainstream, demonstrating that king - man + woman ≈ queen

2022

OpenAI's Embedding API and vector databases make embeddings a practical tool for search, recommendation, and RAG systems

Related Terms

• RAG
• Neural Network
• Attention Mechanism
• Latent Space
• grounding
• retrieval
• regression
• clustering
• dimensionality reduction