feature engineering
/FEE-chur en-juh-NEER-ing/ · noun · AI & Machine Learning · Origin: 2000
Definitions
The process of using domain knowledge to create, transform, or select input variables (features) that improve a machine learning model's predictive performance. This can include combining columns, extracting date components, encoding categorical variables, or computing rolling averages. Often considered more impactful than model selection itself.
In plain English: Crafting the right inputs for an AI model — like turning a raw date into 'day of week' or 'months since signup' — so the model can find better patterns.
Example: Feature engineering turned a mediocre model into the competition winner — they derived 'days since last purchase' and 'average basket size' from raw transaction logs.
Origin Story
The art of turning raw data into signals a model can learn from
**Feature engineering** as a practice is as old as machine learning itself, but the term gained prominence in the 2000s-2010s Kaggle competition era. Andrew Ng famously said: 'Coming up with features is difficult, time-consuming, requires expert knowledge. Applied machine learning is basically feature engineering.'
The idea is transforming raw data into informative representations. A timestamp becomes day-of-week, hour, and is-holiday features. Text becomes TF-IDF vectors or word embeddings. A street address becomes distance-to-downtown and neighborhood-crime-rate.
Deep learning reduced the need for manual feature engineering -- neural networks learn their own features from raw data. But for tabular data (the majority of business ML), feature engineering remains the single biggest lever for model performance. Kaggle grandmasters still win competitions primarily through creative feature engineering.
Coined by: Machine learning community
Context: 2000s-2010s, popularized by Kaggle and Andrew Ng
Fun fact: In the Netflix Prize competition (2006-2009), the winning team's solution was 107 models blended together. But the biggest single improvement came from a simple feature: how many days since the user's last rating. Feature engineering beat algorithm sophistication.