Contents
What is a feature cross?
A feature cross is a synthetic feature formed by multiplying (crossing) two or more features. Crossing combinations of features can provide predictive abilities beyond what those features can provide individually.
Are more features always better?
If the features are not helpful, then a small feature size that provides a similar accuracy to a model with a large feature is always more helpful because performance increases in terms of obtaining classification/regression results faster. Too many features is often a bad thing.
Why do too many features lead to Overfitting?
Too many features can lead to overfitting because it can increase model complexity. There is greater chance of redundancy in features and of features that are not at all related to prediction.
What is synthetic feature?
create a synthetic feature which is a ratio of two other features. use this new feature as an input to a linear regression model. improve the effectiveness of the model by identifying and clipping (removing) outliers out of the input data.
Which is an example of a feature cross?
Think of feature crosses of one-hot feature vectors as logical conjunctions. For example, suppose we have two features: country and language. A one-hot encoding of each generates vectors with binary features that can be interpreted as country=USA, country=France or language=English, language=Spanish .
How are feature crosses used in machine learning?
So far, we’ve focused on feature-crossing two individual floating-point features. In practice, machine learning models seldom cross continuous features. However, machine learning models do frequently cross one-hot feature vectors. Think of feature crosses of one-hot feature vectors as logical conjunctions.
How is a feature cross related to latitude and longitude?
This feature cross is a 25-element one-hot vector (24 zeroes and 1 one). The single 1 in the cross identifies a particular conjunction of latitude and longitude. Your model can then learn particular associations about that conjunction. Suppose we bin latitude and longitude much more coarsely, as follows:
How to increase the importance of a feature?
There are many different ways of increasing your model understanding and feature importance is one of them. Feature importance helps you estimate how much each feature of your data contributed to the model’s prediction.