Is logistic regression good for small dataset?
For small datasets, one thing one must avoid is ‘overfitting the data’ hence simple machine learning like ‘Logistics Regression, Linear Regression and Bayesian Linear Regression will do fine… As others suggested you can use any of the machine learning algorithms that supports your data set attributes.
What constitutes a small dataset?
Small data is data that is ‘small’ enough for human comprehension. It is data in a volume and format that makes it accessible, informative and actionable. Another definition of small data is: The small set of specific attributes produced by the Internet of Things.
Can a regression model be used on a small data set?
I understand that the data set is super small (even smaller if one applies e.g. a 60%/40% train-test split) and that this carries a high risk of over-fitting especially when using complex models like neural networks. What would be a reasonable model to use in such a case and what would be the best way to avoid over-fitting?
How to deal with a small data set?
With scarce data, your goal is to limit the model’s ability to see non-existent patterns and relationships. This means that you want to limit the number of weights and parameters and rule out all models that imply non-linearity or feature interactions.
What is the curse of small datasets in machine learning?
This is Part 1 of Breaking the curse of small datasets in Machine Learning. In this part, I will discuss how the size of the data set impacts traditional Machine Learning algorithms and few ways to mitigate these issues.
What’s the best way to do boosted regression?
The best way to do it is use boosted regression (see xgboost, does a great job right now, very popular and easy to understand) with a good validation process like this one I use a lot right now on small datasets: http://dataneel.github.io/nx2_cross_validation/