Contents
Which is a stronger assumption, GDA or logistic regression?
The converse, is not true though: being a logistic function does not imply is mutlivariate gaussian. This observation shows us that GDA has a much stronger assumption than Logistic Regression.
How is a derivation of a logistic regression done?
Traditional derivations of Logistic Regression tend to start by substituting the logit function directly into the log-likelihood equations, and expanding from there. The derivation is much simpler if we don’t plug the logit function in immediately. To maximize the log-likelihood, we take its gradient with respect to b:
How is the logit of P expressed in logistic regression?
The logistic regression model assumes that the log-odds of an observation y can be expressed as a linear function of the K input variables x: Here, we add the constant term b0, by setting x0 = 1. This gives us K+1 parameters. The left hand side of the above equation is called the logit of P (hence, the name logistic regression).
Can a GDA be used for non Gaussian data?
GDA (and Generative models in general), in the other hand, makes much stronger assumption, and thus is not ideal for non-Gaussian or some-crazy-undefined-distribution data .
Which is better logistic regression or Gaussian discriminant analysis?
GDA makes stronger modeling assumptions than logistic regression, so we would expect it to do better if our modeling assumptions are correct. Logistic regression makes weaker assumptions about our data, which means that it is more robust, so if we are wrong about our data being Gaussian, it will do better.
What’s the difference between LDA and binary logistic regression?
Logistic regression indeed does not assume any specific shapes of densities in the space of predictor variables, but LDA does. Here are some differences between the two analyses, briefly. Binary Logistic regression (BLR) vs Linear Discriminant analysis (with 2 groups: also known as Fisher’s LDA):