How is loss function defined?
In mathematical optimization and decision theory, a loss function or cost function (sometimes also called an error function) is a function that maps an event or values of one or more variables onto a real number intuitively representing some “cost” associated with the event.
Why is loss function used?
At its core, a loss function is a measure of how good your prediction model does in terms of being able to predict the expected outcome(or value). We convert the learning problem into an optimization problem, define a loss function and then optimize the algorithm to minimize the loss function.
When to use loss function in linear regression?
Decision boundary can be described as: Predict 1, if θᵀx ≥ 0 → h (x) ≥ 0.5; Predict 0, if θᵀx < 0 → h (x) < 0.5. Linear regression uses Least Squared Error as loss function that gives a convex graph and then we can complete the optimization by finding its vertex as global minimum. However, it’s not an option for logistic regression anymore.
What is the cost of a loss function?
If y = 1, looking at the plot below on left, when prediction = 1, the cost = 0, when prediction = 0, the learning algorithm is punished by a very large cost. Similarly, if y = 0, the plot on right shows, predicting 0 has no punishment but predicting 1 has a large value of cost.
What do you need to know about logistic regression?
Logistic regression just has a transformation based on it. For logistic regression, focusing on binary classification here, we have class 0 and class 1. To compare with the target, we want to constrain predictions to some values between 0 and 1.
How are loss functions used in supervised learning?
This series aims to explain loss functions of a few widely-used supervised learning models, and some options of optimization algorithms. In part I, I walked through the optimization process of Linear Regression in details by using Gradient Descent and using Least Squared Error as loss function. In this part, I will move to Logistic Regression.