Contents
Is text data linearly separable?
Most text categorization problems are linearly separable.
Why is text data linearly separable?
The higher the dimensionality, the easier it is to linearly separate data, as the VC dimension of a linear classifier in d dimensions is d+1 (e.g. see these slides). The VC dimension is the largest amount of points that a classifier can shatter (separate).
How do you deal with problems which are not linearly separable?
In cases where data is not linearly separable, kernel trick can be applied, where data is transformed using some nonlinear function so the resulting transformed points become linearly separable. A simple example is shown below where the objective is to classify red and blue points into different classes.
When is a data point clearly linearly separable?
Let us start with a simple two-class problem when data is clearly linearly separable as shown in the diagram below. Let the i-th data point be represented by ( X i, y i) where X i represents the feature vector and y i is the associated class label, taking two possible values +1 or -1.
When to use SVM when data is not linearly separable?
SVM is quite intuitive when the data is linearly separable. However, when they are not, as shown in the diagram below, SVM can be extended to perform well. There are two main steps for nonlinear generalization of SVM.
When is data is not linearly separable Stat 508?
The maximal marginal hyperplane found in the new space corresponds to a nonlinear separating hypersurface in the original space. Suppose the original feature space includes two variables X 1 and X 2. Using polynomial transformation the space is expanded to ( X 1, X 2, X 1 2, X 2 2, X 1 X 2 ). Then the hyperplane would be of the form
Can a straight line be drawn to separate the balls?
A straight line can be drawn to separate all the members belonging to class +1 from all the members belonging to the class -1. The two-dimensional data above are clearly linearly separable. In fact, an infinite number of straight lines can be drawn to separate the blue balls from the red balls.