Contents
What is the VC dimension for linear support vector machines in d dimensional space?
“The VC Dimension of affine classifiers of the form f(x)=w⋅x+b in n dimensions − i.e. w∈Rn − is n+1”: this corresponds to the case of what is called a linear SVM. “The VC Dimension of an SVM equipped with an RBF kernel is infinite.”
What is the VC dimension of a linear classifier?
The VC-dimension of the set of linear classifiers is proportional to the data dimension, and equals the number of parameters of the classifiers. Note that in this case the classifiers have an additional parameter (the offset) and thus the VC-dimension equals the number of parameters in both cases.
How sure are you that large margin implies low VC dimension?
Large margin implies bounded Vapnik–Chervonenkis (VC) dimension. Bounded VC dimension implies high probability of low generalization error. So while large margin may indeed imply low VC dimension, we suspect most of our readers have never actually seen a rigorous proof of that step.
How is the VC dimension related to SVS?
The VC dimension doesn’t map to the amount of SVs of a given solution. The VC dimension is the maximum number of dataset samples that can be shattered perfectly by the model for any combination of the labels associate with those points. On the other hand, support vectors are the points that define the hyperplane.
Is the VC dimension of the SVM model proof?
In fact, as far as I know, there is no known proof of the VC dimension of the SVM model (we know, that it is a gap tolerant classifier, which should have lower VC dimension, but it is far from being a dimension proof). The VC dimension doesn’t map to the amount of SVs of a given solution.
Is there a dependency between support vectors and VC dimension?
Unfortunately there is no easy “one-to-one” dependency between the number of support vectors and the VC dimension of the data. In fact, as far as I know, there is no known proof of the VC dimension of the SVM model (we know, that it is a gap tolerant classifier, which should have lower VC dimension, but it is far from being a dimension proof).
Is the empirical VC dimension a true VC dimension?
Empirical VC dimension is not a true VC dimension, the true VC dimension is the analytical object, that cannot be directly computed “from the data”, it requires rigorous proofs, and one (of a few existing) says that for n-dimensional space, VC dimension of the linear classifiers is n+1, it holds no matter how you “get” to this space.