Lecture 23 NOC
Install Tubelator On Chrome
Video Summary & Chapters
No chapters for this video generated yet.
Video Transcript
Suppose I have some data which is not linearly separable right so that is the problem that
we had with perceptrons right so what happens if the data is not linearly separable perceptrons
do not converge so can we tweak our objective function that we have here to make sure that
we can handle non-linearly separable data is that right way of saying it.
to say non-linearly separable data was my question yeah linearly inseparable data right
so you have to be careful where you put the not the negation there right so what we do
in this case yeah somebody had the suggestion yeah so how will you do this right so there
are many ways there are many choices you can make right let me not play around with it
there are many choices you could make but there is one particular choice which is seems
to yield a very nice optimization formulation okay.
So what is the choice I am going to say that I would really like to maximize the margin
right and I would like to get as many data points correct as possible right so if you
think about it so there are a couple of things if this is the margin that I want right so
So what are the problems here well these data points are within the margin right, so I have
some data points that are within the margin so I like to minimize such cases there are
some data points that are within the margin and erroneous right I would like to minimize
such cases as well right.
If you think about it if I try to get this correct right there is a gap here and there
seems to be a gap here between the points if I try to get this correct and moved my
classification surface below then the margin would have been reduced even further right
so it is okay to get this wrong but then what about this guy is he within the margin or
outside the person within right so the margin for that class is defined on the other side
right. So the margin for that class is this side so anything to this side and X is within
the margin does it make sense right this will be yi times this right so this will actually
be negative so it is within the margin we want things to be greater than 1 yi times
f of X we want it to be greater than 1 right greater than or equal to 1 this is going to
be negative so obviously this is within the margin right make sense right so essentially
what I want to do is minimize these distances so you can see the distances I am not yet
so these distances I would like to minimize that make sense right.
So this is a certain small distance inside the margin right this is a large distance
inside the margin is a very large distance inside the margin likewise I can mark each
one of these and I want to minimize these so let us it is not terms as ?1 to ?5 and
I want to minimize those right essentially so if I minimize the sum of these deviations
I make along with my original along with my original objective function right I can handle
why do not I minimize the minimum here again that would minimize the maximum would essentially
mean that I will try to get as many things correct as possible so in this case I do not
mind getting something wrong as long as the overall deviation is not does not exceed a
certain limit see that the difference between minimizing the maximum and minimizing a sum
is that I might as well give up all of the sum to a single data point it might be something
that is very hard to classify right I might have one single outlier somewhere here right
let us draw this data might be perfectly separable and I might have an outlier there okay.
So now if I just say okay minimize the sum of the things it is fine right but if I say
minimize the max okay then it is going to actually give me a some hyper plane somewhere
there okay.
But like I said many different formulations are possible this one actually yields a very
nice computation that is one of the reasons people use this okay.
So what I am going to do is so I am going to say that this has to be that we had already
right I am going to introduce a slack variable so it does not have to be greater than M it
can be some fraction lesser also right M is what I would really like but I allow it to
have a slack right ideally I would want most of these ? is to be ? is to be 0 right I want
