Support vector machine (soft margin): a quadratic programming example, solved and checked
The support vector machine (soft margin) is a classic quadratic programming problem. Here it is solved by a real solver (CLARABEL) and checked: every constraint, and the optimality certificate. Change the numbers below and solve your own.
Solve your own version free, no account needed
The problem
Classify points on a line: 2, 3 and 4 are class +1; 0, 1 and 2.5 are class -1. Find w and b minimizing 0.5 w^2 + C * (sum of hinge losses max(0, 1 - y_i (w x_i + b))) with C = 1.
How the model was built
The objective as written from the problem, before the numbers are multiplied out:
- hi is point i's hinge loss: hi ≥ 1 − yi (w xi + b) and hi ≥ 0, so at the best answer hi = max(0, 1 − yi (w xi + b)). It is 0 once the point is on its own side with a margin of at least 1.
- The point 2 of class +1 gives 1 − (2w + b), so h1 ≥ −b − 2w + 1. The point 0 of class −1 gives 1 − (−1)(w × 0 + b) = 1 + b, so h4 ≥ b + 1.
- 0.5 w2 rewards a wide margin; C = 1 weighs the losses h1 + … + h6 against it.
The model
Each line has a plain-language note on the right.
| make this as small as possible | |
| limit 1: left side must reach at least the right side | |
| limit 2: left side must reach at least the right side | |
| limit 3: left side must reach at least the right side | |
| limit 4: left side must reach at least the right side | |
| limit 5: left side must reach at least the right side | |
| limit 6: left side must reach at least the right side | |
| hinge loss for point x=2, class +1 cannot be negative | |
| hinge loss for point x=3, class +1 cannot be negative | |
| hinge loss for point x=4, class +1 cannot be negative | |
| hinge loss for point x=0, class -1 cannot be negative | |
| hinge loss for point x=1, class -1 cannot be negative | |
| hinge loss for point x=2.5, class -1 cannot be negative |
In matrix form
With its numbers:
What is being decided
There are 8 decision variables.
| Variable | Type | Meaning |
|---|---|---|
| h1 | real number | hinge loss for point x=2, class +1 |
| w | real number | slope of the separating line (per unit x) |
| b | real number | intercept/bias of the separating line |
| h2 | real number | hinge loss for point x=3, class +1 |
| h3 | real number | hinge loss for point x=4, class +1 |
| h4 | real number | hinge loss for point x=0, class -1 |
| h5 | real number | hinge loss for point x=1, class -1 |
| h6 | real number | hinge loss for point x=2.5, class -1 |
Goal and limits
Minimize the objective subject to 12 constraints, of which 6 are simple bounds on a single variable.
Problem type
Convex QP, complexity level 3: a convex quadratic objective or constraint. Solved with CLARABEL at the Medium solver tier.
The answer at a glance
The optimal value of each decision, as the solver returned it.
Which constraints hold the answer in place
Binding (no slack left): constraints 1, 2, 5, 6, 8, 9, 10, 11 (see the table). Changing a binding constraint changes the answer; a slack one can move a little without effect.
| # | Constraint | Value | Limit | Slack | Binding |
|---|---|---|---|---|---|
| 1 | h1 >= -b - 2*w + 1 | 1 | 1 | 0 | yes |
| 2 | h2 >= -b - 3*w + 1 | 9.95069e-09 | 9.5642e-09 | 0 | yes |
| 3 | h3 >= -b - 4*w + 1 | 0 | -1 | 1 | |
| 4 | h4 >= b + 1 | 0 | -1 | 1 | |
| 5 | h5 >= b + w + 1 | 9.4156e-09 | 8.94544e-09 | 0 | yes |
| 6 | h6 >= b + 2.5*w + 1 | 1.5 | 1.5 | 0 | yes |
| 7 | h1 >= 0 | 1 | 0 | 1 | |
| 8 | h2 >= 0 | 9.95069e-09 | 0 | 9.95069e-09 | yes |
| 9 | h3 >= 0 | 0 | 0 | 0 | yes |
| 10 | h4 >= 0 | 0 | 0 | 0 | yes |
| 11 | h5 >= 0 | 9.4156e-09 | 0 | 9.4156e-09 | yes |
| 12 | h6 >= 0 | 1.5 | 0 | 1.5 |
Algebraic model (low level)
Decision variables, objective and every constraint written out; this is exactly what the solver receives.
Model code (cvxpy)
The same model in cvxpy, ready to run.
import cvxpy as cp
h1 = cp.Variable(name='h1') # hinge loss for point x=2, class +1
w = cp.Variable(name='w') # slope of the separating line (per unit x)
b = cp.Variable(name='b') # intercept/bias of the separating line
h2 = cp.Variable(name='h2') # hinge loss for point x=3, class +1
h3 = cp.Variable(name='h3') # hinge loss for point x=4, class +1
h4 = cp.Variable(name='h4') # hinge loss for point x=0, class -1
h5 = cp.Variable(name='h5') # hinge loss for point x=1, class -1
h6 = cp.Variable(name='h6') # hinge loss for point x=2.5, class -1
objective = cp.Minimize(h1 + h2 + h3 + h4 + h5 + h6 + 0.5*w**2)
constraints = [
h1 >= -b - 2*w + 1,
h2 >= -b - 3*w + 1,
h3 >= -b - 4*w + 1,
h4 >= b + 1,
h5 >= b + w + 1,
h6 >= b + 2.5*w + 1,
h1 >= 0,
h2 >= 0,
h3 >= 0,
h4 >= 0,
h5 >= 0,
h6 >= 0,
]
problem = cp.Problem(objective, constraints)
problem.solve(solver='CLARABEL')
print(problem.status, problem.value)
print('h1 =', h1.value)
print('w =', w.value)
print('b =', b.value)
print('h2 =', h2.value)
print('h3 =', h3.value)
print('h4 =', h4.value)
print('h5 =', h5.value)
print('h6 =', h6.value)Solve your own
This is the model as the app reads it. Change any number and solve it in the free box on the front page (no account needed).
minimize 0.5 * w^2 + h1 + h2 + h3 + h4 + h5 + h6
subject to
h1 >= 1 - (2*w + b)
h2 >= 1 - (3*w + b)
h3 >= 1 - (4*w + b)
h4 >= 1 + (0*w + b)
h5 >= 1 + (1*w + b)
h6 >= 1 + (2.5*w + b)
h1, h2, h3, h4, h5, h6 >= 0
# w = slope of the separating line (per unit x)
# b = intercept/bias of the separating line
# h1 = hinge loss for point x=2, class +1
# h2 = hinge loss for point x=3, class +1
# h3 = hinge loss for point x=4, class +1
# h4 = hinge loss for point x=0, class -1
# h5 = hinge loss for point x=1, class -1
# h6 = hinge loss for point x=2.5, class -1Checked: the app's answer matches the reference answer in our test set, which two solvers (CLARABEL, OSQP) agree on. Built 2026-11-10.
Related examples
Try ProvenDecisions free: type, photograph or say an optimization problem; every answer is checked by a solver.