Logistic regression: a convex optimization example, solved and checked
The logistic regression is a classic convex optimization problem. Here it is solved by a real solver (CLARABEL) and checked: every constraint, and the optimality certificate. Change the numbers below and solve your own.
Solve your own version free, no account needed
The problem
Fit P(y = 1) = 1/(1 + exp(-(w1 x1 + w2 x2))) (no intercept) to data (x1, x2, y): (1, 0, 1), (0, 1, 0), (1, 1, 1), (-1, 0, 0), (0, -1, 1), minimizing the negative log-likelihood plus 0.5 (w1^2 + w2^2).
How the model was built
The objective as written from the problem, before the numbers are multiplied out:
- With z = w1 x1 + w2 x2, a point with y = 1 adds log(1 + e−z) to the negative log-likelihood, and a point with y = 0 adds log(1 + ez).
- (1, 0, 1) and (−1, 0, 0) both give log(1 + e−w1), hence the 2 in front. (0, 1, 0) and (0, −1, 1) both give log(1 + ew2). (1, 1, 1) gives log(1 + e−w1 − w2).
- 0.5 w12 + 0.5 w22 is the penalty 0.5 (w12 + w22).
Assumptions:
- the negative log-likelihood is the sum over points of log(1+exp(-z)) when y=1 and log(1+exp(z)) when y=0, with z = w1*x1 + w2*x2
- points 1 and 4 share the same term log(1+exp(-w1)), and points 2 and 5 share log(1+exp(w2)), so each appears twice
The model
Each line has a plain-language note on the right.
| make as small as possible: regularized negative log-likelihood (nats) of the 5 data points, plus the ridge penalty 0.5*(w1^2 + w2^2) |
What is being decided
There are 2 decision variables.
| Variable | Type | Meaning |
|---|---|---|
| w1 | real number | weight on feature x1 (log-odds per unit x1) |
| w2 | real number | weight on feature x2 (log-odds per unit x2) |
Goal and limits
Minimize regularized negative log-likelihood (nats) of the 5 data points, plus the ridge penalty 0.5*(w1^2 + w2^2) subject to 0 constraints.
Problem type
Conic (convex), complexity level 4: convex nonlinear functions such as log, exp, square roots or powers. Solved with CLARABEL at the Medium solver tier.
Picture
Shaded: every point that satisfies all the constraints. Dot: the optimum. Dashed: the objective's level line through it.
The answer at a glance
The optimal value of each decision, as the solver returned it.
Algebraic model (low level)
Decision variables, objective and every constraint written out; this is exactly what the solver receives.
Model code (cvxpy)
The same model in cvxpy, ready to run.
import cvxpy as cp
w1 = cp.Variable(name='w1') # weight on feature x1 (log-odds per unit x1)
w2 = cp.Variable(name='w2') # weight on feature x2 (log-odds per unit x2)
objective = cp.Minimize(0.5*w1**2 + 0.5*w2**2 + 2*cp.log(1 + cp.exp(-w1)) + 2*cp.log(cp.exp(w2) + 1) + cp.log(cp.exp(-w1 - w2) + 1))
constraints = [
]
problem = cp.Problem(objective, constraints)
problem.solve(solver='CLARABEL')
print(problem.status, problem.value)
print('w1 =', w1.value)
print('w2 =', w2.value)Solve your own
This is the model as the app reads it. Change any number and solve it in the free box on the front page (no account needed).
minimize 2*log(1 + exp(-w1)) + 2*log(1 + exp(w2)) + log(1 + exp(-(w1 + w2))) + 0.5*(w1^2 + w2^2)
subject to
w1 free
w2 free
# w1 = weight on feature x1 (log-odds per unit x1)
# w2 = weight on feature x2 (log-odds per unit x2)
# objective = regularized negative log-likelihood (nats) of the 5 data points, plus the ridge penalty 0.5*(w1^2 + w2^2)
# assume: the negative log-likelihood is the sum over points of log(1+exp(-z)) when y=1 and log(1+exp(z)) when y=0, with z = w1*x1 + w2*x2
# assume: points 1 and 4 share the same term log(1+exp(-w1)), and points 2 and 5 share log(1+exp(w2)), so each appears twiceChecked: the app's answer matches the reference answer in our test set, which two solvers (CLARABEL, SCS) agree on. Built 2026-11-10.
Related examples
Try ProvenDecisions free: type, photograph or say an optimization problem; every answer is checked by a solver.