Least absolute deviations regression: a linear programming example, solved and checked
The least absolute deviations regression is a classic linear programming problem. Here it is solved by a real solver (HIGHS) and checked: every constraint, the dual, and the optimality certificate. Change the numbers below and solve your own.
Solve your own version free, no account needed
The problem
Fit y = a + b t to the points (0, 1), (1, 2), (2, 2), (3, 4), (4, 5) minimizing the sum of absolute residuals.
How the model was built
The objective as written from the problem, before the numbers are multiplied out:
- Each term is the miss at one data point. At t = 2 the line gives a + 2b and the data say 2, so the miss is |a + 2b − 2|.
- The solver turns each |…| into a new variable that is at least the miss and at least minus the miss, which makes it a linear program.
The model
Each line has a plain-language note on the right.
| make this as small as possible |
In matrix form
With its numbers:
The problem as a linear program
Each absolute value (or max, min) gets a helper variable, tied to the term it stands for by the lines below: t1 stands for |a - 1|; t2 stands for |a + 4*b - 5|; t3 stands for |a + 3*b - 4|; t4 stands for |a + b - 2|; t5 stands for |a + 2*b - 2|. At the optimum each helper equals its term, so this linear program has the same optimum as your problem; it is what the solver solved, and what the dual is built from.
| the same objective, with each helper in place of the term it stands for | |
| t1 is at least a - 1 | |
| and at least 1 - a, so t1 is at least |a - 1| | |
| t2 is at least a + 4*b - 5 | |
| and at least -a - 4*b + 5, so t2 is at least |a + 4*b - 5| | |
| t3 is at least a + 3*b - 4 | |
| and at least -a - 3*b + 4, so t3 is at least |a + 3*b - 4| | |
| t4 is at least a + b - 2 | |
| and at least -a - b + 2, so t4 is at least |a + b - 2| | |
| t5 is at least a + 2*b - 2 | |
| and at least -a - 2*b + 2, so t5 is at least |a + 2*b - 2| |
What is being decided
There are 2 decision variables.
| Variable | Type | Meaning |
|---|---|---|
| b | real number | slope of fitted line (y per unit t) |
| a | real number | intercept of fitted line (y units) |
Goal and limits
Minimize the objective subject to 0 constraints.
Problem type
LP (after reformulating |.|, max, min), complexity level 1: a small linear program. Solved with HIGHS at the Medium solver tier.
Picture
Shaded: every point that satisfies all the constraints. Dot: the optimum. Dashed: the objective's level line through it.
The answer at a glance
The optimal value of each decision, as the solver returned it.
Algebraic model (low level)
Decision variables, objective and every constraint written out; this is exactly what the solver receives.
Model code (cvxpy)
The same model in cvxpy, ready to run.
import cvxpy as cp
b = cp.Variable(name='b') # slope of fitted line (y per unit t)
a = cp.Variable(name='a') # intercept of fitted line (y units)
objective = cp.Minimize(cp.abs(a - 1) + cp.abs(a + b - 2) + cp.abs(a + 2*b - 2) + cp.abs(a + 3*b - 4) + cp.abs(a + 4*b - 5))
constraints = [
]
problem = cp.Problem(objective, constraints)
problem.solve(solver='HIGHS')
print(problem.status, problem.value)
print('b =', b.value)
print('a =', a.value)The dual problem
Every constraint of the original (primal) problem gets a price, and every primal variable becomes a constraint on those prices. First the problem was written as an exact linear program with helper variables t1, t2, t3, t4, t5: each helper is held at or above (or below) the terms it stands for, so at the optimum it equals them and the optimum is unchanged. This is the dual of that linear program. Strong duality: primal and dual optima are equal (1).
| Dual variable | Prices | Value |
|---|---|---|
| y1 | price of constraint 't1 >= a - 1' (objective change per extra unit of its right-hand side) | 0.5 |
| y2 | price of constraint 't1 >= 1 - a' (objective change per extra unit of its right-hand side) | 0.5 |
| y3 | price of constraint 't2 >= a + 4*b - 5' (objective change per extra unit of its right-hand side) | 0 |
| y4 | price of constraint 't2 >= -a - 4*b + 5' (objective change per extra unit of its right-hand side) | 1 |
| y5 | price of constraint 't3 >= a + 3*b - 4' (objective change per extra unit of its right-hand side) | 1 |
| y6 | price of constraint 't3 >= -a - 3*b + 4' (objective change per extra unit of its right-hand side) | 0 |
| y7 | price of constraint 't4 >= a + b - 2' (objective change per extra unit of its right-hand side) | 0 |
| y8 | price of constraint 't4 >= -a - b + 2' (objective change per extra unit of its right-hand side) | 1 |
| y9 | price of constraint 't5 >= a + 2*b - 2' (objective change per extra unit of its right-hand side) | 1 |
| y10 | price of constraint 't5 >= -a - 2*b + 2' (objective change per extra unit of its right-hand side) | 0 |
Solve your own
This is the model as the app reads it. Change any number and solve it in the free box on the front page (no account needed).
minimize abs(1 - a - b*0) + abs(2 - a - b*1) + abs(2 - a - b*2) + abs(4 - a - b*3) + abs(5 - a - b*4)
subject to
a, b free
# a = intercept of fitted line (y units)
# b = slope of fitted line (y per unit t)Checked: the app's answer matches the reference answer in our test set, which two solvers (HIGHS, CLARABEL) agree on. Built 2026-11-10.
Related examples
Try ProvenDecisions free: type, photograph or say an optimization problem; every answer is checked by a solver.