Principal component as an optimization problem: an eigenvalue problem example, solved and checked
The principal component as an optimization problem is a classic eigenvalue problem problem. Here it is solved by a real solver (eigenvalue decomposition (NumPy)) and checked: every constraint, and the optimality certificate. Change the numbers below and solve your own.
Solve your own version free, no account needed
The problem
Two features have covariance matrix [[4, 2], [2, 3]]. Find the direction (unit vector) of greatest variance and the variance along it.
How the model was built
The objective as written from the problem, before the numbers are multiplied out:
- The variance of the data along a unit direction w is wT S w, with S the covariance matrix: 4 w12 + 2 × 2 w1 w2 + 3 w22.
- w12 + w22 = 1 keeps w a unit vector; without it the variance could grow without limit.
- The best value is the largest eigenvalue of S, and w is its eigenvector (the spectral theorem). That is how it was solved, so the answer is the global best.
The model
Each line has a plain-language note on the right.
| make this as large as possible | |
| limit 1: left side must equal 1 |
What is being decided
There are 2 decision variables.
| Variable | Type | Meaning |
|---|---|---|
| w1 | real number | component of the direction vector on feature 1 (unitless) |
| w2 | real number | component of the direction vector on feature 2 (unitless) |
Goal and limits
Maximize the objective subject to 1 constraint.
Problem type
Eigenvalue problem, complexity level 4: a quadratic form on a sphere: by the spectral theorem the optimum is an extreme eigenvalue. Solved with eigenvalue decomposition (NumPy) at the Medium solver tier.
Picture
Shaded: every point that satisfies all the constraints. Dot: the optimum. Dashed: the objective's level line through it.
The answer at a glance
The optimal value of each decision, as the solver returned it.
Which constraints hold the answer in place
Binding (no slack left): (1) w1^2 + w2^2 = 1. Changing a binding constraint changes the answer; a slack one can move a little without effect.
| # | Constraint | Value | Limit | Slack | Binding |
|---|---|---|---|---|---|
| 1 | w1^2 + w2^2 = 1 | 1 | 1 | 0 | yes |
Algebraic model (low level)
Decision variables, objective and every constraint written out; this is exactly what the solver receives.
Model code (cvxpy)
The same model in cvxpy, ready to run.
import cvxpy as cp
w1 = cp.Variable(name='w1') # component of the direction vector on feature 1 (unitless)
w2 = cp.Variable(name='w2') # component of the direction vector on feature 2 (unitless)
objective = cp.Maximize(4*w1**2 + 4*w1*w2 + 3*w2**2)
constraints = [
w1**2 + w2**2 == 1,
]
problem = cp.Problem(objective, constraints)
problem.solve(solver='eigenvalue')
print(problem.status, problem.value)
print('w1 =', w1.value)
print('w2 =', w2.value)Solve your own
This is the model as the app reads it. Change any number and solve it in the free box on the front page (no account needed).
maximize 4*w1^2 + 4*w1*w2 + 3*w2^2
subject to
w1^2 + w2^2 = 1
# w1 = component of the direction vector on feature 1 (unitless)
# w2 = component of the direction vector on feature 2 (unitless)
# asked: the unit direction (w1, w2) of greatest variance and the variance along it (the objective value)Checked: the app's answer matches the reference answer in our test set, which two solvers (CLARABEL, SCS) agree on. Built 2026-11-10.
Related examples
Try ProvenDecisions free: type, photograph or say an optimization problem; every answer is checked by a solver.