A theory for optimal regularization in the finite dimensional case

Document Type


Publication Date



Consider the matrix problem Ax = y + ε = y ̃ in the case where A is known precisely, the problem is ill conditioned, and ε is a random noise vector. Compute regularized "ridge" estimates, x ̃λ = (A*A + λI)-1 A* y ̃,where * denotes matrix transpose. Of great concern is the determination of the value of λ for which x̃λ "best" approximates x0 = A + y. Let Q = {norm of matrix} x ̃ λ - x0{norm of matrix}2,and define λ0 to be the value of λ for which Q is a minimum. We look for λ0 among solutions of dQ/dλ = 0. Though Q is not computable (since ε is unknown), we can use this approach to study the behavior of λ0 as a function of y and ε. Theorems involving "noise to signal ratios" determine when λ0 exists and define the cases λ0 > 0 and λ0 = ∞. Estimates for λ0 and the minimum square error Q0 = Q(λ0) are derived. © 1982.

Publication Title

Linear Algebra and Its Applications