A Theory for Optimal Regularization in the Finite Dimensional Case

Document Type

Article

Publication Date

1982

Department

Department of Mathematical Sciences

Abstract

Consider the matrix problem Ax = y + ε = y ̃ in the case where A is known precisely, the problem is ill conditioned, and ε is a random noise vector. Compute regularized "ridge" estimates, x ̃λ = (A*A + λI)-1 A* y ̃,where * denotes matrix transpose. Of great concern is the determination of the value of λ for which x̃λ "best" approximates x0 = A + y. Let Q = II x̃λ - x0 II2,and define λ0 to be the value of λ for which Q is a minimum. We look for λ0 among solutions of dQ/dλ = 0. Though Q is not computable (since ε is unknown), we can use this approach to study the behavior of λ0 as a function of y and ε. Theorems involving "noise to signal ratios" determine when λ0 exists and define the cases λ0 > 0 and λ0 = ∞. Estimates for λ0 and the minimum square error Q0 = Q(λ0) are derived.

Publisher's Statement

© 1982

Publication Title

Linear Algebra and Its Applications

Link to Full Text

Share

COinS