Math Blog #3: A Spectral Characterization of Ill-Posed Inverse Problems

In Hadamard's definition of well-posedness of Au=z, there are three requirements: (1) existences of a solution u; (2) uniqueness of that solution; and (3) continuous dependence of the solution u on z.

Let's have a look at this spectrally, by computing the singular value expansion of A:S-->T: for any u,

A*u=\sum_i {s_i*U_i*<V_i,u>},

where {s_i} are the (nonnegative) singular values of A; {V_i} are the right (orthonormal) singular functions (vectors in finite dimensions) that span the Hilbert space (vector space) S containing u; <.,.> denotes the inner-product on S (or T, where appropriate); and {U_i} are the left (orthonormal) singular functions (vectors) that span the function (vector) space T containing all possible Au and z.

Now, note that if any s_i=0, Au=U_i has no solution, so condition (1) fails. If s_i>0 and s_j=0, then Au=U_i has a solution, u=V_i/s_i, but it isn't unique since u=V_i/s_i+c*V_j is a solution for all scalars c. Hence in this case (2) fails.

Finally, if (1) and (2) hold (i.e. s_i>0 for all i), A is invertible with singular value expansion Ainv:T-->S: for any z,

Ainv*z = \sum_i {V_i*<U_i,z>/s_i}.

This mapping is discontinuous if the set of singular values {s_i} has a cluster point at 0. To see this, note that for any M>0, there exist s_i<1/M and Ainv*U_i=V_i/s_i, which has norm 1/s_i>M. Thus Ainv is discontinuous. In the finite dimensional case, Ainv is not discontinuous (since min{1/s_i}>0), but its singular values 1/s_i become extremely large, making the inverse mapping highly unstable.