Eigenvalues with Rayleigh quotient and locally optimized gradient descent

Introduction

In mathematics the Rayleigh quotient for a given real symmetric matrix H and a non-zero vector v is defined as

R(H,v)=(vTHv)/(vTv) .

It can be shown that the Rayleigh quotient reaches its minimum value λ_min—the smallest eigenvalue of H—when v equals v_min (the corresponding eigenvector),

∀v≠0 R(H,v)≥λmin, R(H,vmin)=λmin if Hvmin=λminvmin.

Analogously,

∀v≠0 R(H,v)≤λmax, R(H,vmax)=λmax if Hvmax=λmaxvmax.

In quantum mechanics this is called Variational Method.

Project

Implement a function that calculates the lowest eigenvalue and the corresponding eigenvector of a given symmetric matrix H by minimizing its Rayleigh quotient R(H,v) using the components of the vector v as free parameters. The function should use newton's method with analytic gradient and optimized linesearch.

The gradient ∂R/∂v_k is given as

∂R/∂vT=2[Hv-Rv]/(vTv) ∝ (H-R)v .

Therefore one can search for the optimal Newton's step as -αΔv, where

Δv = (H-R)v ,

and where α is to be found by minimizing R(v-αΔv) which is given as

R(v-αΔv) = (v-αΔv)TH(v-αΔv)/(v-αΔv)T(v-αΔv)
         = (vTHv-2αΔvTHv+α²ΔvTHΔv)/(vTv+α²ΔvTΔv) .

This is a one-dimensional minimization that can be done numerically precisely with you own numerical minimizer.

Hint: you might need to keep the norm of your v-vector close to unity.

Investigate the scaling of this method (time to find the eigenvalue as function of the size of the marix) and find out how far in the size of the matrix you can realistically go (that is, within few seconds of calculation time).

Calculate the ground state of the Hydrogen atom.

Calculate also the second smallest eigenvalue.