Biologically speaking from a cellular level. Funny thing is his 'rival' Jay Leno will turn 103 from this perspective.
To review, the first cell that becomes a new human appears within the womb of the womb of the maternal grandmother. Jay Leno's mother is was born in 1911, while David Letterman's mom was born in 1921.
By conventional means David Letterman is older than Jay Leno by three years. Dave was born in 1947 but Jay was born in 1950. But Jay's egg formed 10 years before Dave. Wierd but true!