You're not too old at all.
At the dojo, we have a lady who started when she was in her mid 40's, and she passed her shodan exam late last year. What started out as her training because her kids were in the program, ended up being a benefit for everyone, since she serves as a great source of inspiration to the others.
One of our instructors here started when he was 71 years old. He trained hard, and is still training to this date, as well as filling in as an instructor as needed, at the young age of 77. Never understimate him, since he has fantastic timing with his punches, and knows how to read people like so many books.