The best time to plant a tree was 20 years ago. The second best time is now.
If it's still in your mind, it is worth taking the risk.