That sounds about right as to our physical and mental destiny. Our demise is one of the great challenges of living and it can take decades to be at peace with it. It's one of those subjects that you need probably at least 30 years of living to start getting your head around it. Everybody has to find their own answers. I think the main goal to aim for is personal fulfillment because this helps to make you feel more complete as a person and if you get that far then there should be less feeling of things not done and any time left is a bonus. From a personal perspective, as fifty-something, I only want to go around the block so many times and i'll happily hang up my clogs.
Every day, your actions and words are affecting people, and sometimes modifying their behaviour for the better through knowing you. Those effects may be subliminal and each one small but the memory of your presence is embedded in their modified behaviour, so in that sense, looking at the bigger picture, your memory lives on. The energy redirected by your actions during your life still continues. You may ultimately be forgotten by people but you will not be forgotten by the universe because the waves you created will ripple far into the future.