Because at that time the person is in past. When does his time-travel, a paradox is created and hence, he is not able to see himself at an different instance, because at that moment, he was in past, thus sort of creating a sequence where a similar frame or sequence repeats itself.
And highly knowledgeable and esteemed critics (to this thread), it is an question. When Einstein created his theory of relativity, he himself was doubtful of its existence, and yes, people did question him, but today, he is a superstar. So who knows, when the above may be proved or be possible.