Well he didn't travel to the future in the first place. You said that he travelled 3 seconds into his past. And then you said he didn't see himself travel into the future. I'm not sure if this is correct, but I don't see how he could see himself going into the future if he didn't go into the future in the first place, he went into the past, not the future.
Imagine the scene, a person travels three seconds into his past. Then he waits three seconds and sees the man going back into the past, not the future. You said he didn't see himself travel into the future.