If time travel would be possible within single Universe, we could travel to our past, then our atoms would be in two places at the same time. Then we and our own copy could enter time machine again, and travel again to our past, and there would be 4 "clones" of our atom, etc. etc. Repeat it infinite number of time, and Universe mass and energy would go to infinity.. It would violate the all known conservations of energy, mass, momentum, baryon and leptons..
Please note that atoms used for our body now, exist in nature all the time, they are in air, ground, other animals, or even Sun, or other stars, at any time (f.e. some proton 1 milion years ago was emitted by star, traveled through whole galaxy, and ended up on Earth, then joined with some O-H ion forming H2O, and ended up in our body that started traveling in time), so we don't need literally meet our own clone. Our atoms during time travel to past might be anywhere in the planet or anywhere in galaxy/Universe at that time.
If time travel would be possible in multi-universe, one Universe would be loosing energy-mass, and other Universe would be increasing its energy-mass.
It would violate conservations as well. Matter would disappear and appear from nowhere in space from point of view of single Universe.