I don't think the universe is infinite. It has boundaries. An edge.
If you were to teleport yourself instantly to the edge of the universe you would be faced with a thin, rubbery membrane. Breaking through this membrane would cause the universe to rapidly implode, (the implosion would be similar to that of a popped balloon, I suppose, caused by the pressure differences between the inside of the balloon and the outside). Assuming that you were not consumed by this enormous implosion, you would find yourself in what I'd like to call "Dimension R". You see, I think universes are not only like balloons, but also like russian dolls. Within one universe is another, and inside that another; and so on.. The most inner universe would be "Dimension A", and all subsequent "dimensions" follow on alphabetically. All "dimensions" are expanding at exactly the same rate.
Beyond the outermost "russian doll" is the "Dining Table of the Gods" on which we are placed. We're basically just part of a finely-crafted ornament of some kind. One which is ever-expanding and a great talking piece.