The use of alternate realities in Fringe could potentially open up a Pandora's box of infinite universes. This could lead to the "Heroes" problem where, among other things, no one really dies because you can always go back in time and rescue them (or in Fringe - another multiverse.)
"Lost" potentially had similar problem with its healing island and time travel, but they were prescient enough to create two paradox busting rules: when you die on Lost you are dead, and you can't change what happens in the past because the time-line will self correct.
Fortunately, similar "rules" have been established for Fringe to keep the show easier to follow and solve some of the side effects of inter-dimensional travel.
Fringe executive producer and "showrunner" Jeff Pinkner has stated this in many interviews - There are only TWO alternate realities in Fringe: Shape-shifting soldiers, and alive Peter come from the Alternate Reality, and as far as we know Olivia, Walter, dead Peter, etc come form Our Reality. William Bell is from our reality, but is living (hiding out?) in the alternate reality.
#2 - "There is a one-to-one time correlation, and also geography. If you are in NY in the alternate reality, you are in NY here."
We have seen this several times - in Broyles' office, the burning Boston, the two charred bodies. It only became confusing when Olivia teleported to the Twin Towers, which was initially though to be her crossing over and changing location at the same time. It turns out the elevator must have been just a teleportation device, because Olivia was already in the alternate universe.
Pinker explained this in a recent TV Guide quote. However, last season may have been a little bit of a trick - even though we knew eventually that Olivia crossed over, it was only recently revealed that it happened during the car crash, and not in the elevator.
BTW, If you need a refresher course on Multiverse theory, here's a funny explanation from Stewie of Family Guy: