Your question reads like you think there's an expansion from something to somewhere else. Like, as though people were periodically allowed to run in opposite directions from a 50 yard line, and we're observing them from the 35 yard line as we slowly run in one direction or the other. But that's not what's going on, given the near-uniformity of cosmic microwave background radiation.