You certainly could make your way around the ground if you were prepared to hop over a lowish wall in each corner, Spurs fans were well known for having a group making their way around the ground in single file, occasionally skirmishing with any Saints fan who objected.
Another example, in early 1970s I was standing in the corner of Milton Rd and loads of hard core looking Wolves fans were pushing past to get over a wall into lower East Stand, we thought we were in trouble but one of them said "excuse us lads" (we were only young boys!).