All comes to the same end: random guy, jealous colleague, nosy employer or landlord snatches your phone while you're not looking and it only takes a few minutes to upload a very thorough archive of "places stephen_g's iPhone has been".
They can get a very rough idea of the places you've been. But if they've stolen your phone, getting access to a list of cell towers you've been near isn't very much of a breach compared to all of your email, bookmarks, facebook, twitter, all of your contacts, etc. The whole thing is a very silly argument and Apple has already addressed it; so lets stop spreading FUD, please.