It's unclear to me why human drivers should be considered unable to adapt to more "tentative, wary" drivers on the roads. After all, humans vary quite a bit in how they drive. There are plenty of cautious drivers already out there.
They do adapt; it's just slightly less safe than being among a set of vehicles behaving more similarly. Same thing with a particularly tentative human driver. (Or a particularly aggressive one, of course.)