That is a false binary. Drivers are inundated with distraction at every moment, from the rapidly changing world around them to idle thoughts arising inside their minds. A competent driver is situationally aware and chooses an appropriate moment to be intentionally distracted with a glance away to adjust the climate or radio.

The job of technology is to be extremely predictable and, ideally, multi-sensory.

as soon as you're taking your eyes off the road or your hands off the wheel to look at or touch anything else you are distracted by definition, half a second is the difference between breaking fast enough to not kill someone in an accident.

The only people who shouldn't be driving are the people not aware of that fact.

Which people obviously do all the time to glance at their nav system just as in past times they'd have done to look at a map or written directions.

You’re supposed to look in your rear view mirror far more frequently than you would glance at your climate controls.

Taking my hand off the wheel and thinking about making the climate change changes my attention and focus, even if only slightly. It's better to not even have to think about it at all.

As the other commenter said, people who don't even notice when their attention changes probably shouldn't be driving.