I dislike Google's (mis)management of Assistant as much as the next guy, but this just has not been my experience. I can tell Gemini on my phone to set timers and it works just fine.