But that's the only thing they are good at, being smarter search engines (and that's why they should be backed by real search results, like Bing does it)
The only thing? You seem to have had a very limited exposure to what ChatGPT can do. Indeed it seems that some people have so little creativity that they can simply not think of asking these things anything except "a smarter Google" questions.
So what, by your estimation, are LLMs best for? Because they seem good for serving up relevant bits of information from vast amounts of information. Why do you think it's the worst thing they are good at?
Because it's the most basic use. In a single prompt you can have the LLM serve up relevant bits covering multiple perspectives, contrast and compare the perspectives, analyze their effectiveness in a given problem domain, and then produce meaningful output towards a solution. Information retrieval is just step 1.
Consider a prompt like the following:
"Given the task: 'TASK GOES HERE', break it down into intermediate steps or 'thoughts'. Consider multiple different reasoning paths that could be taken to solve the task. Explore these paths individually, reflecting on the possible outcomes of each. Then, consider how you might backtrack or look ahead in each path to make global decisions. Based on this analysis, develop a final to do list and complete the first course of action."
