The trick here is that the sexualised but not actually naked pictures of children are not actually illegal in the US (or quite possibly England as well). Just very disturbing.
It's an odd shadow war though, because the government haven't even pulled their own Twitter accounts from the service (which they can and should do).
And so can you run local models which can generate far worse material.
And horny teens have always fantasist about celebrities, or that girl they have a crush one etc. And there always had been people people cutting physical images together to place the head of their obsession on some erotic magazine sourced body.
But like you saied it's creating a feed of all the people which have been sexualized against their will.
There is a huge difference between someone doing something in your mind (or room) and it staying there and it being posted international for billions of people to see (and download, and re-post, and cherry pick preferred pictures and then feed into AI model which will actually full undress people etc.)
and a huge company making money from not just sexualizing people against their will, but also putting creating a public feed about all the people they have sexualized against their will
and then the owner going out of their way to claim that that is all free speech they won't change anything and anyone who tries is fascist, communist, evil etc.
except that definitions of what free speech means vary largely between countries and huge parts of the world have definitions where stuff like "creating sexualized images of people against their will" (or systematic harassment, cyber mobbing, death threads, and a bunch of other things) are very clearly _not_ covered by free speech.
realistically speaking this is also AI output, i.e. not speech of a person (weather natural or legal/company), i.e. it's questionably if Grok posting generated images does even count as speech (in the US and many other countries)...
It's an odd shadow war though, because the government haven't even pulled their own Twitter accounts from the service (which they can and should do).