Yes. However, we only care about compressing natural images and they are not a big subset of the space of all possible images. In practice, we find that neural networks are quite good at making predictions on pixel values, especially when we frame the problem in terms of super-resolution.
Isn't the quality of the prediction heavily influenced by how common the encoded content is?