That's literally true but that's inefficient. The question is whether it's more inefficient than whatever you try to do to solve it and the risk it does more harm than good.
By the literal definition of what an "externality" is.
A company cutting costs by dumping their PFAS into the local river isn't being "paid for" by anyone. Except with our health and all the dead insects and fish and compounding environmental effects brought with it
By definition an externality is a "market failure" because the market completely fails to account for this effect. There are "positive externalities" as well, but that's obviously not relevant here