Yes indeed (and, even more so, the culture of non-corruption which reformers tried to put in place at the same time). But, like in the chemical plant, it is possible to run a government with minimal levels of corruption and many (better-governed) countries do it.
It's not just a public admin problem, either - corruption and self-dealing are potentially endemic in the private sector, too. The concern has traditionally been higher for the public sector, both because of the potential for abuse of state power and because it is/was seen to be corrosive to public trust and self-government. A private company is more likely to trade off the cost of controls versus the dollar value of the future loss (or the controller is the one running the firm strictly for their own benefit, in which case they're likely to get away with it without intervention by the legal system). And the largest companies have had actual public power greater than the smallest countries for centuries by this point, so the two concerns are not completely disjoint.
It's not just a public admin problem, either - corruption and self-dealing are potentially endemic in the private sector, too. The concern has traditionally been higher for the public sector, both because of the potential for abuse of state power and because it is/was seen to be corrosive to public trust and self-government. A private company is more likely to trade off the cost of controls versus the dollar value of the future loss (or the controller is the one running the firm strictly for their own benefit, in which case they're likely to get away with it without intervention by the legal system). And the largest companies have had actual public power greater than the smallest countries for centuries by this point, so the two concerns are not completely disjoint.