The TL;DR of this seems to be: the plaintext metadata accompanying ciphertext ("associated data") is mixed into the ciphertext's encryption (essentially as an initial vector). Thereby, if the plain-text data is altered, the ciphertext cannot be correctly decrypted. The ciphertext is both a secret message, and a signature of the unencrypted data, so a separate HMAC is not required.
We can imagine, e.g. in the context of e-mail, if the DKIM header signature were combined a PGP-encrypted body as one operation. I'm ducking under the table now, though.
The core idea, one that PGP does not "get" (except in newer, non-compatible implementations) is simply that of ciphertext authentication. Once you have authentication, the Associated Data construction is pretty easy to get; put differently, almost immediately after we "had" widespread authenticated encryption, we had AEAD. PGP finds these problems difficult to solve (and so simply doesn't solve them) because it confuses error-detecting integrity checks, signatures, encryption, and message authentication, which are 4 different things. But PGP also predates our modern understanding of the differences between those 4 things.
I looked through that RFC 5XXX that describes AEAD. I somehow feel that there is nothing earth-shattering in it that would confuse cryptographers and crypto coders, if we sent it back to 1993.
Oh, wow, is this ever not true. If you want to blow your head off, go read the IPSEC working group's confrontation with Phil Rogaway over this issue; Rogaway went and canvassed other real cryptographers (notably Rivest) to try to talk sense into them, and failed.
We can imagine, e.g. in the context of e-mail, if the DKIM header signature were combined a PGP-encrypted body as one operation. I'm ducking under the table now, though.