You actually don't need abelian. e.g. the group of 1d affine transformations T(a,b)(x) = ax+b gives a variation of a wavelet transform. But you no longer have 1-D irreducible representations, so your Fourier coefficients become operators instead of numbers or something like that.
Apparently the left invariant measure is 1/a^2 dadb, and the right invariant measure is 1/a dadb. The book I have just sticks to left measures, and is an engineering book so it doesn't really get into details that a math book would (the whole group theory chapter is 31 pages out of a 1500 page book and the wavelet portion is ~2 pages, though it also has an 8 page section on wavelets in a previous chapter). It's very much a "what are groups, what are representations, and why do they matter for physics and signal processing" kind of thing. For reference it's Barrett & Myers Foundations of Image Science.
There is no Royal Road to mathematics. There is also no shortage of people convincing themselves they understand some particular aspect of it by watching a YouTube video.
It is rare when a mathematician, or anyone, can write a book on a topic that no other expert in the field can top. Walter Rudin did that with Fourier analysis.
