It is simply because class divide used to be a lot wider. We perceive things as going down only by comparison with unhealthily equal 1960s society brought about by 91% top marginal tax rate of the day. Pre-WWI world was extremely unequal and it was seen as normal. Anyone from the middle class could hire a servant.
In fact, even in the Soviet Union, engineers and even highly qualified workers (!) had full-time domestic servants in 1930s. Not after the war though.
it's worth noting that even though the top marginal rate today is only 37%, tax as % of GDP stayed mostly the same since the 1960s. The difference can be explained by the fact that the 91% tax rate was accompanied by a plethora of deductions, which reduced the effective tax rate.
In fact, even in the Soviet Union, engineers and even highly qualified workers (!) had full-time domestic servants in 1930s. Not after the war though.