In theory, I agree. In practice, I've realized that most of the jailbreaking phones/tablets are used to download illegal applications and/or without paying. I know this is a somewhat contrived statement, but this is from what I've seen in company/university/engineering departments I've worked/studied.
Note:
I myself had to jailbreak my iphone to install spotify (and asked a friend to pay for it since he's outside of Canada). For some reasons, spotify is illegal here. (Yes, I know, I illegally downloaded an illegal application to legally download/listen to music).
I used a jailbroken iPhone for 2 years, I never once pirated an application, I still purchased plenty of apps from Apple's dumb app store and a few from Cydia. Mostly, I just used it to "background" apps before that was a built-in feature.
Very few of the applications available on Cydia are "illegal", they're just banned from the App Store by Apple. Similarly, there's nothing illegal about linux, but you'll have to jailbreak your ps3 to install it.
Even if 100% of the uses were illegal, this is an issue of principal. When I buy a gadget, I didn't lease it, I bought it. It is mine. As such, I shouldn't have to worry I might go to jail (!!) for modifying my device in a way that doesn't impact anybody else.
Note:
I myself had to jailbreak my iphone to install spotify (and asked a friend to pay for it since he's outside of Canada). For some reasons, spotify is illegal here. (Yes, I know, I illegally downloaded an illegal application to legally download/listen to music).