There's an important caveat with this: It does not work cross-origin, and there is no way to make it work except for including some JS like svg4everybody. CORS does not apply.
It works out of the box with most browsers and like you said there are polyfills for older browsers. BTW the syntax with svg 2.0 is just href.
About the cross-origin issue, you mean it lacks something like a crossorigin attribute?
That's only an issue if you plan to load svgs from other websites, I don't think it's a very common usecase. If you use them at multiples places in your website, you should probably have them served from your sever anyway, if not then I guess caching is less of an issue.
It pretty much solves the cache issue to me.