I tried replying to your article. Since I now notice it's been six years, I wonder whether you did anything with this?
I don't think professional mathematicians would say you shouldn't pursue it. The opposite, actually. If people like myself who never practiced it in their own time can get degrees in it, so could you. You don't need to be any kind of genius.
I also have similar questions. What did you end up doing about this feeling? I don't believe my own intelligence was (or is) on the same caliber as yours, but I do feel like it has atrophied somewhat since joining the working world, and that is in spite of doing a CS masters. Honestly I believe if I hadn't done the masters I would've gone crazy from the deterioration.
I don't think professional mathematicians would say you shouldn't pursue it. The opposite, actually. If people like myself who never practiced it in their own time can get degrees in it, so could you. You don't need to be any kind of genius.