Just because you can, doesn't mean that you should. For some things it's just better to use a rules-based engine that is always correct, rather than a heuristics based algorithm that gives answers that are merely close.
I don't think the author of the piece (or anyone for that matter) thinks GPT-3 should be used for running programs or evaluating functions.
It is being discussed because it is surprising that GPT-3 can do it at all. It is worth investigating what types of emergent knowledge and behavior are encoded in the trained network, as the boundaries of its capabilities may help illuminate future neural network architecture design.
