Alternative-halfprec format only applies for conversions (ie to/from single and double prec); data-processing operations on fp16 ignore the AHP bit and always assume IEEE format. If you convert a NaN from single/double into AHP you get a zero with the same sign as the input; if you convert an infinity you get the max/min representable number; in both cases the InvalidOp fp exception bit is set.

The upshot is that it's basically an in-memory storage format, and all the actual data-processing gets done at either single or double precision.