As another commenter mentioned, this is all done modulo some large prime (or, apparently, in some cases, a carefully-chosen composite number with a large prime factor), so 2^y^x=B^x=R=A^y=2^x^y mod p. (Note that mathematicians use "mod" in a notation that is sometimes surprising to programmers. "x = y mod p" means that _both_ x and y are reduced mod p.) The security of this hinges on the difficulty of the "discrete logarithm problem" -- that is, given R = 2^a mod p, it is computationally intractable to find a = log_2(R) mod p (with appropriate parameter choices.)