GPT-3 always gives an output after a certain amount of time. What should GPT-3 return when running:

  print("A");
  potentiallyHalts();
  print("B");

Potentially time out? I don’t see the difference to, say, a python interpreter with a timeout. What would a human do, are we not Turing complete?

I mean, in the strictest sense that isn’t Turing complete either, because when you have a timeout you cannot run every program a theoretical Turing machine could. But then no practical computer is, because resources are always constrained (e.g. finite memory instead of an infinite tape). So when we talk about something being Turing complete, we usually disregard the resource limitations and effectively substitute something like “we mean Turing complete in the sense that it would be if we also had infinite memory and time”.

So, I still don’t understand why GPT-3 would have to (impossibly) solve the halting problem to be Turing complete[1], but everything else including a python interpreter or lambda calculus doesn’t.

[1] Note that I don’t assert that GPT-3 could or could not be Turing complete, I just don’t know why the halting problem predicates that.

Probably easier to just observe that, if GPT-3 isn't reliably correct, then it's not consistent enough to simulate a Turing-machine and therefore isn't Turing-complete.

As for loops: a Turing-machine could do infinitely many loops, so Turing-completeness implies that a system can do the same. If GPT-3 can't do infinitely many loops, it's not strictly Turing-complete; and if it can't do many loops, then it wouldn't seem like a meaningful approximation of a Turing-complete system.

Yeah, that I agree with.