The amount of software written for CUDA pales in comparison to the amount that has been written for Intel x86, yet two large companies migrated off it.
The lock-in with Intel was due to binary distribution (Windows software), and binary ABIs.
Everything to do with large language models is compiled from scratch, using high level languages.
The LLM codes themselves themselves are trivial things, easily replicated in a matter of hours on some other platforms. The hard part is gathering the training data and the compute.
The hard parts are not dependent on CUDA.
Look at it this way: Google developed Transformers and trained multiple LLMs using TPUs, not CUDA GPUs! The reason their LLMs are stupid is not because of the TPUs.
x86 is a binary artifact instruction set, and software is written in higher level languages that can be recompiled. CUDA is a language that can target multiple ISAs. Microsoft and apple had incentive to keep options open to migrate off x86 and give a lot of support to their users doing so, and provide backwards compatibility in form of emulation etc. NVidia does this even better, users don't even notice when they are switching GPU ISAs.
In principle it's easy to recompile CPU side stuff too, but there are 3rd party component ecosystems, other dependency quirks, familiarity with the platform, familiar well known performance properties, sharing the same questions and answers on stackoverflow, etc. The lock-in network effect can be strong even if its individual contributing factors
I agree it's less locked in than eg Intel had it in the heyday. Ultimately we'll find out when the competition eventually shows up with reasons to consider switching.
The amount of software written for CUDA pales in comparison to the amount that has been written for Intel x86, yet two large companies migrated off it.
The lock-in with Intel was due to binary distribution (Windows software), and binary ABIs.
Everything to do with large language models is compiled from scratch, using high level languages.
The LLM codes themselves themselves are trivial things, easily replicated in a matter of hours on some other platforms. The hard part is gathering the training data and the compute.
The hard parts are not dependent on CUDA.
Look at it this way: Google developed Transformers and trained multiple LLMs using TPUs, not CUDA GPUs! The reason their LLMs are stupid is not because of the TPUs.