Azure and AWS have both offered high-bandwidth cluster options that allow scalin...

		haldujai on May 26, 2023 \| parent \| context \| favorite \| on: AI Is Catapulting Nvidia Toward the $1 Trillion Cl... Azure and AWS have both offered high-bandwidth cluster options that allow scaling beyond a single server for several months now. Pretty sure MosaicML also does this but I haven't used their offering. https://www.amazon.science/blog/scaling-to-trillion-paramete...