Amazon releases TorchNeuron tool to move AI apps onto its Trainium chips

Amazon Web Services (AWS), Amazon's cloud computing arm, on Monday, October 19, released a tool called TorchNeuron. It lets AI startups easily move their artificial intelligence apps onto Trainium, the AI chip Amazon designs. The step targets the software advantage that has helped Nvidia become the default hardware choice for AI.

TorchNeuron works with PyTorch, a free, open-source toolkit developers use to build, train and run AI models. Amazon says about 85% of AI apps are written with PyTorch. Facebook originally developed it, but since 2022 it has been part of the Linux Foundation, which maintains open-source projects. According to the company, running a PyTorch app on Trainium requires changing one line of configuration, with no code rewrite.

Nvidia's advantage

For years Nvidia has made sure its CUDA software, the infrastructure that lets apps run on its GPUs, works seamlessly with PyTorch. As a result, its hardware is embedded in the toolkit almost by default. Amazon is not the first to try to ease the switch: PyTorch users can also port apps to AMD GPUs or Apple chips.

Amazon stresses that Trainium also gives startups access to its whole cloud. AWS argues this beats so-called neoclouds, many tied to Nvidia, such as Coreweave, Nebius and Firmus. Colin Brace, VP of Technology Software at AWS Annapurna Labs, the unit that makes Amazon's custom chips, said the same chip handles both training and inference, which improves the economics even if AWS does not offer the cheapest per-token price. He cited voice AI provider Deepgram, which he said now processes 70% more audio per dollar.

Impact on PyTorch

Amazon also plans to contribute changes to the main PyTorch project to remove code that ties the framework to specific chips, making it more hardware-agnostic. Whether this gives developers more choice and cheaper options, as Amazon promises, or just more configuration headaches remains to be seen.