AWS Trainium and torchtitan Collaboration: Enabling Qwen3 Model Training and Optimization

Summary:AWS has published a technical blog detailing how to train the Qwen3 model on AWS Trainium using torchtitan. The blog covers three training methods: using the Hugging Face Transformers v5 modeling backend, torchtitan's native Qwen3 implementation, and optimizing with Neuron Kernel Interface (NKI) mixture-of-experts (MoE) kernels. It provides an in-depth exploration of training workflows, performance optimization techniques, and the integration of custom kernels, offering AI developers a practical

---

Integration of AWS Trainium and torchtitan

AWS Trainium registers as a native PyTorch device through TorchNeuron, enabling torchtitan to leverage it via standard PyTorch interfaces. This article introduces three methods for training the Qwen3 model on a single trn2.48xlarge instance:

1. Hugging Face Transformers v5 Modeling Backend

   • Utilizes AutoModelForCausalLM to load Hub models and reads/writes published safetensors directly.
   • The configuration example demonstrates setting the model ID, dataset, training steps, and parallelism parameters.
   • Launches training processes with torchrun and employs data parallelism to distribute the model across 64 logical NeuronCores.
   • Provides detailed steps for loading and saving Hugging Face safetensors, ensuring seamless integration with standard Hugging Face workflows.

2. torchtitan's Native Qwen3 Implementation

   • Switches to torchtitan's native Qwen3 model by changing config.model_spec.
   • Uses model_registry to select the 4B scale Qwen3 model and configures training parameters such as global batch size and parallelism strategy.
   • This method showcases how to directly invoke the native model in torchtitan and leverage its optimization mechanisms for improved training efficiency.

3. Optimization with NKI MoE Kernels

   • Adds NKI MoE kernels from torchtitan-neuron recipes and attaches custom kernels via the override registry.
   • The configuration example shows how to set the number of experts, compilation mode, and parallelism strategy.
   • This method optimizes the MoE layers with NKI kernels and supports running in both eager mode and with torch.compile.

Performance Optimization and Custom Kernel Integration

• Custom NKI Kernel Integration: Developers can write custom NKI kernels to optimize specific operations. For example, the article demonstrates how to integrate the rope_hf kernel and customize tensor layout rearrangement and output tensor allocation.
• Performance Validation: Before training begins, torchtitan validates the input constraints of custom kernels to ensure compatibility with the model architecture.
• Multi-level Parallelism Strategies: By setting data_parallel_shard_degree and tensor_parallel_shard_degree, developers can flexibly control data and tensor parallelism strategies to accommodate different model sizes and hardware configurations.

Developer Recommendations

• Version Control: It is recommended to pin the versions of all components before training to ensure configuration consistency and reproducibility.
• Resource Management: As the model size or batch size increases, closely monitor per-rank HBM usage to avoid memory issues.
• Custom Kernel Testing: Before training, perform numerical tests on custom kernels to ensure their outputs match the original implementation.

Conclusion

The combination of AWS Trainium and torchtitan provides AI developers with an efficient and scalable training solution. By supporting multiple training methods and custom kernels, developers can optimize the training workflow for the Qwen3 model and achieve high-performance AI model training on AWS Trainium.

---

Source: AWS Machine Learning Blog (2026-10-21)

— END —

Tags: #AWS Follow #Trainium Follow #torchtitan Follow #Qwen3 Follow #MoE Follow