Reply With Quote
https://medium.com/@mourao.martins/o...g-7beb884d196e1. Development Costs
OpenAI: OpenAI’s GPT-4o model reportedly cost over $100 million to develop, involving extensive computational resources and advanced hardware like Nvidia’s H100 GPUs. The training process required millions of GPU hours, reflecting the high cost of frontier AI models .
DeepSeek: DeepSeek’s V3 model, with 671 billion parameters, was developed at a fraction of the cost—just $5.58 million. This was achieved using Nvidia’s H800 GPUs, which are less powerful but more cost-effective, and innovative techniques like Mixture-of-Experts (MoE) architecture .
2. Hardware and Resource Utilization
OpenAI: OpenAI relies on cutting-edge hardware, such as Nvidia’s H100 GPUs, which are subject to U.S. export restrictions to China. This reliance on high-performance chips significantly drives up costs .
DeepSeek: DeepSeek has optimized its hardware usage by employing Nvidia’s H800 GPUs, tailored for the Chinese market. The company also uses advanced techniques like multi-token prediction and dynamic load balancing to reduce computational requirements .
3. Training Efficiency
OpenAI: OpenAI’s models typically require months of training and millions of GPU hours. For example, Meta’s Llama 3.1, a comparable model, required 30.8 million GPU hours .
DeepSeek: DeepSeek’s V3 model was trained in just two months using 2.78 million GPU hours, showcasing remarkable efficiency. This was achieved through algorithmic innovations and hardware optimizations .
