[ad_1]
Nvidia | Benj Edwards
On Monday, Nvidia announced the HGX H200 Tensor Core GPU, which makes use of the Hopper structure to speed up AI purposes. It is a follow-up of the H100 GPU, launched final yr and beforehand Nvidia’s strongest AI GPU chip. If broadly deployed, it may result in much more highly effective AI fashions—and quicker response instances for current ones like ChatGPT—within the close to future.
In response to consultants, lack of computing energy (usually referred to as “compute”) has been a major bottleneck of AI progress this previous yr, hindering deployments of current AI fashions and slowing the event of latest ones. Shortages of highly effective GPUs that speed up AI fashions are largely accountable. One option to alleviate the compute bottleneck is to make extra chips, however you may as well make AI chips extra highly effective. That second method might make the H200 a sexy product for cloud suppliers.
What is the H200 good for? Regardless of the “G” within the “GPU” title, knowledge heart GPUs like this sometimes aren’t for graphics. GPUs are perfect for AI purposes as a result of they carry out huge numbers of parallel matrix multiplications, that are needed for neural networks to perform. They’re important within the coaching portion of constructing an AI mannequin and the “inference” portion, the place individuals feed inputs into an AI mannequin and it returns outcomes.
“To create intelligence with generative AI and HPC purposes, huge quantities of information should be effectively processed at excessive pace utilizing giant, quick GPU reminiscence,” mentioned Ian Buck, vp of hyperscale and HPC at Nvidia in a information launch. “With Nvidia H200, the business’s main end-to-end AI supercomputing platform simply acquired quicker to resolve a number of the world’s most vital challenges.”
For instance, OpenAI has repeatedly said it is low on GPU assets, and that causes slowdowns with ChatGPT. The corporate should depend on price limiting to supply any service in any respect. Hypothetically, utilizing the H200 may give the present AI language fashions that run ChatGPT extra respiration room to serve extra clients.
4.8 terabytes/second of bandwidth
Nvidia
In response to Nvidia, the H200 is the primary GPU to supply HBM3e reminiscence. Because of HBM3e, the H200 provides 141GB of reminiscence and 4.8 terabytes per second bandwidth, which Nvidia says is 2.4 instances the reminiscence bandwidth of the Nvidia A100 launched in 2020. (Regardless of the A100’s age, it is nonetheless in excessive demand resulting from shortages of extra highly effective chips.)
Nvidia will make the H200 obtainable in a number of type components. This consists of Nvidia HGX H200 server boards in four- and eight-way configurations, suitable with each {hardware} and software program of HGX H100 techniques. It would even be obtainable within the Nvidia GH200 Grace Hopper Superchip, which mixes a CPU and GPU into one bundle for much more AI oomph (that is a technical time period).
Amazon Internet Providers, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure would be the first cloud service suppliers to deploy H200-based cases beginning subsequent yr, and Nvidia says the H200 will likely be obtainable “from world system producers and cloud service suppliers” beginning in Q2 2024.
In the meantime, Nvidia has been taking part in a cat-and-mouse game with the US authorities over export restrictions for its highly effective GPUs that restrict gross sales to China. Final yr, the US Division of Commerce introduced restrictions supposed to “preserve superior applied sciences out of the flawed arms” like China and Russia. Nvidia responded by creating new chips to get around these limitations, however the US just lately banned those, too.
Final week, Reuters reported that Nvidia is at it once more, introducing three new scaled-back AI chips (the HGX H20, L20 PCIe, and L2 PCIe) for the Chinese language market, which represents 1 / 4 of Nvidia’s knowledge heart chip income. Two of the chips fall under US restrictions, and a 3rd is in a “grey zone” that is likely to be permissible with a license. Anticipate to see extra back-and-forth strikes between the US and Nvidia within the months forward.
[ad_2]
Source
