{"id":24122,"date":"2023-11-13T17:10:11","date_gmt":"2023-11-13T17:10:11","guid":{"rendered":"http:\/\/107332253"},"modified":"2023-11-13T17:10:11","modified_gmt":"2023-11-13T17:10:11","slug":"nvidia-unveils-h200-its-newest-high-end-chip-for-training-ai-models","status":"publish","type":"post","link":"https:\/\/wp.worldtechguide.net\/nvidia-unveils-h200-its-newest-high-end-chip-for-training-ai-models\/","title":{"rendered":"Nvidia unveils H200, its newest high-end chip for training AI models"},"content":{"rendered":"<p><span class=\"HighlightShare-hidden\"><\/span><\/p>\n<div class=\"InlineImage-imageEmbed\" id=\"ArticleBody-InlineImage-107327179\" data-test=\"InlineImage\">\n<div class=\"InlineImage-wrapper\">\n<div>\n<p>Jensen Huang, president of Nvidia, holding the Grace hopper superchip CPU used for generative AI at the Supermicro keynote presentation during Computex 2023.<\/p>\n<p>Walid Berrazeg | Lightrocket | Getty Images<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"group\">\n<p><span class=\"QuoteInBody-quoteNameContainer\" data-test=\"QuoteInBody\" id=\"SpecialReportArticle-QuoteInBody-1\"><span class=\"waslinkname\">Nvidia<\/span><span class=\"QuoteInBody-inlineButton\"><span class=\"AddToWatchlistButton-watchlistContainer\" id=\"-WatchlistDropdown\" data-analytics-id=\"-WatchlistDropdown\"><button class=\"AddToWatchlistButton-watchlistButton\" aria-label=\"Add To Watchlist\" data-testid=\"dropdown-btn\"><span class=\"AddToWatchlistButton-addWatchListFromTag\"><\/span><\/button><\/span><\/span><\/span> on Monday unveiled the H200, a graphics processing unit designed for training and deploying the kinds of artificial intelligence models that are powering the generative AI boom.<\/p>\n<p>The new GPU is an upgrade from the H100, the chip OpenAI used to train its most advanced large language model, GPT-4. <span class=\"waslinkname\">Big companies<\/span>, <span class=\"waslinkname\">startups<\/span> and <span class=\"waslinkname\">government<\/span> agencies are all vying for a limited supply of the chips.<\/p>\n<p>H100 chips cost between $25,000 and $40,000, according to an estimate from Raymond James, and thousands of them working together are needed to create the biggest models in a process called &#8220;training.&#8221;<\/p>\n<p>Excitement over Nvidia&#8217;s AI GPUs has <span class=\"waslinkname\">supercharged<\/span> the company&#8217;s stock, which is up more than 230% so far in 2023. Nvidia <span class=\"waslinkname\">expects<\/span> around $16 billion of revenue for its fiscal third quarter, up 170% from a year ago.<\/p>\n<p>The key improvement with the H200 is that it includes 141GB of next-generation &#8220;HBM3&#8221; memory that will help the chip perform &#8220;inference,&#8221; or using a large model after it&#8217;s trained to generate text, images or predictions.<\/p>\n<p>Nvidia said the H200 will generate output nearly twice as fast as the H100. That&#8217;s based on a test using <span class=\"QuoteInBody-quoteNameContainer\" data-test=\"QuoteInBody\" id=\"SpecialReportArticle-QuoteInBody-7\"><span class=\"waslinkname\">Meta&#8217;s<\/span><span class=\"QuoteInBody-inlineButton\"><span class=\"AddToWatchlistButton-watchlistContainer\" id=\"-WatchlistDropdown\" data-analytics-id=\"-WatchlistDropdown\"><button class=\"AddToWatchlistButton-watchlistButton\" aria-label=\"Add To Watchlist\" data-testid=\"dropdown-btn\"><span class=\"AddToWatchlistButton-addWatchListFromTag\"><\/span><\/button><\/span><\/span><\/span> Llama 2 LLM.<\/p>\n<p>The H200, which is expected to ship in the second quarter of 2024, will compete with <span class=\"waslinkname\">AMD&#8217;s MI300X GPU<\/span>. <span class=\"QuoteInBody-quoteNameContainer\" data-test=\"QuoteInBody\" id=\"SpecialReportArticle-QuoteInBody-9\"><span class=\"waslinkname\">AMD&#8217;s<\/span><span class=\"QuoteInBody-inlineButton\"><span class=\"AddToWatchlistButton-watchlistContainer\" id=\"-WatchlistDropdown\" data-analytics-id=\"-WatchlistDropdown\"><button class=\"AddToWatchlistButton-watchlistButton\" aria-label=\"Add To Watchlist\" data-testid=\"dropdown-btn\"><span class=\"AddToWatchlistButton-addWatchListFromTag\"><\/span><\/button><\/span><\/span><\/span> chip, similar to the H200, has additional memory over its predecessors, which helps fit big models on the hardware to run inference.<\/p>\n<\/div>\n<div class=\"InlineImage-imageEmbed\" id=\"ArticleBody-InlineImage-107332323\" data-test=\"InlineImage\">\n<div class=\"InlineImage-wrapper\">\n<div>\n<p>Nvidia H200 chips in an eight-GPU Nvidia HGX system.<\/p>\n<p>Nvidia<\/p>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"group\">\n<p>Nvidia said the H200 will be compatible with the H100, meaning that AI companies who are already training with the prior model won&#8217;t need to change their server systems or software to use the new version.<\/p>\n<p>Nvidia says it will be available in four-GPU or eight-GPU server configurations on the company&#8217;s HGX complete systems, as well as in a chip called GH200, which pairs the H200 GPU with an <span class=\"waslinkname\">Arm<\/span>-based processor.<\/p>\n<p>However, the H200 may not hold the crown of the fastest Nvidia AI chip for long.<\/p>\n<p>While companies like Nvidia offer many different configurations of their chips, new semiconductors often take a big step forward about every two years, when manufacturers move to a different architecture that unlocks more significant performance gains than adding memory or other smaller optimizations. Both the H100 and H200 are based on Nvidia&#8217;s Hopper architecture.<\/p>\n<p>In October, Nvidia told investors that it would move from a two-year architecture cadence to a one-year release pattern due to high demand for its GPUs. The company displayed <span class=\"waslinkname\">a slide suggesting<\/span> it will announce and release its B100 chip, based on the forthcoming Blackwell architecture, in 2024.<\/p>\n<p><strong>WATCH:<\/strong> <span class=\"waslinkname\">We&#8217;re a big believer in the AI trend going into next year<\/span><\/p>\n<\/div>\n<div role=\"region\" aria-labelledby=\"Placeholder-ArticleBody-Video-107326807\">\n<div role=\"button\" id=\"Placeholder-ArticleBody-Video-107326807\" class=\"PlaceHolder-wrapper\" data-vilynx-id=\"7000321058\" data-test=\"VideoPlaceHolder\">\n<div class=\"InlineVideo-videoEmbed\" id=\"InlineVideo-0\" data-test=\"InlineVideo\">\n<div class=\"InlineVideo-wrapper\">\n<div class=\"InlineVideo-inlineThumbnailContainer\"><img decoding=\"async\" class=\"InlineVideo-videoThumbnail\" src=\"https:\/\/wp.worldtechguide.net\/wp-content\/uploads\/2023\/11\/nvidia-unveils-h200-its-newest-high-end-chip-for-training-ai-models.jpg\" alt=\"We're a big believer in the A.I. trend going into next year, says Wolfe Research's Chris Caso\"><span class=\"InlineVideo-videoButton\"><\/span><span><\/span><\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"group\">\n<p><em><strong>Don&#8217;t miss these stories from CNBC PRO:<\/strong><\/em><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Jensen Huang, president of Nvidia, holding the Grace hopper superchip CPU used for generative AI at the Supermicro keynote presentation during Computex 2023. Walid Berrazeg | Lightrocket | Getty Images Nvidia on Monday unveiled the H200, a graphics processing unit designed for training and deploying the kinds of artificial intelligence models that are powering the &hellip;<\/p>\n","protected":false},"author":1,"featured_media":24123,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"hashtags":[],"_links":{"self":[{"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/posts\/24122"}],"collection":[{"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/comments?post=24122"}],"version-history":[{"count":0,"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/posts\/24122\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/media\/24123"}],"wp:attachment":[{"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/media?parent=24122"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/categories?post=24122"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/tags?post=24122"},{"taxonomy":"hashtags","embeddable":true,"href":"https:\/\/wp.worldtechguide.net\/wp-json\/wp\/v2\/hashtags?post=24122"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}