Why AMD considers AI as important as CPUs and GPUs

According to an AMD senior executive, AMD is considering deeper integration of AI across its Ryzen product line. However, there is still one element, the client application and the operating system that will actually make use of it.

AMD released the Ryzen 7000 series with the Ryzen 7040HS in January and announced the Ryzen 7040U, a low-power version of the 7040HS in early May. Both processors are the first chips to feature Ryzen AI XDNA hardware, one of the first AI logic for PCs.

However, AI so far is entirely a service running in the cloud, with a few exceptions, including Stable Diffusion. This is where AMD’s venture is risky. Why put expensive silicon into an IPU (Inference Processing Unit) to develop features no one can properly use on a PC?

That’s one of the questions PCWorld posed to David McAfee, vice president and general manager of AMD’s client channel business unit. According to McAfee, this is on the verge of a series of announcements and events that will help clarify what’s happening in terms of AI processing in general. This is just the tip of the iceberg.

It’s unclear whether McAfee is talking about Google I/O, the Microsoft Build Developers Conference later this month, or something else entirely. However, AMD seems to be taking a ‘smaller stride’ approach to AI than it seems.

“PC AI is not as complex as you think”

Of course, if you have enough storage and memory, you can run AI models on Ryzen CPUs or Radeon GPUs. However, McAfee described Ryzen and Radeon as too heavy a hammer to use for this type of computing.

AMD sees AI in PCs as small, lightweight, frequently triggered tasks that run on AI processors called IPUs. AMD has been experimenting with and optimizing AI technology for PCs with AI for quite some time now, and Sense for Ryzen processors that adjust clock speeds through Precision Boost 2, mXFR, and Smart Prefetch. It blends several technologies under the umbrella of MI (SenseMI). IPUs can take these technologies to another level.

“The IPU and future IPUs are more of a combination of specialized engines that perform certain types of computing in a power-efficient way,” said McAfee. “In some ways, they are integrated with the memory subsystem and the rest of the processor. “Workloads running on GPUs are expected to be a more frequent type of computing in the future (if not continuously running workstreams) on user platforms rather than one-off events.”

AMD considers the IPU to be similar to a video decoder. In the past, video decoding tasks could be pushed by Ryzen CPUs. However, massive computing power was required to ensure a pleasant experience. “Alternatively, this could be done using a relatively small engine that is part of the chip design and extremely efficient,” McAfee said.

That means, at least for now, the Ryzen AI IPUs don’t have a separate card or their own memory subsystem. Stable Diffusion’s generative AI models run using dedicated video RAM. However, when asked about the concept of “AI RAM,” McAfee took a negative stance, saying, “That method would be very costly.”

Ryzen’s AI Future

XDNA and Ryzen AI are like RDNA and Radeon in that the first term defines the architecture and the second term defines the brand. AMD acquired AI capabilities through its acquisition of Xilinx, but did not disclose exact details. McAfee conceded that AMD and its competitors need to define Ryzen AI’s capabilities using language that ordinary consumers can understand, such as core counts or clock speeds that help define a CPU.

“With the IPU, there is something called an architecture generation,” McAfee said. “Integrating into a product this year and integrating into a product of the future may be of a different architecture generation.”

The problem is that no consumer-specific AI metric has been defined (e.g., number of cores per parallel stream or neural layers), and there is no commonly accepted AI metric other than Trillions of Operations Per Second (TOPS) and TOPS per Watt.

“There is still no solid industry-standard benchmark or industry-standard metric to help users understand the difference between AMD’s Widget A and Qualcomm’s Widget B,” McAfee said. With the current language and benchmarks, it is not easy for users to understand what to choose now and which to invest in.”

As Ryzen AI is only being deployed on two Ryzen laptop processors, a natural question arises as to how it will be deployed across the rest of the CPU lineup. McAfee explained that discussions on that part are also ongoing, saying, “We are discussing AI across the Ryzen product line.”

As Ryzen AI core manufacturing comes with additional costs, AMD is evaluating the value added by Ryzen AI, especially in entry-level processors. McAfee said “much more specific” end-user benefits need to be identified before AMD can add Ryzen AI to its low-end mobile Ryzen chips.

Will AMD Add Ryzen AI to Desktop Ryzen? It might be a bit surprising, but I’m not sure. McAfee considered this question in terms of the desktop’s strategic effectiveness. Given the desktop’s performance, McAfee noted that “Ryzen AI can become a testing tool rather than a role in driving the device’s day-to-day value.” McAfee added that while Threadripper with higher core counts could be used for AI training, it is not required.

However, AMD believes AI will take the place of current CPUs and GPUs. “I’m confident that in the not-too-distant future, people will consider AI as a third component of computing that adds value to their systems, in addition to CPUs and GPUs,” McAfee said.

The next step in AI

Traditionally, chip evolution has been fairly straightforward. Developers devise new apps and program them to run on general-purpose CPUs. Over time, the industry settles on a specific task (such as video games) and specialized hardware follows. Data center inference chips have been in development for years, but app developers are still working on what AI can do, as well as what consumers can use them for.

McAfee said there would be two reasons why AI applications would run on PCs rather than the cloud. It would make sense to deploy it on a local AI accelerator residing on a mobile PC platform to reap the benefits.”

The second reason is security. As AI is integrated into business activities, those businesses and their consumers will want dedicated AI services to avoid the risk of personal or business data leaking into the cloud. “I wouldn’t want a situation where a public instance can scan and use all my emails and documents,” McAfee said.

what the software does

McAfee declined to comment on what it knew about Microsoft’s roadmap and speculation that Windows 12 could be more tightly integrated with AI. Consumer AI applications may include games. For example, it would be possible to have an NPC that communicates intelligently rather than according to a script.

“I think this will be key,” McAfee said. Over the next three years, the software and user experience must deliver that value, and this exciting new technology that is just now emerging in the conversation about the PC needs to evolve into a disruptive technology.”

However, McAfee added that responsibility for the success or failure of AI does not solely lie with the hardware industry, including AMD, Intel and Qualcomm.

“The key to the ultimate success of AI is whether it is supported by software,” McAfee said. Does the software and user experience live up to the heightened expectations? This will be the key. Over the next three years, software and the user experience must deliver their value, and this exciting, yet new phase of technology that is now emerging in the conversation about new PCs will revolutionize the way we think about performance, devices and how to use them. It has to be developed into a technology that creates it,” he said.
[email protected]