by Sundeep Bajikar

Jun 30, 2018

This is the second installment of a two-part blog summarizing my presentation at the New Street Research Accelerated Computing conference hosted by Pierre Ferragu on May 30 in New York. I hope this summary will help everyone attending the upcoming #AIDesignForum being hosted by Applied Materials and SEMI on July 10 in San Francisco as a new track within SEMICON West.

In the first installment, I summarized how computing architecture requirements for AI workloads are different from what we became familiar with over the past couple of decades, in the form of traditional computing architectures such as x86 and ARM. In this post, I will also make the case that new AI computing architectures require materials engineering breakthroughs. I’ll give some examples of the types of breakthroughs we are referring to and describe one specific example in the area of new memory.

A Journey from Classic 2D Scaling to Architecture Innovation

In the PC and Mobile eras, which spanned more than 20 years, the industry was driven by “Classic 2D Scaling,” which refers to geometric transistor scaling as predicted by Moore’s Law in 1965. As transistor features got smaller, performance increased and power decreased consistent with Dennard’s Law. Higher transistor density also had the effect of lowering cost per transistor such that total chip cost decreased. During this period, we didn’t see a lot of architecture innovation, as a lot of focus was put on geometric scaling of a prevailing architecture, and this reinforced the rise of the x86 and ARM CPU architectures.

As you are probably aware, classic 2D scaling is no longer working as well as it used to. It has become very complex and very expensive, and the benefits it returns have been diminishing. So as we transition to the era of AI and IoT, a lot of emphasis is being placed on architecture changes to drive performance and power improvements. Architecture changes can drive a dramatically higher level of performance by fundamentally changing how data are processed. As you have probably seen with Google’s new TPUs, NVIDIA’s new GPUs and Microsoft’s Project Brainwave, the performance improvements relative to traditional CPUs are quoted in the order of magnitude and above range, e.g. 30x, 50x or even more. This is clearly a radically higher level of improvement versus what classic 2D scaling could provide, even in its best days. Architecture advances can be enabled by entirely new devices, or 3D techniques that can be used to extend existing logic and memory architectures, or even advanced packaging that brings diverse silicon together in new ways to perform specific functions.

In the sixth edition of their popular textbook published in December 2017, John Hennessy and David Patterson (widely regarded as the fathers of modern computer architecture) have meticulously summarized 40 years of processor performance data to make the case that three of the popular “laws” related to CPU performance progression have all effectively petered out. According to the reference, Dennard scaling ended in 2003, Amdahl’s Law reached its limits in around 2011, and Moore’s Law progress ended in around 2015. Our interpretation of their analysis is that the free ride of performance improvement from classic 2D scaling has effectively ended, and we now must work hard to combine architecture innovation and materials engineering breakthroughs to enable the performance improvements required for AI.

Applied’s Prabu Raja participated in a panel discussion with other industry luminaries earlier this year on this topic. If the definition and nomenclature of logic technology nodes interests you, you might enjoy a three-part blog I published recently.

Materials Engineering Breakthroughs

In the days of classic 2D scaling, materials engineering involved a limited set of materials such as silicon, silicon dioxide, silicon nitride and carbon. But these materials have already hit scaling limitations. In fact, we are effectively running out of materials on the periodic table that work well in isolation.  We increasingly need to create unique variations, and this has given us 10 different types of silicon dioxide and eight types of silicon nitride, for example. In the new AI era, materials engineering increasingly involves creating new combinations of materials that work together as one system to enable specific properties. We call this Integrated Materials Systems.   

As I discussed in the first blog post of this series, we believe there are three major architecture improvements needed for AI workloads: higher bandwidth memory, higher data parallelism and lower precision.

To achieve these improvements, we can draw upon materials engineering breakthroughs. For example, we can integrate new memory structures within existing logic devices. New materials can also enable entirely new memories such as Intel® 3D XPoint™ technology. We can use materials engineering to help self-align dense features such as the billions of vias present in GPUs. We can also use materials engineering to help vertically orient portions of the transistor for greater area density — as exemplified by Intel’s contact over active gate innovation. Materials engineering also plays a role in heterogeneous integration via advanced packaging: unique substrates, polymers and through silicon vias (TSV) can help enable denser packaging of a variety of separate chips.

Example: STT-MRAM

Here is an example of a materials engineering breakthrough used to enable a new type of memory. Below is a picture of an STT-MRAM array. The left side shows an array of memory cells, including a close-up of one of the memory cells. The right side shows how more than 15 materials have been stacked to help form the devices. The thickness of each of the films varies from 0.2nm to 2nm. By way of comparison, you may recall an innovation called High-K Metal Gate (HKMG) from about a decade ago; HKMG involved about six new materials, each about 1nm thick.