The computation demands of 21st-century abundant-data workloads, such as AI /machine learning, far exceed the capabilities of today’s computing systems. For example, a Dream AI Chip would ideally co-locate all memory and compute on a single chip, quickly accessible at low energy. Such Dream Chips aren’t realizable today. Computing systems instead use large off-chip memory and spend enormous time and energy shuttling data back-and-forth. This memory wall gets worse with growing problem sizes, especially as conventional transistor miniaturization gets increasingly difficult.

The next leap in computing performance requires the next leap in integration. Just as integrated circuits brought together discrete components, this next level of integration must seamlessly fuse disparate parts of a system – e.g., compute, memory, inter-chip connections – synergistically for large energy and execution time benefits. This talk presents such transformative Nano Systems by exploiting the unique characteristics of emerging nanotechnologies and abundant-data workloads. We create new chip architectures through ultra-dense (e.g., monolithic) 3D integration of logic and memory – the N3XT 3D approach. Multiple N3XT 3D chips are integrated through a continuum of chip stacking/ interposer/wafer-level integration — the N3XT 3D MOSAIC. To scale with growing problem sizes, new Illusion systems orchestrate workload execution on N3XT 3D MOSAIC creating an illusion of a Dream Chip with near-Dream energy and throughput. Beyond existing cloud-based training, we demonstrate the first non-volatile chips for accurate edge AI training (and inference) through new incremental training algorithms that are aware of underlying non-volatile memory technology constraints.

Several hardware prototypes demonstrate the effectiveness of our approach. We target 1,000X system-level energy-delay-product benefits, especially for abundant-data workloads. Such large benefits enable coming generations of applications that push new frontiers, from deeply-embedded computing systems all the way to the cloud.