James (Jim) Ang: Let's see, Paul, I know we're at the top of the hour. Should we go ahead and get started? I know we're also recording.
Paul McIntyre: Yes, we should get started. Can you, can you hear me?
James (Jim) Ang: I can hear you fine.
James (Jim) Ang: Okay, yes, let's see.
Paul McIntyre: Thanks. Thanks, Jim.
James (Jim) Ang: Okay.
James (Jim) Ang: All right, well, welcome to, all of the Meerkat,
James (Jim) Ang: PIs and their, personnel, their extend… the extended team, for the research center. Today we have, an opportunity to give an overview of the Decode, project, and, many of the,
James (Jim) Ang: I guess the leadership team for our project is online, and we'll be sharing the presentation. I'm going to start off by kicking off with a few, introductory slides, and
James (Jim) Ang: And I think each of our thrust leads will, will take over for me for, sharing their slides, so…
James (Jim) Ang: Let me, you can find the link for sharing.
James (Jim) Ang: Okay, are you all seeing this in presentation mode?
Ryan Coffee: Yep.
James (Jim) Ang: All right, very good. So, for this overview, I'm, Antonino, and I will give kind of an introduction to the democratization of co-design, and
James (Jim) Ang: Then to present, our Thrust 1, David Brooks from Harvard will, will be giving that. And, for Thrust 2…
James (Jim) Ang: I know John Lydell is on. I guess I wasn't sure if I saw Andreas Olofsen, but one or both of them can present Thrust 2.
James (Jim) Ang: Luca Carloni will, from Columbia, will present Thrust 3.
James (Jim) Ang: And, As shown in this graphic, Thrust 1 is co-design of energy efficiency.
James (Jim) Ang: System in a package.
James (Jim) Ang: Technology. Thrust 2 is open source co-design tools.
James (Jim) Ang: Press 3 is prototyping and chiplet integration.
James (Jim) Ang: Let me start off by just describing
James (Jim) Ang: really the core of our objective for Decode.
James (Jim) Ang: I won't read all the text here for you, but basically, our approach is to democratize co-design through the realization that design of R&D prototype hardware
James (Jim) Ang: is analogous to and can apply lessons from the development of R&D prototype software.
James (Jim) Ang: We leverage open source hardware design, generation, and analysis tools.
James (Jim) Ang: That can be used to target prototype and low-volume fabrication versus production. And this is important, an important concept for us to be aware of.
James (Jim) Ang: We try to delay the use of commercial EDA tools.
James (Jim) Ang: Because they are expensive, and that introduces barriers for vendor proprietary design data. And delay that until the conceptual designs, the research.
James (Jim) Ang: That we pursue shows promise that hardware design approaches are ready for, non-recurring engineering investments, and that's where, you know, costs really significantly ramp up.
James (Jim) Ang: We try to leverage, CHIPS Act investments that are really establishing an infrastructure for hardware prototyping, and things like an open chiplet ecosystem that focus on, helps us focus on domain-specialized hardware design and generation.
James (Jim) Ang: While we're using, chiplet designs for conventional general-purpose processors, memory interfaces, network interfaces, etc.
James (Jim) Ang: We introduced this concept. It's actually an old concept from the product world of MVP, Minimum Viable Product.
James (Jim) Ang: And the idea here is to use MVP test hardware to provide concrete measures of performance improvement, power and energy utilization, where we're exercising our concepts with real application workloads.
James (Jim) Ang: In an MVP, a lot of the focus is on core functionality, where we're targeting novel or innovative architecture designs, and for us, that really is based on heterogeneous computing concepts.
James (Jim) Ang: The most important feedback for us comes from, early adopters, and…
James (Jim) Ang: The most critical early adopters for us are software developers. Software developers for algorithms, applications, and the software stack, compilers and runtimes.
James (Jim) Ang: And I guess, again, a lesson from the software world.
James (Jim) Ang: is that we're trying to, support agile, technology development, and, really to support a path to having, rapid design cycles to help explore a design space, to help guide the.
James (Jim) Ang: the development of future, advanced computing concepts, and specifically for Decode, energy-efficient computing concepts.
James (Jim) Ang: Very quickly, this is a summary of the entire team. You'll be hearing from our leadership team that includes representation from
James (Jim) Ang: Most of these institutions, with the exception of Washington State University, Rice University.
James (Jim) Ang: and, University of Minnesota. But, this table, shows all the contributions that the institutions provide to our team.
James (Jim) Ang: Huh?
James (Jim) Ang: Question?
James (Jim) Ang: If you can mute yourself, that'd be great.
Chun-Long Chen (PNNL): Well, today.
James (Jim) Ang: Do you have a question?
James (Jim) Ang: Alright, let me move on. Let's see, where am I?
James (Jim) Ang: Oh, here it goes. So our approach really is focused on the idea that we could, target heterogeneous computing. This arises naturally when you talk about energy-efficient accelerators combined with, general-purpose processors.
James (Jim) Ang: But we actually have some history at, at PNNL in, internal investments, LDRD investments in something called the Data Model Convergence Initiative.
James (Jim) Ang: Where we think about, converged workloads being traditional scientific computing, other types of AI, app, methods and, applications, as well, as well as, data analytics applications. And the,
James (Jim) Ang: The target then for us is, is either, can span scales from, the large-scale, high-performance computing systems to, edge computing with integrated sensors.
James (Jim) Ang: Skipping down here, a lot of our innovation in decode is focused on some concepts for custom, analog accelerator, designs, that, are very energy efficient, but,
James (Jim) Ang: of the examples that we use are really exemplars that we hope should be generalizable to a range of other accelerator types, and really many of the candidates that are being pursued within Meerkat and other projects are,
James (Jim) Ang: Our, opportunities that we think could be relevant for us.
James (Jim) Ang: Let's see… And, Maybe with that, I will,
James (Jim) Ang: Pass the microphone over to Antonino.
Antonino Tumeo (PNNL): Thank you very much, Jim. So, I'll briefly go, detailing a little bit the objectives of the Tree Trust, and then, obviously, you will hear from the leadership team the details.
Antonino Tumeo (PNNL): But, basically our pre-trust, as Jim has introduced, right, are looking at the cheaper ecosystem on one side, including the novel technologies and analog title accelerators.
Antonino Tumeo (PNNL): And, the second class would be about the design solutions, and then we'll bring on, close the loop, basically, with the prototyping. So, for, for the first one, really, the focus is on the co-design process for this heterogeneous system in a package.
Antonino Tumeo (PNNL): that…
Antonino Tumeo (PNNL): Should include, the digital conventional, accelerators, the processors, and basically create this, catalog of open chiplets, but also prepare, like, the, the, the system and…
Antonino Tumeo (PNNL): Bring the co-design to the level of, can we, implement and leverage
Antonino Tumeo (PNNL): new type of accelerators that are energy efficient. And in particulars, in particular with our exemplar for download or three-dimensional computing that can be implemented both in digital or in near-memory analog versions.
Antonino Tumeo (PNNL): And, the resistively coupled, dynamical system machine, which is, of, the, the, electricalizing machine, able to, to deal, with, with,
Antonino Tumeo (PNNL): Spins that are not anymore just, 01.
Antonino Tumeo (PNNL): computing will be, led by, Tayana Rosin in UC San Diego, and, the RDSM, work is led by, Tom Jiang from, from RISE.
Antonino Tumeo (PNNL): And, yeah, like, the… the… the…
Antonino Tumeo (PNNL): There is a part, obviously, on the work, because we… as you see in the tools, we have available a number of generators together with the co-design tools, so we want to be able to assign them, but we want also be able to create
Antonino Tumeo (PNNL): A low-cost, agile, system in the package that can be used, and be available also to, to the market.
Antonino Tumeo (PNNL): teams, so that we can, really, as Jim highlighted, enable this flow from the concept to the prototype.
Antonino Tumeo (PNNL): And I think we can move to the second trust.
Antonino Tumeo (PNNL): Again, suppose the lighting we have available, and you'll see a number of tools already, a number of capabilities of open source tools that come from our various partners.
Antonino Tumeo (PNNL): the focus of Decode will be integrating these tools to create a much larger ecosystem of co-design tools that will enable really us to bring the concept of a system in a package, evaluated and generated, and then tap it out, right? Really, we want to
Antonino Tumeo (PNNL): to provide the subset and also the compiler site to be able to, quickly go from, from the idea, assemble the system, and, and enable, the, the design of the determining a package.
Antonino Tumeo (PNNL): They, they…
James (Jim) Ang: Bust?
Antonino Tumeo (PNNL): Trust, is, bringing everything together with the prototyping and chip-right integration. The, from, from, from, again, from the Trust leads, but, the team has, a lot of experience in prototyping and tape-outs.
Antonino Tumeo (PNNL): So, we bring internal prototyping capabilities, including FPGA platforms that are already available, and to kind of integrate Custom Accelerator and do the test, we'll…
Antonino Tumeo (PNNL): We'll enable that to, emulate our system in a package, but also, like, with the EDA tools, we'll enable integration with the capabilities, the standard capabilities. Now, we'd need to see what happens with the National Silicon Center, but one of the,
Antonino Tumeo (PNNL): TOFUS will be trying to establish connection with NSTC and foundries so that we can enable this, little
Antonino Tumeo (PNNL): And thinking about the last slide is the list of our background repositories.
Antonino Tumeo (PNNL): I think, switching slowly, but… it's a long list, right? So I understand. So these are already available, right? We'll, so we start from strong foundation.
Antonino Tumeo (PNNL): These are all the tools that, you can go already use and leverage, and hopefully you already knew that. But, with the, the, with the code, we'll, will bring this, to, Plus Plus Basher, right, that, will enable,
Antonino Tumeo (PNNL): eco-design, really echo design tool chain available for, the fast prototyping cycle. And,
Antonino Tumeo (PNNL): I think this was the last slide, Jim, right?
James (Jim) Ang: Right. I will stop sharing, and then pass… we can pass the,
James (Jim) Ang: We'll feed over to David Brooks.
James (Jim) Ang: David, take it away.
James (Jim) Ang: Great.
David Brooks: Are you able to see my slide okay?
James (Jim) Ang: Yes.
David Brooks: Great.
David Brooks: Let's see… here we go. Okay,
David Brooks: So yes, I'm happy to present, Thrust One on behalf of Deanna Rosing and myself, and the Thrust One co-leads for the project.
David Brooks: If you think about the overarching motivation for a lot of what we're looking at these days, it's this notion of scale of AI compute and the cost growing at an enormous rate.
David Brooks: And, this is just a graph from Epic AI. If you're ever interested in looking at trends, they have a lot of really interesting data on the website. This is showing the training compute needed for these frontier LLMs that are available today, and you can see the growth rate has gone up by between 4 and 5x per year over the last decade or so.
David Brooks: We have, of course, have seen increases in the flops and the memory bandwidth to go with that, but not nearly at the same rate. And this is really increasing the training costs at a pretty large, hefty, hefty, rate of 3.5x per year. So the training computes requirements are going up.
David Brooks: the ability to provide the flops and the computation is not scaling at the same rate, so we're in this deficit that is part of the reason we're seeing such huge investments in AI, because people still want to proceed, but the costs are going up. And as Jim mentioned, we want to think about lower-cost ways of driving computing, and we believe chiplets are one of those solutions that, particularly for DOE
David Brooks: applications could allow drastically improved performance without this enormous cost.
David Brooks: In Thrust 1, we're thinking about this in the context of the co-design of energy-efficient chiplets for the system and packages.
David Brooks: And the goal is to explore a mix of different types and styles of accelerators for AI applications. So…
David Brooks: The goal really is to come up with this mix of heterogeneous chiplets that combine computing paradigms, so some of them conventional, and some of them new and addressing specific aspects of AI that have been less well explored. So, for example, Tiana's work on hyperdimensional computing.
David Brooks: The work that we have in the next task on icing machines and dynamical systems are really new approaches to addressing machine learning. And likely what we will see in the future is mixtures of these approaches using mixtures of different technologies, some digital, some analog, in these system and packages. And that's the goal, the thrust is to also understand how to bring those different, diverse.
David Brooks: types of chiplets together. So there's integration of different IP styles.
David Brooks: I'll give you a flavor of how this would this work, and why analog and HD computing are… are interesting.
David Brooks: So just, we start from the observation that the human brain doesn't process information in simple low-dimensional forms, like pixel grids or individual signals. Instead, it converts dense sensory inputs, things like the images that come through our eyes, into high-dimensional sparse representation across neurons. And this mapping step is basically how we
David Brooks: how we think human perception and cognition works. So if you look at the picture on the left that's showing light hitting a retina, the signal passing through these visual areas, the brain representing that information as a distributed pattern, basically a high-dimensional vector.
David Brooks: And there's studies from about a decade ago that showed that this sparse, high-dimensional encoding makes the brain's computation both efficient, robust, and flexible. HD computing takes that principle and applies it to machine learning. It encodes data in what are called hypervectors. These vectors have thousands of dimensions, and then uses simple algebraic operations to manipulate them. And that's really the key. These operations are fully parameterized.
David Brooks: They're very simple, Makes them much more energy efficient and resilient to noise.
David Brooks: Hd computing is…
David Brooks: seen a lot of research interest over the past decade, and people have shown that it can be used for online learning, symbolic reasoning, explainability, and so forth. Big challenges for today's AI.
David Brooks: On the right, you can see the domains that this has been applied to, image classification, LiDAR segmentation, genomics, mass spectrometry, even recommendation systems, and so the point is that with this mathematical framework.
David Brooks: Hyperdimensional representation, we can handle data that spans very different kinds of representations, from images to molecules to graphs.
David Brooks: So Tiana's, research group is one of the leaders in the flair. She's been working on this for, for quite a while. And, if you look at some of the progression, the original work in this area, which was looking at
David Brooks: conventional, image recognition applications using purely HD approaches, what you found is that the results work really well for small data sets like MNIST, but didn't do so well when you looked at the larger ones, like CFAR and Imminionet.
David Brooks: And, more recently, the algorithm investment in this area has shown that HD combined with CNN layers can get the best of both worlds, meaning the accuracy of conventional
David Brooks: image classification using CNNs, but with the efficiency of hyper-dimensional computing. And this hybrid architecture style works… you can see the picture on the right, works by pruning many of the CNN layers and inserting an HD encoding step that converts the extracted features into high-dimensional binary hypervectors. And then these hypervectors, are classified using the algebraic operation, so instead of using these
David Brooks: These compute-heavy layers, it's using much simpler algebraic operations for those final steps.
David Brooks: And you can see the results in the table showing that,
David Brooks: we can achieve almost the same performance, using, the HDNN approach, but with,
David Brooks: with basically the same accuracy as traditional CNNs. Tiana's group has also explored implementing these techniques in test ships, so she has 3 test ships that she's worked on in collaboration with, TSMC. Some of these are digital, and some of these are analog, and I think that's one of the interesting approaches to this, is that there are opportunities to leverage things like resistive RAM
David Brooks: And, do this analog-style computing using, hyperdimensional computing and
David Brooks: Again, this is why this is a promising approach, because with our chiplet-style architecture, we can imagine including an HDC chiplet next to a CNN chiplet, next to some other kind of chiplet, and combine the best of both worlds in terms of the
David Brooks: The neural network style, or machine learning style.
David Brooks: So now I'm going to spend a little bit of time talking about how we imagine integrating that existing IP, and most of this work is building on the efforts from Andrea Olson's company called ZeroASIC, which has been one of the leaders in trying to establish
David Brooks: standards for triple ecosystems, particularly for these
David Brooks: style of prototypes, like the one that… that I just looked… that I just showed, where there's, a little bit different from the NVIDIA style, where you have these massive chiplets that are… that are… that are, taking up
David Brooks: theoretical. These are looking at smaller triplets and how to put them together.
David Brooks: So the goal that Andreas' team has is to build a full-stack triplet standard, with the protocol layer, basically the way that the memory is addressed using what's called Universal Memory Interface, basically make everything addressable, that's the goal, is to be able to fully address all the different parts of
David Brooks: the triplets using this universal memory interface, and, inspired by, sort of, the extraction standards that have been built up over time in other areas of computer science, they have to find these different layers of the stack, from the protocol down to the electrical… electrical layer.
David Brooks: They've also been looking at trying to come up with standardized footprints that allow you to imagine plugging into something like a breadboard, except this would be, plugging in triplets with
David Brooks: very fine pitch for high connectivity. Rotational symmetry, so it doesn't matter how you put your triplet there. Be able to use this for both analog with various power domains, passing through to other systems and whatnot. And then also defining some of the electrical and standards of how you communicate between the triplets.
David Brooks: Let me see, I messed up my screen. Here's the next slide. Okay, so standardized discrete chiplets, so the goal, again, building on what Sarah Isaac has been pushing, is to come up with this LEGO brick standard, except for chiplets, where we define that standard, and then just like LEGO bricks have remained the same for…
David Brooks: so many years, can we do that for discretized triplets, so that I could come up with a grid, come up with standard-sized triplets, and then as long as you build these triplets with this size, you'd be able to plug it in.
David Brooks: to a package. This is how I imagine low-cost prototyping moving in the future, and also perhaps low-volume
David Brooks: But high important computing strategies where, you know, you could imagine prototyping with thousands or tens of thousands of chiplets instead of needing to have millions that you would have to build a custom ASIC.
David Brooks: And being able to do this across different technology nodes with
David Brooks: Different chiplet spacings and pitch month ratios, and ideally building this and thinking about all the constraints early so that we have a long-lasting legacy of the ability to carry forward.
David Brooks: Zeroasic has built
David Brooks: several composable triplet prototypes already. This is just a mix of designs from a CPU, an FPGA, to a memory, and showing… they show that using these standard approaches
David Brooks: It can drastically reduce the amount of design time, amount of designers needed to be able to design these.
David Brooks: Very impressive results in, in, in, in, in some of these diverse, very diverse sets of applications.
David Brooks: So with that, I will… Wrap things up, and happy to take any questions.
James (Jim) Ang: Thank you, David. I think, we might hold questions for the end. So, and, and actually now we'll hear next from, I'm not sure if it's Andreas or John Lydell,
James (Jim) Ang: who are co-leading Thrust Tube.
aolofsson: I think we're… we're gonna share… share the burden here.
James (Jim) Ang: Okay, okay, very good.
aolofsson: I'll bring up the slides, and then,
aolofsson: John's gonna speak on the first three tasks, and I'll speak about the last… 3.
aolofsson: So let me just, let me share my screen here.
James (Jim) Ang: And David, thank you for giving a very good overview this closing session on Xero ASICS capabilities. Fantastic.
aolofsson: Alright, here we go.
aolofsson: Okay, you see that okay?
James (Jim) Ang: Yes.
John Leidel: Yes.
aolofsson: Alright, so,
aolofsson: So, you know, David spoke about actualizing, you know, real circuits. Us 2 is more about the tooling to create the circuits, and, you know, creating open source co-design tools.
aolofsson: to, lift all boats, or, you know, lift all ships.
aolofsson: So there's, there's 6 different tasks in this area,
aolofsson: And, you know, ranging from, efficient algorithms for analog computing, compilerless frameworks.
aolofsson: architectural simulations, and then, you know, heterogeneous SIP, thermal analysis, co-design, and then, IP generators. So it's sort of a stack from, like, a full-stack software, you know, algorithms to…
aolofsson: to bits, is the way I would look at it.
aolofsson: And, represented by participants from PNNL, from, from Harvard, from TCL, Wisconsin, Washington State, Minnesota, and Zero ASIC.
aolofsson: So with that, I'm gonna hand over to… to John to speak about, 2.1 to 2.3.
John Leidel: All right, perfect. So the first task in this is led by P&L, and this is where everything sort of starts.
John Leidel: One of the issues that we've had, previously is going from traditional, sort of instruction set-based, digital designs into.
John Leidel: more chiplet ecosystems, system and package, and analog designs, is there's really a lack of focus on the algorithm infrastructure necessary to port applications and actually get real kernels and real
John Leidel: Real workloads, executing efficiently to realize all of the energy efficiency and programmatic efficiency of the various different devices.
John Leidel: So this is very analogous to, sort of, the quantum world, where, they had to sort of crawl out of the primordial ooze, and start developing new programming models, new compiler techniques, and new algorithms sufficient to actually utilize real quantum systems.
John Leidel: So as we move into the space of brain-inspired HDCs, there's a lot of interesting proposals with respect to porting things like neural networks, sparse linear algebraic solvers, and various different high-dimensional random vector algorithms, onto these analog computing platforms.
John Leidel: And this thrust area is specifically focused on developing those algorithmic constructs in a way that we can move from a very high-level conceptual design down to more of a co-design ecosystem where we start cross-collaborating with the compiler.
John Leidel: subtask.
Keren Bergman: Whoa, what are you doing?
John Leidel: As well as, collaborating with some of the other, hardware implementation thrusts that we can begin, mapping, sort of, traditional instruction set concepts down to more analog-style computing devices. So… so this is going to be really key and very,
John Leidel: influential across all the different domains of the Decode project.
John Leidel: Okay, on to the next slide, Andreas.
John Leidel: So, so this is where, things start to move more into traditional computer science. So P&L is, again, leading the compiler and accelerator generate… Compiler framework and accelerator generators. I, I, I hate saying those two words together.
John Leidel: So, with the advent of accelerated computer architectures, we've done a fairly good job of
John Leidel: putting together very coarse-grained programming models like CUDA and SYCL and OpenCL, and even some of the OpenMP and OpenACC target offload specifications to take traditional, sort of, iterative approaches to algorithmic design and begin doing,
John Leidel: parallel computing with host and target offload. That's been fairly simplistic. You know, accelerators traditionally live on the other end of a relatively inefficient bus or inefficient interconnect.
John Leidel: And now with the system and packages, as we start to standardize the interfaces across these various different, heterogeneous devices.
John Leidel: That may or may not live inside even reasonably similar clock domains or, or architectural models.
John Leidel: We need to start raising the level of abstraction in the compiler frameworks such that we have any chance of all at actually taking traditional parallel applications and begin executing them with some degree of efficiency. So this particular thrust area is going to really focus on using LLVM and MLIR
John Leidel: To start building dialects around performing explicitly heterogeneous computing across computing domains on a system and package.
John Leidel: So this is going to, quite literally, reutilize a lot of the existing dialects that are being used for, parallel computing in the AI and ML space, as well as, we can hoist some of the interesting efforts being done in the quantum space for, MLIR, as well as looking at things,
John Leidel: like some of the hardware generators that are built around MLIR,
John Leidel: To literally start doing, co-design,
John Leidel: at the system and package level, sort of taking the Pandas and Bamboo work previously done at PNNL, and start bringing that ecosystem into the sort of top-level, application-level compilation and optimization frameworks.
John Leidel: So… so this is going to be a very interesting effort that could, not only bear fruit in the system and package world, but also bear fruit in…
John Leidel: The future, you know, sort of, digital-centric, large-scale, monolithic chiplets, ecosystem as well.
John Leidel: Okay, on to the next slide, Andreas.
John Leidel: So the, Thrust 2.3, which, is a co-effort between, TCL and David Brooks' group at Harvard, is the architectural simulation of heterogeneous SIPs.
John Leidel: So, architectural simulators is something that is both near and dear to my heart.
John Leidel: And something that everyone loves to hate. Historically, architectural simulators are both fantastic and allow you to accelerate co-design workflows, and are also demonized in the sense that
John Leidel: If the architectural simulator is not well-formed and well-written, then it doesn't accurately represent the particular device you're attempting to simulate.
John Leidel: So, what we seek to do with this is actually combine two efforts, that have sort of started, holistically at Harvard and TCL, the first of which is what's called Harvard Cascade.
John Leidel: Which is a high-level design exploration infrastructure that allows users to explore application domains, whether they're dense or sparse, or whether you're targeting a specific numerical representation.
John Leidel: And combine that with some kernel partitioning logic that allows you to then, using topological concerns.
John Leidel: And energy and device scaling concerns, and start looking at the design space exploration of, okay, here's an application, here's what the kernels are specifically going to do.
John Leidel: you know, what is the prescriptive level of chiplet or chiplet devices that best fit this protect… this particular application? You know, I have this much energy, this much device area, you know, what's the correct mixture of where these things go?
John Leidel: The flip side of that is the TCL SST infrastructure, which is entirely based upon Sandia's structural simulation toolkit, is a large-scale parallel discrete event simulator, really designed to
John Leidel: take applications or application traces, and then do very fine-grained analysis of how those interact with specific architectural components. So we have a very large catalog of existing components we've simulated.
John Leidel: things like, RISC-V cores, network on chips, system and package devices that we're currently working on, memory devices, into that list, we're going to start adding analog computing devices.
John Leidel: And the interesting thing about the extensions that we've made to SST is we actually have a series of SST components that allow you to directly replace an existing functional
John Leidel: simulation component with an RTL component as well. So basically, we wrap up your, your, Verilog, or system Verilog RTL in some C++ library magic.
John Leidel: And then we allow you to co-simulate this sort of multimodal simulation framework.
John Leidel: So, what we seek to do with this particular thrust area is combine these high-level design space exploration concepts from Cascade with these very low-level mechanisms in TCL's SST infrastructure, and then do backward validation using a massive amount of telemetry that comes out of SST with, okay, here's your designs,
John Leidel: design space exploration, and here's what the high-level tools are sort of prescribing in terms of SIP architecture. What does this actually do at a more functional level with, very high,
John Leidel: high-fidelity functional simulation, or even RTL simulation. So, you'll be able to sort of rinse and repeat a large number of designs back and forth through simulation, and get a very, very,
John Leidel: High degree of confidence in terms of, you know, what you seek to actually construct in real silicon.
John Leidel: And I think I can pass it off to Andreas for Thrust 2.4.
James (Jim) Ang: Thank you.
aolofsson: Okay, all right, so, so thrust to 4 to 2, 6, I think can be best described as, as filling out the, the picture and, and tying the…
aolofsson: the simulation to the bits and the atoms. So, as was discussed, right, if you're… you can have the best architectural simulator in the world, if it's uncoupled from physics, it's still open loop, right? You need a cost function that's grounded.
aolofsson: And that is incredibly important. It's something that's been missing from architectural simulations in the past.
aolofsson: Sometimes, right? Not always, but sometimes. And so… and of course, one of the problems is that you'd like to run everything at the atomic level, but you can't afford to.
aolofsson: And so you have to make choices. And, you know, do you have these knobs, you know, accuracy versus, performance, that are, you know, those are conflicting.
aolofsson: And, and so, so in 2.4, one of the ideas here is that, for example, for thermal modeling, is that you're going to try to trade off
aolofsson: and be able to leverage the full spectrum of accurate models, like FEM models, to analytical models that run very fast.
aolofsson: And then, you know, being able to do trade-offs that span from days to milliseconds, right? From, you know, many, many degrees of, like, you know, 5 to 10 degrees, let's say, to, you know, sub-1 degree accuracy.
aolofsson: And, you know, I can tell you that, you know, this is something that we've, we've run into in the past, where, especially early in the design process, we don't have all the information, to really run the, you know, the ANSYS console-type, physics,
aolofsson: FEM solvers, or open source packages, but we'd like to have, you know, ballpark of what the system's like, and that's been a real gap, and so it's, it's great that the team is working on this area.
aolofsson: Second, which is, you know, kind of rhymes with Thrust 2.2 in a way, is this idea of chiplet packaging and co-design optimization. And so, so as you're defining your chiplets, you're defining the structure and your accelerators.
aolofsson: With… with heterogeneous, integration and system and package.
aolofsson: you actually have a new design space that… that has not been fully fleshed out before. You know, there's been a lot of talk about co-design for SOCs, where you might have IP blocks, and
aolofsson: The interfaces are completely fluid, and you do partitioning of logic into blocks.
aolofsson: But once you start having discrete devices.
aolofsson: That may be hard-coded, and you might be mixing discrete devices.
aolofsson: With, with new devices, new designs, new core processors. You do have a pretty interesting search space,
aolofsson: And especially when you start bringing economics into the picture. So, you know, you're not talking about not just size, weight, and power, and performance, but possibly also, you know, how many,
aolofsson: How many dies are you going to get out of the wafer? Which technology node are you going to be working on? Are you going to be working at 0.18 micron or at 3 nanometer? Where do you put your I.O? Where do you put your, your compute? Where do you put your memory? Do you want to do 2D, 3D, organic substrate?
aolofsson: It's such a, a squishy, search space, and it's something that most of us, as architects.
aolofsson: We used to use… we resort to spreadsheets, still today.
aolofsson: And, so that's a… that's a huge gap in the architect's toolbox that,
aolofsson: I think, this team can address.
aolofsson: And then, and then the last one is, is really, down to the metal, and this is a bit of a repeat, because David already showed this, so ignore the picture, I'll just talk about what we're actually doing in this task.
aolofsson: So… so we're, we're, we're creating a full-stack, open-source, chiplet interface generator, that's portable. It's gonna be digital only.
aolofsson: So, you know, imported any node, it's gonna be open source, and so if you look at and support the full-stack standard that Dave mentioned, that we're working on.
aolofsson: So, so if you look at the chiplet interface base today, it is a, it is a place of the, of the folks with the money, the rich.
aolofsson: You know, the big data center players, there's very few open implementations. There was one sort of opened, AIB, that was funded by the CHIPS program at DARPA, but, but since then, everything has pretty much been closed source.
aolofsson: And most of it's been targeted at data center rates, so very, very high frequency with analog PHYs, which are very expensive to port, or complicated to port.
aolofsson: And, so it's a huge barrier to access. So if you want to create a, low-barrier, democratized, chiplet-based design system, we need the artifacts, we need the IP for these, for these interfaces. So, so this task is all about creating a…
aolofsson: A generator, an RTL generator that supports the standard that was discussed in Thrust 1.
aolofsson: And so, you know, if you look at the, especially the right part, the electrical side, the protocol and footprints, those are pretty easy to define, it's just a spec, basically, and certainly protocols are very RTL-friendly, but when it comes to the electrical interface, you have to make some very careful choices, and, also.
aolofsson: Have an ability to decouple things like clocking, synchronization.
aolofsson: ESD and so forth from the technology node in a very light matter, or zero-cost matter. So that's going to be the bulk of the effort here. But the outcome is, you know, delivering an open-source, full-stack, piece of IP to the community.
aolofsson: So I think that's… that's our thrust, too, so I'll hand it off to…
aolofsson: Luca or… or Andrew for those three?
James (Jim) Ang: It's LUCA for Thrust 3.
luca: Yes, good afternoon, everyone, can you hear me?
James (Jim) Ang: Yes.
luca: Okay, and let's see if I can share properly…
luca: Let it work?
James (Jim) Ang: Yep.
luca: Unless maybe take the video out, so to avoid.
luca: So, I think we are running a few minutes late, so I'll try to,
luca: Let's see if we can cover some time for Q&A. Task 3 is, the, about…
luca: Prototyping and integration of this system in a package solution that we are pursuing, it consists of two tasks. The first one, called internal prototyping.
luca: is led by myself and Guyon at Harvard, and tackles the challenge on how we enable in-house agile design, rapid prototyping, and early software development of this advanced chiplet-based system in a package.
luca: With an emphasis on DOE applications.
luca: And the approach that we will,
luca: Portillo is to develop scalable, open-source platforms, which really are meant to make it possible to have a distributed, collaborative engineering of these components, which are highly heterogeneous, as you've seen from
luca: The previous two presentations.
luca: And, we are going to do that by leveraging quite a bit of experience that, we had accumulated both at Columbia and Harvard over the years, experience in design and developing a heterogeneous system, a chip.
luca: So, a monolithic system, not cheap, so one of the challenges in terms of the research that we are going to do in this task is to extend this experience to a chiplet-based system in a package.
luca: And similarly, we have over 10 years of experience in developing an open-source hardware platform for a system on chip that has been silicon proven, as I'll show you in a minute. And again, we want to extend that to a chiplet-based system. And Jim mentioned about… Antonino mentioned about
luca: whether… which is still unclear which chips we will be able to develop as demonstration in this… as part of this project, but we certainly aim to have quite a bit of prototyping, early prototyping using FPGA's technology, something for, which we have, again, extensive experience.
luca: So experience starts with, on the side of Harvard, with a variety of chips, that, Harvard has designed.
luca: and fabricated and successfully tested over the years, and you'll see them here, and I'm going to focus on one of them, maybe a couple of them, but particularly one of them, which are these EPOX chip that you see at the bottom, EPOX0 and EPOX 1, as they were not designed just by Harvard, but they were designed by the Harvard and Columbia together, as well as IBM.
luca: In particular, I'm gonna tell you, as a demonstration of our approach, this EPOX-1 system on ship, and… which, has been really, the ultimate, deliverable of a, five…
luca: years plus project, supported by DARPA, the Domain Specific System on Ship project, led by IBM Research, and had, Columbia and, Harvard and UIUC Sky partners. And in fact, for the design of this chip, the PhD students at Columbia and Harvard really took the lead. This is a fairly complex chip to be designed in academia, 12 nanometer technology, 64mm square.
luca: It has a fairly sophisticated frequency and voltage.
luca: management, quite a bit of, SRAM memory, and is highly heterogeneous. And we already heard from Jim and others the importance of heterogeneous computing.
luca: It contains, it has a tile-based architecture and contains, 23 different accelerators, which are very important for energy efficient computing, of 14 different types.
luca: And some of them are composed in a cluster, which is the one with the red border there, which has a fairly, quite novel, actually, distributed hardware power management scheme developed at IBM, and for the first time tested on silicon.
luca: And this chip, all the tiles are connected by a network on chip, in fact, a multiple network on chip, 6 planes of 2D mesh, delivering 74 terabits per second.
luca: And this old chip was designed by a fairly small team of mostly PhD students, led by PhD students, a couple of industry researchers, in a period of about 3 months, using our approach, in particular the ESP open source platform for agile SOC design.
luca: This was the ultimate chip for this project. Before that, we had the so-called pipe cleaner, which was this epoch Zero, which was a smaller version. You can see here the impact of our tile-based approach. Again, these are monolithic chips, not chipless, just to be clear.
luca: And as you can go from epoch 0 to epoch 1, you can see, even at a quick glance, that epoch 1 is much more complex in term… along many different metrics, number of accelerators, numbers of tiles, power domains, and so forth.
luca: And in something that is a demonstration of the scalability of our approach to design these integrated circuits, is that essentially the same team of about 8 people, in fact, a junior member replaced a more senior member.
luca: Design a much more complex chip in, essentially the same amount of time, about 3 months.
luca: How that was possible.
luca: with various things, but in particular, when it comes to the design of the components and the RTL, by doing a lot of reuse of open-source intellectual property blocks.
luca: In fact, the whole chip is based on hardware, which is open source, and not all the hardware was designed by the members of the team. In fact, in particular, the four RISC-V processors, which are capable of booting Linux SMP, were designed from, come from an open source project.
luca: from ETH Zurich, the… for NVIDIA Deep Learning Accelerator come from NVIDIA Research.
luca: But the other components, for the most part, were designed by the team, although fairly independently. So Harvard contributed with four accelerators, IBM contributed with one accelerator and the power management scheme, and Columbia really contributed to the accelerator, and the overall glue, if you will, the memory hark in the network on chip, and the approach to bring things together.
luca: So, this was built on top of this open-source platform, ESP, which is this 12-year project that I've been leading at Columbia, and this is a snapshot of the website that I invite you to visit if you are interested.
luca: And there are various things that can be said about ESP, but the things that I think is most important in this context
luca: is how ESP was designed to begin with, with the idea of doing… of promoting collaborative engineering and reuse of design. In fact, even the components that are developed using ESP design flows.
luca: And there are a variety of design flaws, are meant to be designed in a way that are highly reusable.
luca: And when I say a variety of design flow, it's part of the philosophy of ESP that we don't use a particular language, or we don't dictate one language on one CAD tool, and we try to integrate a variety of design flows, starting from different languages, whether we start at RTL or a C++ system C with high-level synthesis to develop accelerator, for instance, for…
luca: AI applications, so we even have domain-specific design flow leveraging other open-source projects, like HLS4ML. And then all these things can be brought together to the library, and through a fairly just push-button approach, we can arrive to FPGA prototyping, and
luca: we could do a chip like that, maybe it wasn't exactly push-button, but in terms of design productivity, I think it was fairly impressive, what we were able to achieve.
luca: So now, as we look at how we want to do things in the code, one of key components is to take this lesson and all this experience and, if you will, chipletize this approach, and we're gonna do it in collaboration with Harvard, that have already started working on a system quite scalable and modular for developed chiplets components, so we're gonna bring the two things together.
luca: In a platform that really is meant to, not only allow,
luca: Do demonstration of design.
luca: with all the various components developing the other two trusts of the code, but also bring together the, some of the open source tools that you have heard more at the architectural and software level. And yes, this particular task that we have is meant to allow integration of this and going towards the physical design.
luca: For the physical design, there will be the second task, 3.2, led by Andrew Kang and Kevin Cowell from UC San Diego and University of Minnesota.
luca: Respectively, and I'm giving the presentation on their behalf. So the overarching challenge here is, again, leverage. Leverage the development of an energy-efficient, domain-specialized design, particularly for DOE scientific application.
luca: And while doing this, in the context of the code, at the same time, making sure that we can also leverage a lot of interaction that many of us, and particularly the leaders of this task, have with other programs, ecosystem, and standards.
luca: And here, again, we have quite a bit of experience that we can leverage in chiplet design and tape-out, in the development of EDA tool and infrastructure, particularly from RTL design all the way to tape-out.
luca: And, in collaboration with, and in leading, effectively, many tasks, particularly in, coming with standards.
luca: in EDA, that, particularly Andrew Kang has. So Minnesota brings, capability in terms of, chiplet prototyping, and these are examples of ongoing projects. San Diego brings, brings,
luca: and really remarkable leadership in developing the open road infrastructure, which is meant to produce a completely open source CAD flow for design advanced integrated circuits. To be clear, the integrated circuits that I showed before, Epoch 1, was designed in part with open source tool, but when it comes to RTL to GDS2, meaning to Synergy and physical design.
luca: it leveraged commercial tools. So one of the challenges that we plan to tackle in the code is to bring together what you have seen with ESPEED and what OpenRoad does to really enable a fully open source design of,
luca: the chiplets, meaning the monolithic chips, as well as a collaboration also with the other.
luca: trust the chiplet, base system in a package.
luca: And in doing that, there is ongoing work in collaboration with SELA in terms of system by finding four chiplets.
luca: And quite a bit of leverage of other projects led particularly by the two copies that lead to this trust from these two NSF Chip Design Hub, and this NSF POS Phase 2 project, which, is meant to support further development of the open road open source ecosystem. So it's really… this is about bringing everybody together, and, in fact, I would say, is a…
luca: very proper lead for the Q&A, because what we are planning to achieve today is to show what we plan to do, and see if we can foster collaboration with all of you.
luca: Thank you.
James (Jim) Ang: All right, thank you, Luca.
James (Jim) Ang: Alright, well, I know that was a whirlwind, but maybe we have a little bit of time here for…
James (Jim) Ang: Questions from any of the audience?
James (Jim) Ang: Maybe I'll just add Luca's comments about a lot of these external collaborations. I recognize that there are linkages across Meerkat with several of the other projects that also have engagement with NSF,
James (Jim) Ang: with, maybe DOD, some of those microelectronics hubs. So there, there's many areas where, we can, I think, join forces and…
James (Jim) Ang: And, speak with a common voice, and within the Meerkat Research Center for, for the need and use of these, harbor prototyping, capabilities.
Paul McIntyre: Yeah, thanks so much. I know Valerie's got her hand up, so I'll try to keep this one quick. First of all, thanks for a great overview. It's very exciting, and great to see how the three thrusts are so well connected with each other in such a logical way.
Paul McIntyre: I, I guess one question I had just sort of…
Paul McIntyre: Cindy are taking a step back,
Paul McIntyre: Democratization of co-design in this, context obviously is very important.
Paul McIntyre: And a lot of the, innovations that are happening, sort of, in the hyperscale regime are, not democratized at all. But, of course, a lot of the really major energy challenges, societally, are at the data center.
Paul McIntyre: level. So, to what extent are the,
Paul McIntyre: the tools that are going to be developed by Decode, likely to…
Paul McIntyre: be, adopted by the bigger players, in the,
Paul McIntyre: In this… in this field, in the… in the data… data-rich, AI computing field, and particularly, you know,
Paul McIntyre: companies like, just recently, you know, in the last couple of weeks, the announcement of, this, alliance between AMD and OpenAI, and they're actually quoting the compute, in terms of the number of gigawatts.
Paul McIntyre: That they're going to be supporting to OpenAI. So it's really the energy… the energy consumption is the new currency, it's not really necessarily the computing power. So with that in mind.
Paul McIntyre: How transferable or how, how… how much, influence would… do you think these tools will have on the broader, ecosystem?
James (Jim) Ang: Yeah, I can share my thoughts, but I'll also invite my leadership team to also add their feedback. For me, Paul, I don't view that our open source tools will ever be, you know, like a replacement for commercial tools. That, when it comes to productization, a lot of the
James (Jim) Ang: a lot of the infrastructure and the expense for using commercial tools will still remain, but but I think there's an important…
James (Jim) Ang: opportunity, especially as we enter this post-Moore's Law era, where… where architectural innovations are… are going to prevail over… are going to have much, much greater potential for impact than, than, innovations based on, you know, improvements in process technology.
James (Jim) Ang: And, I come from the DOE exascale computing project world, where Path Forward and StarForward investments were not meant to lead to the development of new,
James (Jim) Ang: you know, first-of-a-kind product designs. They were meant to actually provide a platform for hardware-software co-design between the national lab software developers, algorithm developers, and the industry, industry developers that were pursuing
James (Jim) Ang: maybe they're analogous to MVPs, again. You know, not products, but actually working prototypes to help, develop a co-design… help provide a foundation for a co-design collaboration.
James (Jim) Ang: So our hope is that what we can do is provide existence proofs for advanced architecture concepts that would be worthy of proceeding to NRE investments. At that stage, I can't speak for DOE, but I think the natural step, as was done in the exascale computing project.
James (Jim) Ang: You would, you would actually look to competitive procurements for, non-recurring engineering investments and follow-on platform procurements.
James (Jim) Ang: That's for large scale, and that's an important step. For edge computing, and many of the DOE application drivers that Meerkat's targeting, we're going to have to think about how low-volume production
James (Jim) Ang: Is actually going to be a viable outcome of our research center, where you'll never attract a commercial market to produce the kinds of microelectronics, edge computing devices that we need to deploy in our experimental user facilities.
James (Jim) Ang: Because there's not enough production volume that would be interesting to NVIDIA and AMD, etc.
James (Jim) Ang: So, let me… before we move on to the other hands for questions, the leadership team, do you have any additional, comments to add for Paul's question?
James (Jim) Ang: Okay, let's…
luca: Nice.
James (Jim) Ang: Luca, go ahead.
luca: No, no, I said that I think you captured the thought, so I think we can ask it.
James (Jim) Ang: Very good, very good.
luca: To the next questions.
James (Jim) Ang: Okay, Valerie, I think you were next, and then Maya.
Valerie Taylor: So, thanks, Jim, and thanks, Luca, Antonia, David. So, it was a really great presentation, and really nice to know a lot more about Decode.
Valerie Taylor: My question has to do with the higher level, so I found it very interesting, too, that you were looking at, the compiler level as well, which I think is great and so much needed.
Valerie Taylor: And, and often overlooked. So, I guess one question that comes up is that around, programming models.
Valerie Taylor: And if you can comment on what you're looking at with respect to programming models, are you taking into account different programming models? Are you looking at having a meta-programming model?
Valerie Taylor: to interoperate with different programming models. Just was wondering,
Valerie Taylor: About your thoughts around the programming models with different applications.
James (Jim) Ang: Sure, sure. And we've done a lot of background work on compiler frameworks and programming models, runtime systems. Maybe I'll actually let Antonino take this question, because…
James (Jim) Ang: He's been very engaged in compiler frameworks for… for over a decade. So, Antonino?
Antonino Tumeo (PNNL): Sure. So,
Antonino Tumeo (PNNL): Long story short, we obviously are going to leverage something that is task-based, and is very amenable on what we can do right now with frameworks that are based on MLIR. One of the opportunities that we have with MLIR is that we already have, actually.
Antonino Tumeo (PNNL): infrastructure, that was listed in the open source tools, right, Comet, that, was developed, started by Bertrand Castor, and is continuing, being developed by other team members. And, the…
Antonino Tumeo (PNNL): The thing that we are using there is leveraging domain-specific languages, because for scientific computing, that's potentially the way to go.
Antonino Tumeo (PNNL): And as we explore opportunities, especially for the new type of accelerators, as you have seen in the presentation, the ideas that we have is leveraging
Antonino Tumeo (PNNL): potentially specific VSTs and specific libraries, right? And obviously, MLIR will enable us to do this, this mapping.
Antonino Tumeo (PNNL): So, there's definitely a lot of opportunities there, there's a lot of discussion that we can have, and
Antonino Tumeo (PNNL): Oh, God.
James (Jim) Ang: And Valerie, you can have a longer, deeper follow-up conversation.
Antonino Tumeo (PNNL): Yes.
James (Jim) Ang: beam on this, okay?
Valerie Taylor: Okay, that would be great, thank you, but enjoyed the presentation.
James (Jim) Ang: Very good. Maya?
Maya: So, I have a question that probably has a lot of different answers, but I know time is short, but it seems as if when you're doing chiplet and chip design that the majority of the time is spent in verification, and I wondered what
Maya: how you were addressing that, or for the MVP concept, that isn't as important as getting the ideas out there.
James (Jim) Ang: Yeah, you know, part of… part of our, approach, I think, is to… is to try and…
James (Jim) Ang: What, maybe… Sideline a lot of the verification, tasks, through rapidly and
James (Jim) Ang: inexpensive lead, creating test hardware so that
James (Jim) Ang: Our verification of, chiplet packaging and chiplet integration
James (Jim) Ang: comes from the use of testing, but maybe this is something Andreas could talk about, how the definition of our chiplet interfaces will also help address the verification challenge.
aolofsson: Yeah, hi Maya. I think that there's a couple of things, that, that we're doing. One is, the open source aspect, of,
aolofsson: Of some of the IP that we're developing, and so, you know, you certainly… there's less verification when you don't have to write code, so when you download, you know, TensorFlow, PyTorch, right, you… somebody else has done the verification, so…
aolofsson: Yes, you do have to worry about how you compose those library functions and create the upper-level logic, but at least somebody has
aolofsson: unit testers and fuzz and verify the lower level stuff, right? So that's a net gain. And in hardware, we generally don't do that. We, you know, we do reuse some IP out there, but for the most part, we write a lot of code
aolofsson: From scratch, and so we can say that verification, but I think the much more important thing is the fact that once you go into chiplets.
aolofsson: you just short-circuit the verification altogether, especially all the physical stuff, the running DRC on the deck, running signal integrity, power resolution. Once you have a die, you have a physical device that you can do a data sheet for.
aolofsson: And then, the verification goes down drastically.
luca: So…
aolofsson: The idea of turning
aolofsson: IP into a hardened physical device is sort of raising the level of abstraction by a lot. Now we just have a box with pins on it, as opposed to,
aolofsson: a white box that you really have to verify every time you tape it out. And we see this a lot with, especially with complex IP, like certies and memory controllers, where even though we license it from big vendors and we pay them millions of dollars, we still have to spend 6 to 9 months before we tape out.
aolofsson: And imagine if instead of that, you had a memory controller chiplet that just worked, and all you had to do was hook up the pins properly, and you're done, you can imagine how much, how much you can save in that. So, that's my… that's my spiel.
James (Jim) Ang: Thank you. Thank you.
Maya: So, let me just quickly say to Andreas, right now, at the OCP thing, ARM is presenting a chiplet… their chiplet interface. But at this moment, it's on their screen, because Ron just sent it to me in a…
aolofsson: I saw that. The more standards, the merrier, they say.
James (Jim) Ang: Yes, I learned from Sadash Shankar that we're in conflict with the…
James (Jim) Ang: For the OCP, workshop today.
James (Jim) Ang: Okay, I think that's all. Any… any last questions or comments from anyone?
James (Jim) Ang: All right, if not, invitation's open to all the Meerkat teams. Feel free to reach out to me, or any of our leadership team here at Decode. We're happy to have follow-up discussions.
Paul McIntyre: Thanks so much, Jim.
Paul McIntyre: Thank you. Thanks, everybody.
luca: Everyone.
James (Jim) Ang: Thanks, everyone, and we will provide our slides.
John Leidel: So, thanks.