How 3D Memories Help Networking ASICs to be Energy Efficient

Powerful but “Green” - How 3D Memories Help Networking ASICs to be Energy Efficient

When I was on stage at the NANOG 49 in San Francisco (Abstract) in June of 2010, explaining the importance of memories to networking, lamenting the slow pace of progress of the memory technologies in the past decade, and urging the memory industry to step up to the plate and break the memory bottleneck, little did I know that the breakthrough was already in the works secretly in a lab in Micron’s Boise, Idaho, headquarters, just 600 miles away. When Micron representatives presented the technology, which eventually led to the creation of Hybrid Memory Cube (HMC), to Juniper later that year, it was like a marriage made in heaven. Both teams immediately recognized the mutual benefits and sprang into action. The rest, as they said, is history.

Fast forward four and a half years. Last week, we announced products and services based on the ExpressPlus and Q5 ASICs, with HMC as the companion memory. With a number of other innovations including virtual output queueing, efficient lookups, and high performance packet filter technology, the ExpressPlus and Q5 ASICs are the industry’s first 500Gbps (1Tbps half duplex) single chip ASICs with large delay bandwidth buffers and large lookup tables, and with breakthrough power efficiency to enable high density systems.

The HMC is made of multiple DRAM “layers” stacked together in a 3-D fashion, communicating with each other and to the base logic layer by ways of Through-Silicon-Vias (TSVs), as illustrated in the following diagram. The TSVs are denser and shorter than the regular wires in conventional DDR3/4 memories, and therefore can support much higher bandwidth with lower latency and lower power. In addition, the base logic layer implements SERDES IOs that are up to 7 times faster than the IOs in conventional memories, which leads to considerably fewer IO pins for the ASICs to communicate with the external memories.

With the ever increasing bandwidth in the network making the new high density systems nearly reaching their power and cooling limits, the power efficiency of the networking silicon is becoming one of the most important factors in satisfying customers’ demands.

In designing these ASICs, we first adopted 28nm technology, the latest generation of silicon fabrication technology, for the manufacturing of our ASICs. With the new generation of technology, we were able to nearly double the amount of logic as compared to the previous generation ASICs with little increase in power.

Next, we integrated logic dispersed across multiple ASICs in our previous generations of products into a single ASIC. Typically, a packet forwarding engine consists of multiple ASICs working together, because of the many external memories the engine needs to connect to and the limited number of IOs each ASIC can have. As illustrated in the following diagram, moving data between the ASICs in this type of system consumes extra power.

With the much faster IO connections provided by the HMC, the division of logic into multiple ASICs was no longer necessary. In the ExpressPlus and Q5 ASICs, we are able to combine all the functions of the packet forwarding engine, including packet buffering and scheduling, congestion management, packet lookups and processing, network interfaces, and switch fabric interfaces, into a single ASIC, eliminating the extra power consumed by moving data between the ASICs.

Finally, we implemented aggressive “clock stopping” design techniques to remove many of the unnecessary activities in the ASICs. Not all circuits are in use at the same time in an ASIC, and in modern ASICs, much of the energy is consumed by circuits “toggling,” that is, the transitions of the circuits by discharging and charging the capacitances. In the ExpressPlus and Q5 ASICs, we dynamically identify the inactive circuits from the active ones, and stop many of the unnecessary “toggles.” This leads to a reduction in overall power, as well as additional power savings when incoming traffic is not running at full speed.

We here at Juniper applaud the breakthrough performance of the Hybrid Memory Cube in multiple dimensions, and we are very proud to bring this technology to the networking market with the ExpressPlus and Q5 ASICs. By leveraging advancements in memory technology, silicon fabrication technology, architectural, and design technique enhancements, we have designed the industry’s first 500Gbps single chip packet forwarding engine with no compromises in packet buffers and lookup tables, and with industry-leading power efficiency of less than 1/2 W per Gbps.

I hope you enjoy the new systems as much as we enjoyed creating them.

(c) http://forums.juniper.net/t5/Silicon-and-Systems/Powerful-but-Green-How-3-D-Memories-Help-Networking-ASICs-to-be/ba-p/270271