Lightelligence Hummingbird Low-Latency Optical Connection Engine

At Hot Chips 2023, Lightelligence showed off its Hummingbird low-latency optical connection engine. We managed to find one outside the auditorium to take a snapshot of for a cover image.

Still, this is being written in real-time so please excuse typos.

Lightelligence Hummingbird Low-Latency Optical Connection Engine

This is yet another table saying that AI compute demand is far outpacing transistor scaling.

Lightelligence is focused on increasing optics in compute. Today, the company is talking about the Optical NOC. This is using a test chip and card that we saw outside, but that is only a part of the company’s portfolio.

The company is saying that creating domain-specific architectures to address specific numeric formats, math, and parallelism is one area of performance improvement.

Another area for increasing performance is adding more silicon to packages.

The interconnects using electrical signals are seen as inefficient.

Here Lightelligence is talking about an Optical Network-on-Chip or oNOC. The idea is the use light instead of electrical signaling on a package to increase efficiency.

It also allows for different types of topologies since the optical waveguides can span longer distances.

It allows better scaling as the distance between chiplets increases. That is important since chiplets can be lower cost to manufacture, but then they need to be connected on packages.

Hummingbird is the example of using the oNOC. It has a FPGA, an external laser, and a 3rd party SiP packaged onto a card. That oNOC allows for things like all-to-all broadcast.

Hummingbird is focused on putting optical connectivity on a package using a photonic integrated circuit or PIC along with electronic silicon chips.

The interposer provides power to the chips. The bottom PIC is the optical component. The EIC is again on the top. That SiP is SIMD architecture with a custom ISA with clusters of eight cores.

The big difference is the optical broadcast network with the U-shaped structure. Using that, every core can update every other core and not have any waveguide crossings in the PIC.

Here is the microarchitecture of the core. This is an AI DSA inference core. Every other core can send its data via the oNOC receive and also each core can be sent data via the oNOC quantize and transmit.

Here are the metrics on the Hummingbird.

Here is a system with Hummingbird in a PCIe GPU chassis.

Here are the performance metrics.

Lightelligence says it can stitch different sizes of chips. It seems like it is focused on building something beyond a rectile limited chip like the NVIDIA H100.

Here are the solutions again:

Final Words

Utilizing 3D packaging will make this technology more interesting. The next question, of course, is whether other companies will look for this for inter-chip communication, or if other types of technologies will take over. This is an interesting technology, but many of the big vendors are also looking at optical technologies.

Hopefully, we will get an update at a future Hot Chips.