Feed: AnandTech
Feed Link: https://www.anandtech.com
---

Title: Tenstorrent Launches Wormhole AI Processors: 466 FP8 TFLOPS at 300W

Date: Fri, 19 Jul 2024 14:30:00 EDT
Link: https://www.anandtech.com/show/21482/tenstorr...

Tenstorrent has unveiled its next-generation Wormhole processor for AI
workloads that promises to offer decent performance at a low price. The
company currently offers two add-on PCIe cards carrying one or two Wormhole
processors as well as TT-LoudBox, and TT-QuietBox workstations aimed at
software developers. The whole of today's release is aimed at developers
rather than those who will deploy the Wormhole boards for their commercial
workloads.

"It is always rewarding to get more of our products into developer hands.
Releasing development systems with our Wormhole� card helps developers scale
up and work on multi-chip AI software." said Jim Keller, CEO of Tenstorrent.
"In addition to this launch, we are excited that the tape-out and power-on
for our second generation, Blackhole, is going very well."

Each Wormhole processor packs 72 Tensix cores (featuring five RISC-V cores
supporting various data formats) with 108 MB of SRAM to deliver 262 FP8
TFLOPS at 1 GHz at 160W thermal design power. A single-chip Wormhole n150
card carries 12 GB of GDDR6 memory featuring a 288 GB/s bandwidth.

Wormhole processors offer flexible scalability to meet the varying needs of
workloads. In a standard workstation setup with four Wormhole n300 cards, the
processors can merge to function as a single unit, appearing as a unified,
extensive network of Tensix cores to the software. This configuration allows
the accelerators to either work on the same workload, be divided among four
developers or run up to eight distinct AI models simultaneously. A crucial
feature of this scalability is that it operates natively without the need for
virtualization. In data center environments, Wormhole processors will scale
both inside one machine using PCIe or outside of a single machine using
Ethernet.

From performance standpoint, Tenstorrent's single-chip Wormhole n150 card (72
Tensix cores at 1 GHz, 108 MB SRAM, 12 GB GDDR6 at 288 GB/s) is capable of
262 FP8 TFLOPS at 160W, whereas the dual-chip Wormhole n300 board (128 Tensix
cores at 1 GHz, 192 MB SRAM, aggregated 24 GB GDDR6 at 576 GB/s) can offer up
to 466 FP8 TFLOPS at 300W (according to Tom's Hardware).

To put that 466 FP8 TFLOPS at 300W number into context, let's compare it to
what AI market leader Nvidia has to offer at this thermal design power.
Nvidia's A100 does not support FP8, but it does support INT8 and its peak
performance is 624 TOPS (1,248 TOPS with sparsity). By contrast, Nvidia's
H100 supports FP8 and its peak performance is massive 1,670 TFLOPS (3,341
TFLOPS with sparsity) at 300W, which is a big difference from Tenstorrent's
Wormhole n300.

There is a big catch though. Tenstorrent's Wormhole n150 is offered for $999,
whereas n300 is available for $1,399. By contrast, one Nvidia H100 card can
retail for $30,000, depending on quantities. Of course, we do not know
whether four or eight Wormhole processors can indeed deliver the performance
of a single H300, though they will do so at 600W or 1200W TDP, respectively.

In addition to cards, Tenstorrent offers developers pre-built workstations
with four n300 cards inside the less expensive Xeon-based TT-LoudBox with
active cooling and a premium EPYC-powered TT-QuietBox with liquid cooling.

Sources: Tenstorrent, Tom's Hardware

Gallery: Tenstorrent Launches Wormhole AI Processors: 466 FP8 TFLOPS at 300W

---
VRSS v2.1.180528