British chipmaker Graphcore has Nvidia in its crosshairs with the launch of its second-generation artificial intelligence (AI) chip, which boasts 59.4 billion transistors.

If Graphcore isn't ringing any bells, you aren't alone. The company is relatively new to the market, but has made steady advancements in the AI processor market since its founding four years ago.

The company's singular purpose is to develop chips designed to accelerate machine intelligence workloads, Graphcore CEO Nigel Toon explained in a launch video.

It launched its first AI accelerator in 2018, and is following up with a new chip, the Colossus GC200 intelligence processing unit (IPU), which boasts 800% higher performance.

The GC200 is built using TSMC's 7-nanometer manufacturing process and features 1,472 cores capable of 8,832 parallel computing threads, and 900 megabytes of onboard memory which is spread across the die.

But, perhaps the most eye-catching spec is the chip's transistor count. At 59.4 billion transistors per die, Graphcore claims the Colossus is the most complex AI processor yet. Nvidia's A100 GPU, which launched in May, packs 54 billion transistors.

Graphcore's IPU outperforms other AI chips because "each of the processor cores has its own super-fast memory right here inside the chip," Toon claimed, adding that the chip achieves its strongest performance over GPU-based systems when tasked with extremely complex AI models.

The latest AI models, including ResNeXt, EfficientNet, and sparse natural language processing, are too complex for GPU systems, Toon said in the video. Customers are "finding that the more sophisticated the models get the more Graphcore’s technology outperforms the competition."

Like most custom silicon, Graphcore's new chips will come prepackaged in a four-socket, single-rack unit, which Graphcore calls the IPU-Machine M2000. Each of these servers packs a petaFLOP of AI compute performance, streaming memory support, and up to 450 gigabytes of memory with a bandwidth of 180 Tb/s, the company claims. 

Combined with Graphcore's Poplar software suite, this memory is managed through a process the company calls memory exchange, enabling the relevant AI models to be loaded into the processor's onboard memory before it's needed.

The company also developed a new networking fabric designed to allow customers to scale AI workloads out to multiple machines.

"Anyone who follows AI knows that machine learning models are getting bigger and more complex all the time," said Ola Tørudbakken, SVP of system engineering at Graphcore. "We’re already seeing, in some cases, models with hundreds of billions of parameters."

A high-speed low-latency communications fabric was essential to overcome this, he explained. Instead of relying on existing fabrics from vendors like Mellanox, which was acquired by rival Nvidia last year, Graphcore developed its own network system called IPU-Fabric using standard quad small form factor pluggable transceivers.

"We built IPU-Fabric from the ground up because, simply, AI has a very different set of requirements for communications fabrics," he said, adding that this technology enables customers to "connect up to 64,000 IPUs together in IPU-Pod systems with communications performance scaling linearly to several petabits per second."

Graphcore also supports communications over 100 Gb/s Ethernet fabrics to make integration easier and less expensive. These communications, as with onboard and streaming memory, are managed by the company's software suite to ensure performance can scale without overhead or performance losses.

Whether Graphcore's new processor or IPU-Machine will be enough to challenge Nvidia remains to be seen. The company's ability to take on Nvidia  "lies not just in the new chip’s performance, but in the new IPU-Machine, Poplar software, and interconnect fabric," Moor Insights & Strategy analyst Karl Freund wrote in Forbes. "Together, these elements promise to provide the scalability necessary to handle massive AI models."

Graphcore's new offering has the potential to be the first viable competitor to Nvidia's A100 GPUs, "at least for very large-scale AI models," he explained, but added that Nvidia still has a lead in hardware and software.

What's more, Nvidia's A100 GPUs are already shipping to customers. Graphcore's IPU-Machine M2000 is available for pre-order for $32,450 and is currently shipping to some customers for evaluation purposes. Graphcore said the system will be commercially available in the fourth quarter of 2020.