Ai Chips

3,000 Tokens/Sec Locally: Testing the ZGX Fury: A video tests HP's ZGX Fury, a larger version of the ZGX Nano (DGX Spark). The workstation has 748 GB of memory and NVIDIA's GB300 chipset, with local speeds claimed up to 3,000 tokens per second.

Technology • Sam Witteveen

DeepSeek releases DeepInfer inference framework for Nvidia and Huawei chips: DeepSeek released DeepInfer, its large-scale model inference framework, on October 20. It supports Nvidia GPUs and Huawei Ascend chips, offers distributed inference, speculative decoding and structured output, and aims to improve throughput and latency while cutting computing costs and energy use.

AI Infrastructure • TechNode

Amazon launches TorchNeuron open-source framework for PyTorch on Trainium chips in public beta: Amazon launched TorchNeuron, an open-source framework in public beta that lets developers run PyTorch code on its Trainium chips without rewriting. The tool aims to ease switching from Nvidia to Amazon's AI chips.

AI Development • Techmeme