A significant $400 million (approximately £310 million) loan has been granted to General Compute, an AI inference cloud startup, by investment firm Upper90. This financing deal is notable as it is believed to be the first of its kind to use inference-specific chips as collateral. These chips are designed to run already-trained artificial intelligence models quickly and efficiently, contrasting with the more expensive chips primarily used for developing and training AI models.
This substantial investment signals a potential shift in the AI infrastructure market. As concerns grow over the high cost of advanced AI tools and tokens, investors are increasingly looking towards infrastructure that can run open-source models more affordably than the latest large language models (LLMs) from leading AI labs. General Compute, founded by CEO Finn Puklowski, previously secured a $15 million seed round in May to develop an inference 'neocloud' utilising silicon from SambaNova, an Intel-backed chipmaker. Neoclouds are specialised data centres built specifically for AI workloads, differing from the general-purpose infrastructure offered by major hyperscalers like AWS or Azure.
The SN50 chips, central to General Compute's offering, are engineered for inference tasks. They boast power efficiency and do not necessitate expensive water-cooling systems, enabling quicker deployment across a wider array of data centres. General Compute claims these new chips can deliver inference speeds up to 16 times faster than current GPU-based cloud solutions. Securing a large volume of these advanced chips presents a challenge for a new company, but Upper90's involvement provides a strategic solution.
Upper90's co-founder and CEO, Billy Libby, has a history in this niche financing sector. His firm previously financed GPU purchases for Crusoe, an energy-focused data centre startup, in 2021, a deal he believes was the initial loan backed by advanced chips. Traditional lenders were initially hesitant due to the perceived risks and uncertainties surrounding GPU depreciation. However, as companies like CoreWeave successfully established chip-backed loans as a viable business model, eventually leading to a major IPO, this form of financing has gained wider acceptance.
With GPUs now more widely understood and potentially over-purchased, Upper90 is now focusing on companies like General Compute to capitalise on the next phase of the AI boom. Libby stated that the firm believes open-source models will play a crucial role and that not every entity requires a supercomputer, but all will need inference capabilities. This thesis is supported by the increasing valuations of companies providing access to open models, and new chips from manufacturers like Groq and Cerebras attracting significant market interest, further indicating a diversifying landscape beyond Nvidia's ecosystem.