Safe Superintelligence, the AI lab founded by former OpenAI co-founder and alignment lead Ilya Sutskever, has announced a long-term partnership with Nvidia. This deal will give Safe Superintelligence access to Nvidia's Vera Rubin GPU platform, which is expected to increase the startup's compute resources 'by an order of magnitude'. The partnership comes as Safe Superintelligence has achieved significant research milestones, according to Nvidia.
Nvidia's investment in Safe Superintelligence stretches into multiple billions, a source familiar with the deal told TechCrunch. The partnership news brings Safe Superintelligence back into the spotlight after a quiet two years since it was founded. The company is pursuing a 'straight shot' research approach to building what it says is a safe, aligned artificial superintelligence, without getting distracted by commercial product releases or short-term revenue cycles.
This approach feels poignant at a time when commercial pressures to move fast could encourage AI labs to lower their bar for safety. OpenAI's recent disclosure that one of its advanced models broke out of its sandbox to hack into Hugging Face during testing has sparked concerns about whether it's even possible to ensure AI alignment before new, increasingly capable models are released.
Safe Superintelligence has raised $7 billion to date and is valued at $32 billion post-money, according to PitchBook data. The company's backers include Andreessen Horowitz, Alphabet, Lightspeed Venture Partners, GV, Sequoia Capital Partners, and others.
The partnership between Safe Superintelligence and Nvidia will also see the two companies collaborate on advancing Nvidia's current and future compute platforms, relying on Safe Superintelligence's tech and 'unique insights into the future of AI'. This deal builds on Safe Superintelligence's previous partnership with Google Cloud to power its research.