Tokyo-based artificial intelligence startup Sakana AI, established by former Google researchers, has just unveiled open-source AI models. The key difference in their creations, developers say, is that these models are crafted using an innovative method inspired by evolutionary principles, reminiscent of breeding and natural selection.
Sakana AI logo.
Sakana AI employed a methodology termed “model merging,” which blends existing AI models to generate a novel model, also integrating evolutionary-inspired techniques that resulted in the development of numerous model iterations. The most successful models from each iteration were identified as the “parents” for the subsequent generation.
According to Sakana AI founder David Ha, the company is releasing three Japanese language models, with two models being made available as open-source projects.
The founders of the company, David Ha and Llion Jones, are both former Google researchers. Jones notably contributed to Google’s 2017 research paper “Attention Is All You Need,” which introduced the transformative “transformer” deep learning architecture, laying the groundwork for ChatGPT and fueling the emergence of generative AI-powered products.
Ha, previously holding positions such as head of research at Stability AI and Google Brain researcher, joined Jones in this new endeavor.
Notably, all authors of the groundbreaking Google paper have departed the organization, embarking on new ventures fueled by substantial investment. Among these ventures are AI chatbot startup Character.AI, led by Noam Shazeer, and large language model startup Cohere, founded by Aidan Gomez.
Sakana AI aspires to position Tokyo as a prominent AI hub, following in the footsteps of OpenAI in San Francisco and DeepMind in London. In January, Sakana AI announced securing $30 million in seed financing, led by Lux Capital, signaling robust support for its mission and endeavors.
Written by Alius Noreika