Be a part of our every day and weekly newsletters for the newest updates and unique content material on industry-leading AI protection. Be taught Extra
Liquid AI, a startup co-founded by former researchers from the Massachusetts Institute of Expertise (MIT)’s Laptop Science and Synthetic Intelligence Laboratory (CSAIL), has introduced the debut of its first multimodal AI fashions.
In contrast to most others of the present generative AI wave, these fashions aren’t based mostly across the transformer structure outlined within the seminal 2017 paper “Attention Is All You Need.”
As a substitute, Liquid states that its purpose “is to explore ways to build foundation models beyond Generative Pre-trained Transformers (GPTs)” and with the brand new LFMs, particularly constructing from “first principles…the same way engineers built engines, cars, and airplanes.”
It appears they’ve finished simply that — as the brand new LFM fashions already boast superior efficiency to different transformer-based ones of comparable dimension reminiscent of Meta’s Llama 3.1-8B and Microsoft’s Phi-3.5 3.8B.
Generally known as the “Liquid Foundation Models (LFMs),” these fashions presently are available three totally different sizes and variants:
- LFM 1.3B (smallest)
- LFM 3B
- LFM 40B MoE (largest, a “Mixture-of-Experts” mannequin just like Mistral’s Mixtral)
The “B” of their title stands for billion and refers the variety of parameters — or settings — that govern the mannequin’s data processing, evaluation, and output era. Typically, fashions with the next variety of parameters are extra succesful throughout a wider vary of duties.
Already, Liquid AI says the LFM 1.3B model outperforms Meta’s new Llama 3.2-1.2B and Microsoft’s Phi-1.5 on many main third-party benchmarks together with the favored Huge Multitask Language Understanding (MMLU) consisting of 57 issues throughout science, tech, engineering and math (STEM) fields, “the first time a non-GPT architecture significantly outperforms transformer-based models.”
All three are designed to supply state-of-the-art efficiency whereas optimizing for reminiscence effectivity, with Liquid’s LFM-3B requiring solely 16 GB of reminiscence in comparison with the greater than 48 GB required by Meta’s Llama-3.2-3B mannequin (proven within the chart above).
Maxime Labonne, Head of Publish-Coaching at Liquid AI, took to his account on X to say the LFMs had been “the proudest release of my career :)” and to make clear that the core benefit of LFMs: their capability to outperform transformer-based fashions whereas utilizing considerably much less reminiscence.
The fashions are engineered to be aggressive not solely on uncooked efficiency benchmarks but in addition when it comes to operational effectivity, making them perfect for quite a lot of use circumstances, from enterprise-level functions particularly within the fields of economic companies, biotechnology, and client electronics, to deployment on edge gadgets.
Nonetheless, importantly for potential customers and prospects, the fashions aren’t open supply. As a substitute, customers might want to entry them via Liquid’s inference playground, Lambda Chat, or Perplexity AI.
How Liquid goes ‘beyond’ the generative pre-trained transformer (GPT)
On this case, Liquid says it used a mix of “computational units deeply rooted in the theory of dynamical systems, signal processing, and numerical linear algebra,” and that the result’s “general-purpose AI models that can be used to model any kind of sequential data, including video, audio, text, time series, and signals” to coach its new LFMs.
Final yr, VentureBeat coated extra about Liquid’s method to coaching post-transformer AI fashions, noting on the time that it was utilizing Liquid Neural Networks (LNNs), an structure developer at CSAIL that seeks to make the factitious “neurons” or nodes for reworking knowledge, extra environment friendly and adaptable.
In contrast to conventional deep studying fashions, which require 1000’s of neurons to carry out complicated duties, LNNs demonstrated that fewer neurons—mixed with progressive mathematical formulations—might obtain the identical outcomes.
Liquid AI’s new fashions retain the core advantages of this adaptability, permitting for real-time changes throughout inference with out the computational overhead related to conventional fashions, dealing with as much as 1 million tokens effectively, whereas conserving reminiscence utilization to a minimal.
A chart from the Liquid weblog reveals that the LFM-3B mannequin, as an illustration, outperforms common fashions like Google’s Gemma-2, Microsoft’s Phi-3, and Meta’s Llama-3.2 when it comes to inference reminiscence footprint, particularly as token size scales.
Whereas different fashions expertise a pointy enhance in reminiscence utilization for long-context processing, LFM-3B maintains a considerably smaller footprint, making it extremely appropriate for functions requiring massive volumes of sequential knowledge processing, reminiscent of doc evaluation or chatbots.
Liquid AI has constructed its basis fashions to be versatile throughout a number of knowledge modalities, together with audio, video, and textual content.
With this multimodal functionality, Liquid goals to handle a variety of industry-specific challenges, from monetary companies to biotechnology and client electronics.
Accepting invites for launch occasion and eyeing future enhancements
Liquid AI says it’s is optimizing its fashions for deployment on {hardware} from NVIDIA, AMD, Apple, Qualcomm, and Cerebras.
Whereas the fashions are nonetheless within the preview part, Liquid AI invitations early adopters and builders to check the fashions and supply suggestions.
Labonne famous that whereas issues are “not perfect,” the suggestions obtained throughout this part will assist the crew refine their choices in preparation for a full launch occasion on October 23, 2024, at MIT’s Kresge Auditorium in Cambridge, MA. The corporate is accepting RSVPs for attendees of that occasion in-person right here.
As a part of its dedication to transparency and scientific progress, Liquid says it is going to launch a collection of technical weblog posts main as much as the product launch occasion.
The corporate additionally plans to interact in red-teaming efforts, encouraging customers to check the boundaries of their fashions to enhance future iterations.
With the introduction of Liquid Basis Fashions, Liquid AI is positioning itself as a key participant within the basis mannequin house. By combining state-of-the-art efficiency with unprecedented reminiscence effectivity, LFMs provide a compelling various to conventional transformer-based fashions.
