Even though OpenAI is grabbing a large amount of the hype in the generative AI environment, it isn’t the only seller constructing a big language model (LLM).
Nowadays, Israeli startup AI21 Labs introduced the launch of its latest generative AI design, known as Jurassic-2. AI21 Labs was established in 2017, and introduced its Jurassic-1 Jumbo LLM in 2021, boasting that it had been qualified on 178 billion parameters. The firm raised $64 million in a sequence B funding round back again in July 2022 and is targeted on textual content technology use cases.
With Jurassic-2, AI21 Labs has up to date the teaching data for the product and is aiming to speed up the response instances for era by up to 30%. The corporation is also integrating new abilities that assist a lot more sophisticated directions to enable end users to get extremely custom-made benefits.
AI21 will be integrating Jurassic-2 into its organic language processing (NLP)-as-a-assistance platform, AI21 Studio, as perfectly as by using a collection of APIs for developers to integrate into their individual custom applications.
“Large language versions are magical and they’re very broadly applicable it’s consistently astonishing to see what can be accomplished with them,” Ori Goshen, co-CEO and cofounder of AI21 Labs explained to VentureBeat. “At the identical time, we see some limitations and which is why we commenced the firm — to try out and carry far more reasoning and more semantics into the statistical tactic.”
How A21 Labs is getting a semantic strategy to generative AI
The tactic that a lot of LLMs just take is a statistical product that is capable to infer outcomes dependent on schooling through a equipment learning course of action.
Goshen described that there are some types of processes that do not have a tendency to do the job effectively with a statistical approach to AI. For case in point, essential arithmetic is not realized just by education on examples and then generalizing primarily based on those illustrations. Somewhat, he noted that people discover basic arithmetic by becoming taught regulations, these as the fundamentals of how to conduct addition or subtraction. The target for A21 Labs with Jurassic-2 is to combine semantic reasoning alongside with statistical representation.
The direction is to aid offer what Goshen referred to as a a lot more guided and exact response to a user’s intent with generative AI. For instance, he observed that if a consumer asks the method to make a statistical fact or historical simple fact, it will create coherent textual content but it will also be factual and will cite the resource of wherever the details is coming from.
>>Follow VentureBeat’s ongoing generative AI protection<<
In general, Goshen said that the way to move forward with LLMs and apply them in a productive way for work environments is to have more reliability.
“We’re trying to focus on reading and writing use cases like summarizing text and generating text that is highly guided and reliable,” Goshen said.
You can teach a ‘dinosaur’ new tricks
The term “Jurassic” refers to a geological period in Earth’s history in which dinosaurs were very much active. With Jurassic-2, AI21 Labs is literally teaching its dinosaur-era-named LLM new techniques.
Goshen explained that AI21 Labs had a multiphase approach to building out the Jurassic-2 LLM. The first phase involved a self-supervised approach where the model was trained on a very large corpus of unstructured and unlabeled data. The next phase involved taking a large volume of labeled data to help teach the LLM to be able to follow instructions.
With Jurassic-2, a focus for AI21 Labs was also on more selectively picking the right data to train on.
“There’s a lot of text out there and there’s a lot of repetitiveness on the web,” he said. “So one of the key things we worked on was how to selectively pick examples for the model that actually boost its learning, [which] obviously improves efficiency of training and the performance of the model in general.”
It’s not a MRKL (miracle), it’s just AI
One key approach that isn’t yet in the Jurassic-2 LLM is an implementation of AI21 Labs’ MRKL (pronounced “miracle”) modular reasoning knowledge and language system.
The promise of MRKL is an advanced form of reasoning to help better infer results from an LLM. The company has been talking about its MRKL technology since at least May 2022, when it first demonstrated its Jurassic-X architecture. Goshen said that Jurassic-2 is not implementing MRKL into its architecture at launch, but he hinted that AI21 Labs has some future model releases that will carry forward the spirit of MRKL.
The Jurassic-2 LLM is available to developers via APIs that they can implement, and it’s also part of AI21’s products, including the Wordtune suite of services.
“We don’t just develop our own models. We also serve our applications that are built on top of these models,” Goshen said.
VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact. Discover our Briefings.
