Red Hat CEO Hicks: Open Source ‘At The Center’ Of AI Innovation

“Training techniques that once ran in the hundreds of millions of dollars are now being replicated for a few thousand,” Red Hat CEO Matt Hicks said.

Open-source technology has helped academic researchers in artificial intelligence introduce their work to the world faster and will continue to assist in bringing AI to the masses, Red Hat CEO Matt Hicks said during his keynote address at Red Hat Summit 2024.

“Capabilities that, just a year ago, were coupled to high-end, fairly exotic hardware, can now run on a laptop,” Hicks said during his keynote address. “Training techniques that once ran in the hundreds of millions of dollars are now being replicated for a few thousand. And what's at the center of all this innovation? Open source and academia.”

Hicks used his address to highlight updates to a variety of enterprise open-source tools offered by the Raleigh, N.C.-based vendor and member of CRN’s AI 100 and to drive home the role open-source technology will play in making AI possible and personalized for businesses and users. Red Hat Summit 2024 runs through Thursday in Denver.

Red Hat Summit 2024

“Believe me, every organization is trying to determine what is real for the industry and what is real for them,” Hicks said in his address. “But wherever you are today, Red Hat can meet you there. This is truly an opportunity for each of you, for all of you, to create your AI.”

Red Hat parent company IBM is also investing in the open-source community’s role in AI, with the Armonk, N.Y.-based tech giant revealing this week that it has open-sourced four variations of its Granite code model.

IBM Research explained in an online post that the open-sourced series of decoder-only Granite code models can be used for code generation, fixes, explanations, vulnerability tests, translating legacy codebases such as COBOL into modern languages such as Java, and more. The open-sourced Granite Code Instruct models are available for fine-tuning.

The models are offered under the Apache 2.0 license and adhere to IBM’s AI ethics principles and the legal guidelines for enterprise use. They are trained with 116 programming languages and range in size from 3 billion parameters to 34 billion, according to IBM. The models are available on Hugging Face, GitHub, Watsonx.ai and the newly unveiled Red Hat Enterprise Linux (RHEL) AI offering.

IBM Research positioned the open-sourcing of these models as necessary for enterprise adoption of AI. Models with tens of billions of parameters are useful for generalized chatbots but cost a lot in compute resources to train and run.

“For enterprises, massive models can become unwieldy for more specific tasks, full of irrelevant information and running up high inferencing costs,” according to the post. “Many enterprises have been reluctant to adopt LLMs for commercial purposes for several reasons beyond just the cost. The licensing of these models is often unclear, and how these models were trained, and how the data was cleaned and filtered for things like hate, abuse, and profanity are often unknown.”

On stage, Hicks revealed some of Red Hat’s innovations in AI including a developer preview of Red Hat Enterprise Linux AI, a foundation model platform for developing, testing, and running Granite family large language models for enterprise apps – including indemnification by Red Hat for the LLMs. General availability of RHEL AI is expected “in coming months,” according to Red Hat.

Khurram Majeed, general manager of the Middle East and Africa (MEA) for Systems Limited, a Pakistan-based Red Hat partner, told CRN in an interview that RHEL AI was among the most interesting reveals by Hicks. Like other Red Hat offerings, RHEL AI’s vendor neutrality should help bring in users, Majeed said.

“That will create lot of buzz as this will be cloud native and is not dependent on any hyperscaler,” he said. “Mistral is there, but RHEl AI will take this to the next level and will help organizations to adopt Gen AI based capabilities – even if they are not ready with hyperscaler deployment.”

Red Hat has also launched InstructLab, an open-source project for improving LLM alignment and enabling contributors who lack machine learning expertise. InstructLab promises to enhance LLMs that require less human-generated information and computing resources than typically needed for model retraining.

Here’s what else Hicks had to say.

An Explosion Of AI Choices

The most interesting thing that happened to AI over the last year has been the explosion of choices.

What used to be only achievable in trillion-parameter models is now being replicated in models orders of magnitudes smaller.

Capabilities that, just a year ago, were coupled to high-end, fairly exotic hardware can now run on a laptop. Training techniques that once ran in the hundreds of millions of dollars are now being replicated for a few thousand.

And what's at the center of all this innovation? Open source and academia. Twenty years ago, academic breakthroughs took a long time to make their way into mainstream software.

As a rule of thumb, I would often say – it was about 10 years to never. This was largely because the rate and pace in academia and software were so fundamentally different that it was hard to find just a meaningful intersection.

But over the last decade, academia has embraced open source. And along with it, the ability to not only improve concepts in papers, but in software.

From Breakthroughs To Models

The staggering rate and pace of academic advancements have found the perfect intersection in open source and AI models.

These days, it feels like we go from a breakthrough to a model in a week. The capabilities in AI are being developed by some of the best academic researchers on the planet. And open source gives them a channel for their work.

Red Hat's ‘why’ is that open source unlocks the world's potential. And that's what's happening right now. Because the world's potential isn't limited to software developers.

We are now seeing the impact of broadening the open-source ecosystem in the world of AI. And this isn't going to slow down. As models get smaller, as training gets cheaper, as capabilities grow, the reach of this technology will expand across the planet.

The breakthrough from MIT [Massachusetts Institute of Technology] today will be expanded on by the ITs in India tomorrow.

Researchers will use this as a channel for their next concept. And in turn, they will equip developers with constant new capabilities to build their next idea. Now I am not one [for] trying to predict the future of technology.

But I think this is safe production – AI won't be built by a single vendor. It isn't going to revolve around a single monolithic model. Your choice of where to run AI will be everywhere. And it's going to be based on open source.

InstructLab Seeks To Democratize AI Contribution

When I saw Meta take the first step into the open-source arena with their Llama model, making it accessible to the world, it was a tectonic shift. Then I witnessed Mistral expand on that by releasing a model under the Apache license, allowing unhindered use of what they released.

And all the while, I've watched Hugging Face aggregating the data and models to allow research scientists to iterate on this work. … Red Hat is going to add the next link in this chain of open-source contributions. … And as much progress as we've made in the ecosystem here, the ability to contribute to a model has yet to be solved. I mean, you can get a model from Hugging Face and fine-tune it today. But your work can't really be combined with the person sitting next [to you].

Also, open source has always thrived with a very broad pool of contributors willing to contribute their knowledge. But the barriers to doing a fine-tune of Mistral or Llama 2 without a background of data science have been too high.

We hope to change that. … InstructLab is a new technology to make it simple for anyone, not just data scientists, to contribute to and train large language models. Why is this so important? We believe that to unlock the real potential of AI in your business, you have to be able to close the gap in that last mile of knowledge of your use case.

But how do you activate that knowledge on a foundation model that was literally training on all the information in the world? Now InstructLab provides a method of instruction … on two concepts, knowledge and skills.

And we categorize this data so that we can train models similar to how you or I learn ourselves. Think of knowledge as things you just need to memorize. … Knowing the chapters on addition and subtraction doesn’t mean that you know multiplication and division. … Skills on the other hand represent what you can do with that knowledge. … One plus one equals two and one plus three equals four.

And with just a few examples, combined with knowledge on mathematics, you can teach a foundation model a new skill.

Foundation Models In Business Settings

Extend this to a business setting. Your knowledge might be your historical support cases. And the skill might be how a customer service representative could interact or debug better with that knowledge.

Or your knowledge might be your quarterly results. And your skill is finding the anomalies based on the results that you look for.

Or your knowledge might be marketing copy, and the skill is how you would apply your words, your style, your brand to your website.

Now, this technique does not solve every problem with AI. But I think it solves a really important one. For the class of problems that you can structure in this way, you can now teach a foundation model a new skill with a few simple examples.

We use synthetic data generation in real time to extrapolate on these skills, to give you a result with five examples that before might have taken 5,000.

And with the ability to teach smaller models the skills relevant to your use case, everything gets better. Training costs are lower. Inference costs are lower. Deployment options expand.

These all in turn create options for how you might want to use this in your business. Even when using InstructLab, you will always have the choice to keep your data private and maintain it as your intellectual property.

But I have the feeling that there is a lot of knowledge, a lot of problems, that can be structured in this way that people are willing to contribute to add their own link in the chain of open-source contributions.

Open Sourcing IBM’s Granite Models

But to see their work in action, you also need these skills built into a model. And if anyone hasn’t noticed, GPUs [graphics processing units] can be hard to get right now.

I'm excited to announce that IBM Research and Red Hat are open sourcing the Granite family of the language and code models under an Apache license…. But more important than just the models, these will serve as the upstream for training any open-source contributions made in InstructCloud.

If you choose to contribute your knowledge and your skills, we will put it into a foundation model for you and for the world. And we'll do this in days, not months.

Our goal is to expand the open-source ecosystem around AI one notch further to include those willing to contribute their knowledge and skills – and also close the gap for businesses looking to train smaller open-source models and their needs.

Open Source An AI ‘Force Multiplier’

Some of you might be all-in on AI while some of you are still skeptical. But this technology isn't going away.

And as the open-source element expands, it will be a force multiplier. We need to figure out how to make it work best for you. I believe open source is the best model to be able to do this. But let me try to explain why based on my own experience.

I've said before that I truly fell in love with technology when I was first exposed to Linux and the concept of open source more than 25 years ago. … I started my career off as a Linux admin like many of you. I earned the opportunity to build the products and lead teams, like many of you. And my passion for technology has been reinvigorated over the last couple of years by seeing what AI can do, just like many of you.

In our careers, we are fortunate if we get to be part of one huge technology movement. I feel fortunate to be experiencing the convergence of AI and open source and proud to be contributing to it.

I know that you all are at different points in your AI journey. And that's the good part. There is no set end state or end date. Every organization will take its own path. But the goal is the same – making AI work for you and your unique needs.

These needs could be paired programming, an analytics engine, a chatbot or something even more complex. But believe me, every organization is trying to determine what is real for the industry and what is real for them.

But wherever you are today, Red Hat can meet you there. This is truly an opportunity for each of you, for all of you, to create your AI.

An AI that knows about your business and builds on your internal experience. An AI that enables anyone in your company to have their experience influence the entire company. This is a journey that many of you have taken with us over the years, and it has made open source what it is – the world's best innovation model for software.

Red Hat, along with all of you, made open source a business reality. … Let's show the world what we can do with open-source AI.