What comes after LLMs?
Have you noticed the extreme acceleration in change and technological progress? I definitely have. It’s only been 1 year since ChatGPT was released and by this point, it is already old news and we’re already talking about general intelligence.
I think that, from now on, change will only continue to happen faster. What now takes years will take mere months in the future. Like Ray Kurzweil said: “The whole 20th century is equivalent to 20 years of progress at today’s rate of progress… and because of this exponential growth, the 21st century will be equivalent to 20,000 years of progress at today’s rate of progress,” which is interesting to think about.
Already now, AI can produce breakthroughs in math and science. Recently, Deepmind's AI made 800 years’ worth of discovery. Their AI tool called GNoMe (Graph Networks for Materials Exploration) has discovered 2 million new crystals. These new materials could power future technologies — from computer chips to batteries to solar panels. Then, just some days ago, Deepmind’s FunSearch AI produced new math discoveries.
We are living through a profound moment in the history of technology, and you can see it reflected very well in the stock market. While the S&P 500 is up just 9% this year, NVIDIA’s stock has shot up 240%. Symbotic AI has gone up 210%. Palantir is up 208%. C3.AI is up 149%. Do you know why? Stock prices are actually a projection of how well a given company will perform in the future.
So, today, we will discuss the projection of AI technology. What comes next, after LLMs, after ChatGPT and OpenAI?
How it all started
At first, we trained machines to do classification. We used deep learning to train a computer to classify existing data — images, audio, and text.
Now, we’re in the Generative AI phase, where we can feed input data into a machine and produce new data — e.g. images, video, or text. Which companies are benefiting most from this Generative AI boom? Mostly those who are building AI hardware and those providing cloud-based services. The cloud business is doing great: shares of Microsoft and Google are up around 50% this year, and Amazon is up about 60%.
Now, we are transitioning to the next phase, where our focus will be on how we interact with AI. That’s where business will be focusing in the near future. We can call this phase an “Interactive AI”. At first, we used our hands to interact with LLMs. Now we are already interfacing with it using our voice. Actually, the recent release of the Humane AI pin (a little device you can pin on your shirt) perfectly fits here. It runs a custom GPT-based assistant on a Snapdragon chip. You can not only use your voice to interact with it, but can also take AI with you wherever you go.
That’s just the beginning. The biggest shift will happen when AI is provided with the power to make decisions and take actions in the real world. We’ve already taught LLMs to solve problems with “Tree-of-Thought Prompting”. The second step will be to teach AI to use external tools independent of human involvement. Here we already have tools like Toolformer LLM from Meta, which can use other tools through calling APIs.
Modern technology is fantastic, but it does exactly what you tell it to do. However, from this moment on, LLMs are going to have the freedom to take action on their own. That’s something that has never happened before. This will be a huge shift in what technology is capable of doing. The upside of this is huge.
Physical AI
And continued natural development is taking AI from software into the physical world. We call it physical AI. In this phase, our focus will be on the physical aspect. The idea is to use real-time information from sensors and multimodal LLMs to make decisions and directly implement them in the real world.
One of the most intuitive examples of this trend can be seen in recent developments with robots. At the moment, robots are quite static: they only execute the instructions that I ask them to do. But this won’t be the case with physical AI — these machines will be intelligent and operate autonomously, communicating with other robots and tools.
Before robots can interact with the real world successfully, they need to learn how to move independently or use their hands to work with tools. One of the big recent breakthroughs in this direction is NVIDIA’s Eureka algorithm, which uses GPT-4 LLM to help robots learn faster, 1000 times faster than ever — and that’s a huge development. Instead of making a physical robot try and fail again and again in a lab, they run parallel training sessions in many virtual worlds at once, which massively speeds up the time it takes to train it. It seems that using an LLM and iterative feedback from the environment is the future of AI for robotics.
This trend isn't limited to robotics alone but extends to AI in general. Currently, the trajectory toward artificial general intelligence (AGI) seems to be through iterative deployments — getting feedback from the real world and incorporating it back into the technology.
What comes after LLMs?
It could be Extra Large Language Models, because in technology, we always like to size up our problems :)
One of the main trends is Physical AI — bringing AI into the physical world. Watch an update on Tesla's Optimus — “incredible hardware improvements from the team” .
We will definitely see a lot of exciting progress in Large Behavioral Models, which look for patterns of behavior in the real world captured by a variety of sensors. For more insights on LBM, check out this video featuring Ivan Poupyrev, Founder and CEO of Archetype AI.
I think Small Language Models will be developing at an unprecedented pace in 2024. Like Phi-2 released by Microsoft. It’s just a 2.7 billion-parameter language model, and it has remarkable reasoning capabilities for its size. Also, Apple is putting quite some effort into this space.
Source: Microsoft
So, get ready for a lot of exciting innovations and a lot of cognitive dissonance in 2024.
If you loved this letter, please share it with your friends who might be interested.
Fabulous article covering the most profound change in known earth intelligences, since the arrival of homo-sapiens, 300,000 years ago. It is all so reminiscent of the internet began when many senior people thought it was just a fad. I remember a meeting with our elected representative to the House of Commons, UK. I had recently returned from working at NASA & I suggested to the MP, that the libraries at least needed internet connections. He didn’t really see that they did & thought one terminal would be enough! I see exactly similar thinking in many elected people now & in the UK most of the people who will be directly concerned: Medics, Teachers, Civil Servants etc are like ostrich, assuming that AI is all a fad & means nothing.
I have no idea where all of this leads. We are in a regime that few if any science fiction writers imagined, but there is certainly the prospect of both universal benefits to all or universal misery if it all goes wrong & we end up with an AI dictator worse than any of the monsters from human history.
It is only with the help, writings, video from people like you that we can hope to enter broad sunlit uplands of extraordinary opportunities.
Thank you for all that you do!
"Because of this exponential growth, the 21st century will be equivalent to 20,000 years of progress at today’s rate of progress -- which is interesting to think about."
Interesting yes! Can humans cope with all that progress? We are supposed to be the beneficiaries. Look around and you have to question whether the average human will be able to learn enough to take part. Soon we may have make to human internal augmentation part of our human external technology.