Lifting the lid on LLMs

In a casual observation made last year, Robin Li, CEO of Baidu said China is producing one LLM a day. A decades-long quest to build intelligent machines has led us down the road that mimics the human brain, culminating in the advent of the Large Language Model (LLM). Conceptualized in 2017 by researchers at Google and powered by the key scientific development called the transformer model, LLMs form the basis of Generative AI. Businesses across the world are experimenting with Generative AI in the belief that it could transform media, finance, law and professional services, as well as public services such as education. But as enterprises rapidly advance AI strategies by prioritizing the selection of the right foundation models, they often overlook the orchestration of these models. Between sourcing the best ingredients and cooking up a wholesome dish lies the magical cooking processes.

Ah, tension is all you get

AI orchestration mirrors the craft of a chef cooking up a symphony in the kitchen. The process begins with the sourcing of diverse, relevant and high-quality data sets, akin to the careful selection of ingredients. Just as a chef doesn’t work alone but coordinates with sous chefs, line cooks and waitstaff, AI orchestration involves the facilitation of frictionless interactions among frontline actors and components: data scientist, machine learning engineer, NLP expert, ethics & compliance management etc. Examples of components of AI orchestration are plug-ins, which can retrieve real-time information and data from enterprise assets, access control (so that users see only what they are privileged to) and security. Polished, insightful outcomes – all while balancing cost and energy usage – are the fruits of a well-orchestrated AI program. The value-add comes from amplifying the capabilities of the foundation model by incorporating it into the enterprise infrastructure. In other words, AI orchestration serves as an integration bridge between an enterprise’s data assets and applications, retaining conversational memory, linking multiple AI models in a chain, and executing complex tasks that foster continuous learning and operational efficiency. The end goal is the implementation of a friction-free interaction between applications and models.

AI Orchestration – Chief considerations

• Data security and privacy: Protecting enterprise data as it moves and interacts within the orchestrated system is paramount.
• Scalability: The framework must scale with the business, accommodating more models and workflows.
• Complexity: Managing diverse models with unique operational needs is challenging.
• Integration with existing systems: Ensuring easy connection with enterprise systems and avoiding redundant automation code.
• Talent shortage: Finding skilled experts in this emerging field is difficult, akin to choosing a skilled doctor without medical expertise.

LLMs occupy a niche in this intelligent automation continuum, by enabling systems to learn, adapt and evolve almost autonomously. Iterations of LLMs – called expert systems – have been around since the early 1980s. Acting as consultants, they responded intelligently to questions and suggestions, in domains as intricate as medicine. Interacting in some subset of natural language, expert systems based their diagnoses not just on statistical analyses of databases but by an in-depth reasoning, backing up their assertions with details. Range of vision and power that limited them then have virtually no limits now.  

Today, unlike traditional AI models, LLMs thrive on continuous learning from real-time data, enhancing the agility and responsiveness of automated systems. But how much more do LLMs need to learn? What else is out there, yet to be consumed by LLMs? Epoch AI estimates that by 2028, the industry would have hit the “data wall” – when the stock of high-quality textual data on the internet will have all been used. Where LLMs stand out is by minimizing the need for manual re-training and tuning, reducing operational overheads and accelerating decision-making processes. Over time, the performance of systems orchestrated with LLMs is poised to improve as they learn from new data and experiences, making intelligent automation progressively more effective.

For now, an LLM seems like the proverbial hammer with the converts viewing every problem as a nail, even if it blatantly is a screw in need of a screwdriver. While it does undoubtedly have applications, in many cases, the force-fitting of LLMs seems to be almost like a solution in search of a problem. Just as we had to wait for the evolution of 3G and Apps to take full advantage of the iPhone, just as the Internet was born in the 90s when modems and broadband were unknown, the viable utility of LLMs today is limited to very specific areas such as coding and marketing. Fresh, new data sources (Private? Synthetic? Multimodal?), more powerful GPUs and infusing LLMs with more of the deliberate “system 2” thinking, are some of the other pieces that need to evolve, to mainstream its applicability within the broader AI framework. 

And, so…

Going where the puck is going is just fine so long as (a) we know there’s indeed a puck around and (b) it’s within the hockey field. Most enterprises are still seeking convincing business cases for the use of LLMs. Companies pouring resources into much hyped LLMs risk neglecting more proven technologies, advanced analytics and their established value for improving business decisions and processes, such as predicting the next best offer for each customer or optimizing supply chains. Not even the engineers who build these LLMs claim to understand them entirely or to know how to predict or control the outcomes.

Ashwin Ramasamy of Moative Labs, a Partner, sums it up thus: “LLMs are the magic ingredient. They cannot do the entire thing today. Dozen other boring ingredients go into re-engineering the process. But the magic ingredient is a great touch. It's getting the user something that wasn't available before but no one has put a gun to your head and ask that the LLM be 99% of it.” The real promise of AI lies in dealing with sets of data that exist at a scale humans cannot operate at.

Data, the raison d'être: It's data that helped Whirlpool in vaulting its valuation from 2-3x revenues to 10x revenues, shares Andy Walter, Senior VP - IT, Procter & Gamble (Retd) in this exclusive podcast. 

From noise to signal: A 2024 AI Memo
The hardest problem in front of us: Undoing the tech debt from our collective bad habits. The journey involves unraveling sub-optimal, complex, and over-engineered AI and data-centric systems while simultaneously aiming to tame the modern data stack. Dial in.