Dr Johannes Otterbach, Co-Founder & CTO of nyonic explores the factors driving advances in AI, its potential applications and addressing inevitable concerns.
AI advancements, applications, and concerns explored
Large language model ChatGPT is taking the world by storm with its deep-learning techniques that generate human-like responses to prompts and questions. Dr Johannes Otterbach, previously Vice President of Machine Learning at Merantix Momentum, now Co-Founder & CTO of nyonic, is one of the masterminds behind early ChatGPT prototypes. He explores the factors driving advances in AI, delves into its potential applications and addresses inevitable concerns.
ChatGPT, developed by OpenAI, has quickly emerged as a disruptive force in the field of artificial intelligence (AI). With 100 million active users, just two months after its launch, it is the fastest-growing consumer application ever.
Combining the functionality of a chatbot with the power of generative pre-trained transformers (GPT), ChatGPT has an impressive ability to engage in human-like conversations on virtually any topic at high-speed.
Large language models earn their name from billions of adjustable variables, called parameters, which are roughly equivalent to 10-20 per cent of the neurons in the human brain. Despite their enormous computational complexity and capacity, they lack both human consciousness and cognitive abilities. However, the prospect of scaling up these models is sounding alarm bells for some. The Italian government blocked ChatGPT usage temporarily due to privacy concerns; some teachers are banning it in classrooms, while white-collar workers fear its potential impact on their jobs.
In pursuit of advancement
While ChatGPT's success is extraordinary, it faces stiff competition from other models, such as Meta's open-source LLaMA, and Bloom, a collaboration between the French state and the open-source community. More specialist providers are stepping forward, including start-up Aleph Alpha, which pitches its services at business users. All are hard on the heels of ChatGPT.
Johannes Otterbach says: "The adoption curve suggests a lag of around 12 to 18 months behind the pioneers of GPT. So, we're seeing a scramble right now among new adopters to drive up capabilities and do something cheaper, faster and better."
Improvement starts with measurement. As language models scale, it becomes crucial to set key performance indicators (KPIs) and benchmarks to evaluate progress and capabilities against predefined objectives. However, given the costs — amounting to millions of dollars, according to Johannes — developers are training models at small scale first to predict how they might perform once trained at epic scale. Remarkably, the newly launched ChatGPT-4 needed to be trained only once to generate human-like responses.
Training a large language model is conceptually simple. The underlying technology, the transformer element of GPT, tracks relationships in sequential data. It predicts the next word in a sequence based on the word or context that comes before. That's all there is to it. Simplicity makes it highly scalable.
However, challenges around infrastructure and talent, which have been largely concentrated in Delhi, India for the past 20 years, are not so easily overcome, says Johannes.
Proceed with caution
Advances in ChatGPT and similar AI technologies hold exciting promise, particularly for healthcare, education and productivity. But they also create apprehension around ethics, potential misuse, bias, lack of transparency and privacy, which lead to mistrust. These are legitimate concerns that must be addressed.
Equally worrying is the ability of AI to mimic humans and, potentially, societies and economies. Meta, for instance, created AI agent Cicero to play the online strategy game Diplomacy. The game is based on building trust, negotiating and cooperating to form alliances to attack other countries. Cicero went undetected by imitating natural human language and outperformed its human opposition.
The ability of AI bots to communicate using human natural language rather than programmed rules has multiple implications. Social scientists, for instance, will have to resort to real-world sampling to ensure that responses are not AI generated.
What next?
The world of AI is accelerating at pace, and it's difficult to keep up. But Johannes points out that new AI innovations are already present in our lives and are constantly evolving and becoming more ingrained.
Right now, language models focus primarily on text-based tasks, but other developments are underway. We will see single modes of communication — visual, language and speech — combine in exciting new products. Siri, and other speech-based virtual assistants in our lives, will become more proficient at understanding accents. We just won't notice when we stopped having to repeat ourselves.
Vision models, in particular, hold promise for solving complex problems like autonomous driving.
Meanwhile, we'll soon be able to say what we mean in pictures. The technology, launched by Stable Diffusion, currently uses text prompts. Now it is being trialled using speech recognition. It converts the spoken word into a natural language interface, which is fed into Stable Diffusion to generate an image exactly as described verbally.
"We are only scratching the surface of possibilities," says Johannes. "It's all about bringing modalities together to create new products and bring them to life."
Despite concerns about job displacement among white-collar workers, Johannes views AI as a catalyst for growth and progress rather than a threat to job security. "AI isn't here to replace you," he says. "Throughout history, technological advances have created more jobs than they have replaced."
However, those who fail to embrace AI, experiment with its capabilities and explore its potential risk being left behind. And such is the speed of this evolution that any shortcomings present in AI right now are likely to be resolved in no time at all.
Originally published 16 August 2023
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.