Prompt, LLM… The six words you need to know to understand the generative AI boom

AI towards a common code of conduct between the EU

LLM

Without LLMs, for Large Language Model, no generative AI. The best known are called GPT, and are designed by the American OpenAI. They feed the ChatGPT chatbot. But there are dozens of them, like LaMDA (Google), Claude (Anthropic) or Bloom (BigScience). It is thanks to them that text or image generators look so creative and intelligent. But their operation could not be more mechanical. An LLM owes its ability to build sentences by learning data sets, made up of billions of texts. “An LLM is a bit like a child learning to speak. At first, he listens a lot to what is happening around him. Little by little, he will imitate what he hears, illustrates Wassym Kalouache, co-founder of the IA start-up Corolair, trained at Polytechnique. Similarly, GPT has read many sources, from Wikipedia articles to Harry Potter and learned to make sentences that make sense.”

deep learning

in English deep learning. Current progress in AI stems from this revolution, brought about by neural networks. Since the middle of the 20th century, scientists have tried to reproduce the functioning of the human brain with algorithms. In short, to imitate his ability to distinguish at a glance, thanks to chain reactions, an object from a place or a person. The beginnings are summary and the first program, Perceptron, in 1957, recognizes only certain forms. A clear acceleration took place after the 2010s. The sophistication of networks, operating in several layers, as well as the boom in computing capacities and the contribution of substantial data sets, made these artificial neurons increasingly efficient. Face and voice recognition technologies are born, as well as the powerful translation functions present on our smartphones. The development of LLMs comes on the heels of a new scientific advance. The Transformer architecture, described by Google teams in the article, “Attention is all you need“, published in 2017, endows neural networks with a “contextual sensitivity”, indicates Hugues Bersini, professor of computer science at the Free University of Brussels. This is what allows LLMs today to better understand the requests made to them, and to respond to them by choosing the most appropriate terms by probability.

GAN

THE Generative Adversarial Network, or generative adversarial networks, have been used since 2014 for the creation of ultra-realistic fake images. Their operation is also based on neural networks, but placed in competition, with a generator and a discriminator supposed to determine whether the image is true or false. The first is trained to deceive the second.

Supercomputers

These are the gigantic and very expensive machines by which the great language models are trained. Intense advances in computing over the past ten years, in particular thanks to graphics chips (GPU), have made it possible to build more massive LLMs and therefore, richer in vocabulary. The Jean Zay machine, near Paris, used to develop Bloom, is an example. The most powerful perform more than 1 billion billion operations per second.

ChatGPT

It is a web interface to chat with a language model. Developed by OpenAI, it is the most popular to date: more than 100 million people connect to it monthly. A billion have already tested it. This required, in addition to the LLM, special training, called Reinforcement Learning from Human Preferences (RLHF, in English). OpenAI notably called on workers based in Kenya to polish its tool, eliminate profanity and hate speech. “These humans have the mission, among several answers offered by the chatbot, to rank them from the best to the worst. This, in order to learn to recognize the right one. He then learns to choose it every time thanks to a reward mechanism. Each time he delivers the right formula, he earns a point, otherwise he takes a penalty”, explains Françoise Soulié-Fogelman, eminent expert, scientific advisor to the France IA Hub. This does not prevent certain biases, linked for example to the over-representation of opinions of Western values, from emerging on this platform developed in the United States from a corpus that is mainly English-speaking.

Prompt

This term designates the command formulated to the generative AI application, in natural language (English, French, Spanish, etc.). The limit is set to several thousand words, or tokens, the basic unit for LLM, which are small portions of words.

lep-sports-01