Generative artificial intelligence models have captured the imagination of many business leaders with the promise of automating and replacing millions of people in their jobs. However, researchers at the Massachusetts Institute of Technology (MIT) warn that AI, while providing plausible answers, does not actually understand complex systems. and limited by predictions In real-world problems, be it logic, navigation, chemistry, or games, AB has significant limitations.

Modern large-scale linguistic models (LLMs) such as GPT-4 give the impression of a reasonable answer to complex user queries, when in fact they only accurately predict the most likely words to be placed next to previous words in a given context.To test whether AB models can to truly "understand" the real world, MIT scientists have developed measures designed to objectively test their intelligence.

One of the goals of the experiment was to assess how well AB was able to generate step-by-step instructions for navigating the streets of New York To improve this, researchers have created formalized methods that allow analyzing how correctly AB perceives and interprets real situations.

MIT's research focuses on transformers, a type of generative AI model used in popular services such as GPT-4. Transformers are trained on vast arrays of textual data, allowing them to achieve high accuracy in word sequence selection and create credible texts.

To further explore the possibilities of such systems, scientists used a class of problems known as Deterministic Finite Automaton (DFA). It includes areas such as logic, geographic navigation, chemistry and even strategy in games.In the experiment, the researchers chose two different tasks: driving a car on the streets of New York and playing "Othello" to test how well AB can correctly understand the rules behind these games.

As noted by Harvard University researcher Keyon Wafa, the main purpose of the experiment was to test the ability of AB models to reconstruct the internal logic of complex systems. "We needed experimental platforms where we knew exactly what the model of the world was. Now we can think rigorously about what it means to restore this model of the world."

Experimental results showed that transformers were able to provide correct routes and suggest correct moves in the Othello game when the task conditions were well defined. routes, suggesting random transitions that didn't actually exist.

The MIT study found that generative AI models have fundamental limitations, especially in tasks that require flexibility of mind and the ability to adapt to real-world conditions predictive tools and not fully intelligent systems.