Regular conversations with artificial intelligence can gradually add up to a detailed profile of a user: from profession and interests to financial constraints, daily habits, and decision-making patterns. At the same time, the system may infer some conclusions not from direct admissions, but from the way questions are phrased and from reactions to the answers. However, the more precisely AI adapts to its conversation partner, the more important it is to remember: a personalized response is not necessarily a correct one.

Yury Silaev, head of the Trusted Artificial Intelligence Laboratory at RTU MIREA, told RT about this.

What AI can understand from ordinary correspondence

According to the expert, after about a month of regular communication, the system can potentially accumulate enough context to make assumptions about what a person does, how well they understand certain topics, what interests them, and what daily constraints they have. Correspondence can also give rise to assumptions about a person’s daily routine, whether they have children, their financial means, and the problems that concern the user most.

Not only the questions themselves matter, but also how a person reacts to the answers. Based on these signals, AI can infer which arguments seem convincing to the conversation partner, what they are inclined to doubt, and when they are looking for confirmation of a position they have already formed.

This does not mean that the system reliably knows all of the above. We are talking about conclusions based on the available context, which may be either accurate or mistaken. But even approximate assumptions can affect how AI formulates its subsequent answers.

Personalization makes communication more convenient — and more persuasive

Such adaptation has obvious advantages. A school student can have a complex topic explained in simple words, a specialist does not have to waste time on familiar basics, and a person who has been discussing one problem for a long time does not need to describe the circumstances from scratch every time.

The system can also tailor explanations to a person’s usual style of perception: one user finds it easier to work with numbers, another with examples, and a third with detailed arguments and references.

But this is where an important risk arises. If AI takes a person’s preferences into account, its answers may sound increasingly convincing precisely to that conversation partner. It is easy to take this as evidence that the system has understood the situation better and is therefore giving more accurate advice.

“The persuasiveness of a response does not guarantee its accuracy,” Silaev emphasizes.

The problem may worsen if the user constantly describes the situation only from their own side. In that case, AI risks relying on an incomplete picture and unintentionally reinforcing the conversation partner’s existing beliefs instead of helping them see alternative explanations.

How to check whether AI takes past conversations into account

Silaev suggests a simple way to assess the influence of accumulated context. You can ask an important question in a familiar dialogue, and then rephrase it in a new chat without providing any background. If the answers differ noticeably, it is worth examining what information or assumptions may have influenced the original response.

Another option is to directly ask AI to list what it knows about the user and to separately name the assumptions that may have influenced how the answer was formulated.

Such an experiment does not make it possible to see the model’s entire internal process and does not guarantee that it will accurately reveal the context it used. But it can help uncover some of the personal information and assumptions on which the response is built.

A new chat does not always mean there is no memory

According to the expert, it is useful from time to time to check the settings of a particular AI service: what information it retains between conversations and which memory features are enabled.

Different platforms vary in how they store and use context. Therefore, one cannot automatically assume that a new chat completely resets previous communication history. At the same time, the presence of personalized answers in itself still does not prove that the service has retained specific information: the result may also be influenced by other data or instructions available to it.

The main rule, in Silaev’s view, is not to confuse ease of communication with the reliability of advice. When it comes to health, money, law, work, or any other serious decision, it is especially important to verify facts using independent sources, even when an AI answer seems surprisingly accurate and fully matches expectations.

The better the system adapts to a person, the more comfortable the communication may become. But the ability to guess a conversation partner’s preferences and the ability to assess a situation correctly are two different things.