WO2026027403A1

WO2026027403A1 - User device including artificial intelligence safety solution

Info

Publication number: WO2026027403A1
Application number: PCT/EP2025/071391
Authority: WO
Inventors: Karin CVETKO VAH; Thomas CARETTE; Francis MEYVIS; Justas ZYDELIS
Original assignee: Sony Europe Ltd; Sony Group Corp
Current assignee: Sony Europe BV United Kingdom Branch; Sony Group Corp
Priority date: 2024-07-31
Filing date: 2025-07-24
Publication date: 2026-02-05
Anticipated expiration: 2027-01-31

Abstract

A user device and method for protecting a user using artificial intelligence (AI) solution. An AI safety module in the user device monitors communications of a user, receives a direct input from the user, and/or receives telemetry data measured on the user and/or surroundings of the user from a telemetry device. The AI safety module detects a threat to the user or a need for help for the user based on the communications of the user, the direct input received from the user, and/or the telemetry data. The AI safety module generates an action based on the detected threat or the need for help. A large language model may be used for processing the communications of the user and the threat to the user or the need for help may be detected based on an input received from the large language model.

Description

User device including artificial intelligence safety solution

Field

[001] Examples relate to a user device including artificial intelligence (Al) safety solution, more particularly a method and user device for protecting a user from scams, fraudulent transactions, or other threats and providing help to the user in dangers and predicaments using Al safety software tool.

Background

[002] As the world is becoming increasingly complex and new forms of communications are evolving, scams and frauds are issues in the modern society. Some people (e.g., elderly people, young people, or those who feel left out and lonely, etc.) are more vulnerable to the scams and frauds and could easily become victims to the scammers and fraudulent transactions. Some people might not fully understand and follow up with the fast-paced modem society and might get confused in unfamiliar situations.

[003] Scammers are constantly finding new ways to steal money from people and people are exposed to an increasingly broad spectrum of threats. Potentially, anyone could become a victim of scams or frauds. Scammers may approach people in sophisticated and unprecedented ways, such as via email demanding their bank accounts and passwords, or by phone using artificial intelligence (Al)-generated deep fake audios sounding like their loved ones asking for help.

[004] Therefore, it would be desirable to provide means to protect people from scams and fraudulent transactions and provide help to those in dangers and predicaments.

Summary

[005] An example relates to a user device for protecting a user. The user device includes an input/output (VO) interface, and a processor. The user device may further include a telemetry device(s). The I/O interface is configured to receive an input and generate an output. The processor is configured to run an Al safety module. The Al safety module is configured to monitor communications of a user, receive a direct input from the user via the VO interface, and/or receive telemetry data measured on the user and/or surroundings of the user from a telemetry device. The Al safety module is further configured to detect a threat to the user or a need for help based on the communications of the user, the direct input received from the user, and/or the telemetry data, and generate an action based on the detected threat or need for help. [006] The Al safety module may be configured to use a large language model for processing the communications of the user and detect the threat to the user or the need for help based on an input received from the large language model. The Al safety module may be configured to take a first level of action after detecting the threat or the need for help and take a second level of action if the first level of action does not remove the threat or satisfy the need for help. In some examples, the Al safety module may be configured to detect a stress level of the user based on the telemetry data and take the first level of action and the second level of action based on the detected stress level of the user.

[007] The first line of action may include at least one of interacting with the user to confirm a cause of the threat or the need for help or a status of the user, disrupting the communication of the user, providing a warning of potential scam or danger to the user, providing follow-up questions that the user can ask a caller, or providing advice to the user. The second line of action may include at least one of providing advice, warning, or follow-up questions to the user, sending a warning to a pre-determined person or contact, calling an emergency, terminating a call of the user, or requesting for blocking bank transactions. The Al safety module may be configured to generate the actions via a trusted embodiment of the Al safety module.

[008] Another example relates to a method for protecting a user. The method includes by an Al safety module, monitoring communications of a user, receiving a direct input from the user, and/or receiving telemetry data measured on the user and/or surroundings of the user from a telemetry device. The method further includes detecting, by the Al safety module, a threat to the user or a need for help for the user based on the communications of the user, the direct input received from the user, and/or the telemetry data, and generating, by the Al safety module, an action based on the detected threat or the need for help.

[009] A large language model may be used for processing the communications of the user and the threat to the user or the need for help is detected based on an input received from the large language model. A first level of action may be generated after detecting the threat or the need for help and a second level of action is generated if the first level of action does not remove the threat or satisfy the need for help. In some examples, a stress level of the user is detected based on the telemetry data and the first level of action and the second level of action are generated based on the detected stress level of the user.

[0010] The first line of action may include at least one of interacting with the user to confirm a cause of the threat or the need for help or a status of the user, disrupting the communication of the user, providing a warning of potential scam or danger to the user, providing follow-up questions that the user can ask a caller, or providing advice to the user. The second line of action may include at least one of providing advice, warning, or follow-up questions to the user, sending a warning to a pre-determined person or contact, calling an emergency, terminating a call of the user, or requesting for blocking bank transactions. The actions may be generated via a trusted embodiment of the Al safety module.

[0011] Another example relates to a machine-readable medium including code, when executed, to cause a machine to perform the method for protecting a user in accordance with the examples disclosed herein.

Brief description of the Figures

[0012] Some examples of apparatuses and/or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which

[0013] FIG. l is a block diagram of an example user device including an Al safety module;

[0014] FIG. 2 shows an example of the user’s stress level detected by the Al safety module;

[0015] FIG. 3 shows two action triggers based on the detected stress level of the user;

[0016] FIG. 4 shows an example flow for a protection of the user by the Al safety module; and

[0017] FIG. 5 is a flow diagram of an example process for protecting a user.

Detailed Description

[0018] Various examples will now be described more fully with reference to the accompanying drawings in which some examples are illustrated. In the figures, the thicknesses of lines, layers and/or regions may be exaggerated for clarity.

[0019] Accordingly, while further examples are capable of various modifications and alternative forms, some particular examples thereof are shown in the figures and will subsequently be described in detail. However, this detailed description does not limit further examples to the particular forms described. Further examples may cover all modifications, equivalents, and alternatives falling within the scope of the disclosure. Like numbers refer to like or similar elements throughout the description of the figures, which may be implemented identically or in modified form when compared to one another while providing for the same or a similar functionality.

[0020] It will be understood that when an element is referred to as being “connected” or “coupled” to another element, the elements may be directly connected or coupled or via one or more intervening elements. If two elements A and B are combined using an “or”, this is to be understood to disclose all possible combinations, i.e. only A, only B as well as A and B. An alternative wording for the same combinations is “at least one of A and B”. The same applies for combinations of more than 2 elements.

[0021] The terminology used herein for the purpose of describing particular examples is not intended to be limiting for further examples. Whenever a singular form such as “a,” “an” and “the” is used and using only a single element is neither explicitly or implicitly defined as being mandatory, further examples may also use plural elements to implement the same functionality. Likewise, when a functionality is subsequently described as being implemented using multiple elements, further examples may implement the same functionality using a single element or processing entity. It will be further understood that the terms “comprises,” “comprising,” “includes” and/or “including,” when used, specify the presence of the stated features, integers, steps, operations, processes, acts, elements and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, processes, acts, elements, components and/or any group thereof.

[0022] Unless otherwise defined, all terms (including technical and scientific terms) are used herein in their ordinary meaning of the art to which the examples belong.

[0023] Examples are disclosed herein for a user device including artificial intelligence (Al) solution for protecting a user or providing help to the user. The Al solution (Al safety software module) in accordance with the examples disclosed herein can protect a user from scams, fraudulent transactions, or other threats and provide help to the user in dangers and predicaments. The Al solution is configured to help the user navigate constantly changing everyday situations and keep the user safe and provide support/help and explain context of complex situations to the user when needed.

[0024] FIG. 1 is a block diagram of an example user device 100 including an Al safety module for a user. The user device 100 may be a mobile device or a wearable device, such as a smartphone, a smartwatch, or any device that the user may carry or wear. In some examples, the user device 100 may have a physical shape of a necklace, a wrist band, a humanoid robot, a robotic puppy, or any shape. The user device 100 including the Al safety solution may be referred to as an Al safety companion. The Al safety companion (the user device 100) is an interactive tool that the user can interact with.

[0025] The user device 100 includes a processor 110, an input/output (I/O) interface 120, and a telemetry device 130 (optional). The processor 110 is configured to run an Al safety module 112. The Al safety module 112 is a software module that is configured to implement the safety mechanisms for protecting the user from scams, fraudulent transactions, or the like and providing help to the user in dangers and predicaments in accordance with the examples disclosed herein. The Al safety module 112 is configured to monitor communications of a user, receive a direct input (voice, text, etc.) from the user via the I/O interface 120, and/or receive telemetry data measured on the user or surroundings of the user from the telemetry device 130, and implement the safety mechanism based on the communications of the user, the direct input, and/or the telemetry data.

[0026] The Al safety module 112 (e.g., a machine-learning model) may be a data structure and/or set of rules representing a statistical model that the processor 110 uses to protect a user from scams, fraudulent transactions, or other threats and provide help to the user in dangers and predicaments without using explicit instructions, instead relying on models and inference. The data structure and/or set of rules represents learned knowledge (e.g., based on training performed by a machine-learning algorithm as described below). In machine-learning, instead of a rule-based transformation of data, a transformation of data may be used, that is inferred from an analysis of training data.

[0027] The Al safety module 112 may be a trained machine-learning model. The Al safety module 112 (e.g., a machine-learning model) is trained by a machine-learning algorithm. The term "machine-learning algorithm" denotes a set of instructions that are used to create, train, or use a machine-learning model. For the machine-learning model to protect a user from scams, fraudulent transactions, or other threats and provide help to the user in dangers and predicaments, the machine-learning model may be trained using training data as input and as target output. For example, the training input data to the machine-learning model may include specific values of stress levels and other bodily functions, voice inputs, voice anomaly measurements, facial images obtained from a camera, user’s conversation, or the like, while the training output data from the machine-learning model may be binary such as scam/not-scam or certain actions (e.g., making an emergency call, or providing an advice, or any desired action(s) corresponding to the training input data). Moreover, the training input data may also include data monitoring the surroundings of the user, such as footages from cameras and detecting a threat, need for help, or emergency situation there, and the training output may be certain corresponding actions. These training data is merely some examples and any other training data may be used to train the model, and these training data may be used for any training algorithm. By training the machine-learning model with a large set of training data and associated training content information, the machine-learning model "learns" to protect a user from scams, fraudulent transactions, or other threats and provide help to the user in dangers, needs, and predicaments in the training data. By training the machine-learning model using the training information, the machine-learning model "learns" a transformation between the input training data and the desired output, which can be used to provide an output based on non-training data (actual data in real life) provided to the machine-learning model.

[0028] The machine-learning model may be trained using training input data as provided above. For example, the machine-learning model may be trained using a training method called "supervised learning". In supervised learning, the machine-learning model is trained using a plurality of training samples, wherein each sample may comprise a plurality of input data values, and a plurality of desired output values, i.e., each training sample is associated with a desired output value. By specifying both training samples and desired output values, the machine-learning model "learns" which output value to provide based on an input sample that is similar to the samples provided during the training. For example, the input to the machinelearning model (the Al safety module 112) may include specific values of stress levels and other bodily functions, while the output from the machine-learning model may be binary such as scam/not-scam or certain corresponding actions, as explained above. Moreover, the input may also include data monitoring the surroundings, such as footages from cameras and detecting a threat there and the output may be certain actions as explained above.

[0029] Apart from supervised learning, semi-supervised learning may be used. In semisupervised learning, some of the training samples lack a corresponding desired output value. Supervised learning may be based on a supervised learning algorithm (e.g., a classification algorithm or a similarity learning algorithm). Classification algorithms may be used as the desired outputs of the trained machine-learning model restricted to a limited set of values (categorical variables), i.e., the input is classified to one of the limited set of values. For example, based on features measured by the sensors, user can be assigned to a group of similar users within the training data. Based on that, their appropriate values of bodily functions that indicate high stress can be identified. Similarity learning algorithms are similar to classification algorithms but are based on learning from examples using a similarity function that measures how similar or related two objects are. [0030] Apart from supervised or semi-supervised learning, unsupervised learning may be used to train the machine-learning model. In unsupervised learning, (only) input data are supplied, and an unsupervised learning algorithm is used to find structure in the input data such as training physical properties of the user (e.g., by grouping or clustering the input data, finding commonalities in the data). Clustering is the assignment of input data comprising a plurality of input values into subsets (clusters) so that input values within the same cluster are similar according to one or more (pre-defined) similarity criteria, while being dissimilar to input values that are included in other clusters.

[0031] Reinforcement learning is a third group of machine-learning algorithms. In other words, reinforcement learning may be used to train the machine-learning model. In reinforcement learning, one or more software actors (called "software agents") are trained to take actions in an environment. Based on the taken actions, a reward is calculated. Reinforcement learning is based on training the one or more software agents to choose the actions such that the cumulative reward is increased, leading to software agents that become better at the task they are given (as evidenced by increasing rewards).

[0032] Furthermore, additional techniques may be applied to some of the machine-learning algorithms. For example, feature learning may be used. In other words, the machine-learning model may at least partially be trained using feature learning, and/or the machine-learning algorithm may comprise a feature learning component. Feature learning algorithms, which may be called representation learning algorithms, may preserve the information in their input but also transform it in a way that makes it useful, often as a pre-processing step before performing classification or predictions. Feature learning may be based on principal components analysis or cluster analysis, for example.

[0033] For example, the machine-learning model may be an Artificial Neural Network (ANN). ANNs are systems that are inspired by biological neural networks, such as can be found in a retina or a brain. ANNs comprise a plurality of interconnected nodes and a plurality of connections, so-called edges, between the nodes. There are usually three types of nodes, input nodes that receiving input values (e.g., specific values of stress levels and other bodily functions), hidden nodes that are (only) connected to other nodes, and output nodes that provide output values (e.g., scam or not scam). Each node may represent an artificial neuron. Each edge may transmit information from one node to another. The output of a node may be defined as a (non-linear) function of its inputs (e.g. of the sum of its inputs). The inputs of a node may be used in the function based on a "weight" of the edge or of the node that provides the input. The weight of nodes and/or of edges may be adjusted in the learning process. In other words, the training of an ANN may comprise adjusting the weights of the nodes and/or edges of the ANN, i.e., to achieve a desired output for a given input.

[0034] Alternatively, the machine-learning model may comprise a different structure and, e.g., be a support vector machine, a random forest model or a gradient boosting model. Alternatively, the machine-learning model may be based on a genetic algorithm, which is a search algorithm and heuristic technique that mimics the process of natural selection.

[0035] In some examples, the machine-learning model may be a combination of the above examples.

[0036] The I/O interface 120 is configured to receive an input and generate an output. The user may communicate with the Al safety module 112 via the I/O interface 120 (e.g., in natural language). For example, the I/O interface may include a speaker, a microphone, a keyboard, a touch screen, and/or a touch pad, etc. so that the user may communicate with the Al safety module 112 by speech through microphones and speakers included in the user device or connected to another user device such as hearing aids, or by text or input using a touch screen or keyboard, or combination thereof.

[0037] The Al safety module 112 may receive the telemetry data from the telemetry device(s) 130. In some examples, one or more telemetry devices 130 may be included in the user device 100. Alternatively, the one or more telemetry devices 130 may be separate from the user device 100 and the Al safety module 112 may receive the telemetry data from one or more external telemetry device(s) via the I/O interface 120.

[0038] The telemetry device 130 may monitor the user’s well-being as well as the surroundings of the user and send the telemetry data to the Al safety module 112. As an example, the telemetry device 130 may be, but is not limited to, a camera, a Global Positioning System (GPS) device, a fall detection device for detecting a fall of a user, a personal emergency response device, a health monitoring device, and/or various sensors tailored to monitor and track a wide range of physiological and biological functions of the user. For example, the sensors may be a heart rate monitor for measuring the pulse of the user, an accelerometer to track movement and activity levels of the user, a temperature sensor for monitoring body temperature of the user and the ambient temperatures, a galvanic skin response sensor to measure stress levels of the user through skin conductivity, a sleep tracker for assessing sleep quality and duration of the user, an oximeter for measuring blood oxygen saturation of the user, a blood pressure monitoring sensor, or the like.

[0039] The Al software module 112 (Al solution) is configured to detect and assess threat (danger, predicament, anomaly, etc.) to the user or a need for help based on the communications of the user, the direct input received from the user, and/or the telemetry data (e.g., images of the user or surroundings of the user from a camera, or physiological or biological sensing data from sensors, etc.) received from the telemetry device 130, and generate an action based on the detected threat or need for help. For example, the Al software module 112 may rely on a large language model (LLM) for processing natural language (both the input from the user and the output to the user) in combination with a classification model for processing various detected features in detecting the threat. For example, the communication of the user may be sent to the large language model (e.g., in the cloud) and the Al safety module may detect, in real time, the threat to the user or the need for help or take an action based on an input/suggestion from the large language model. The conversation of the user may be understood by processing the communications of the user using the large language model, and the Al safety module 112 may detect and assess a threat (danger, predicament, anomaly, etc.) or a need for help in real time, and take an action based on the understanding of the communication of the user. The LLM may be a part of the Al safety module/solution in the user device 100. Alternatively, the LLM may be running on a cloud and the Al safety module may utilize the LLM via network connection. [0040] Large language models are Al systems capable of understanding and generating human language by processing vast amounts of text data. Large language models are trained on immense amounts of data to understand and generate natural language and other types of content to perform a wide range of tasks. . The LLM acts as a safety companion with appropriate advise and action essentially due to the following: 1. It uses a technology called role-playing that has been described in research literature and implemented. Role play in LLMs has been introduced in the paper Role-Play with Large Language Models by M. Shanahan, K. McDonell and L. Reynolds, (https://arxiv.org/pdf/2305.16367) The technology used for role-playing in dialogue agents as described in the paper "Role-Play with Large Language Models" involves leveraging large language models trained on vast corpora of human-generated text. Dialogue agents, when prompted, can mimic human language use by adopting roles that align with the context of the conversation. The role is refined as the dialogue progresses, similar to an improvisational actor who adapts to the ongoing scene. 2. It has been fine-tuned in production of the device (Al Safety Companion) to be able to do this task, i.e., be able to detect the threat, interact with the user and alert them. 3. The Al engine is monitored, and external feedback is integrated, where external feedback can refer to the user or a monitoring person (i.e. remote operator) that could intervene if needed. External feedback will be mostly about false positive, which is why we also want the system to fine tune an adversarial network generating scam related datapoints for the Al to use in order to stay vigilant. This network can be trained on real life scam datapoints using personalized federated learning.

[0041] Protection of the user is the core function of the Al safety module 112. The Al safety module 112 detects when the user is in stress or in threat, danger, or predicaments or in need of help, and can take certain actions. For example, the Al safety module 112 may provide advice to the user. In certain situation or setting, the Al safety module 112 can also take further actions, such as contacting the user’s trusted person, calling an emergency or an authority, etc. [0042] In some examples, the Al safety module 112 can monitor the stress level of the user and take an action based on the detected stress level of the user. FIG. 2 shows an example of the user’s stress level detected by the Al safety module 112. For example, the stress level of the user may be detected based on the signals from the body function sensors (e.g., a galvanic skin response sensor) in a user device worn by the user, the change of the voice or tone of the user detected through microphones/ speakers installed in or connected to the user device, images of the user (e.g., facial images) obtained by a camera, or based on a direct input from the user (e.g., voice, text, etc.), or the like. In the example in FIG. 2, the user’s stress level increases as the user receives a call from a scammer and remains high and then decreases a bit as the victim’s reasoning is off. The Al safety module 112 detects the increased stress level of the user and may take an action at certain point (e.g., when the stress level goes above a threshold). In examples, the Al safety module 112 includes a stress detector that processes an array of inputs such as bodily sensors data, images from cameras, speech monitoring, etc. and measures the stress level of the user.

[0043] In some examples, the Al safety module 112 may be activated by two (or more) action triggers. FIG. 3 shows, as an example, two action triggers based on the detected stress level of the user. The Al safety module 112 may determine that the user is under a severe stress (threat/danger/predicament/anomaly). For example, the user might have fallen on to the floor, or the health condition of the user might be in danger, or the user is under stress due to a call from a scammer, etc. In those situations, the Al safety module 112 may detect the increased stress level of the user and take an action accordingly. When the Al safety module 112 detects an abnormally high stress level of the user but not yet extremely high (e.g., the stress level is higher than a first threshold but below a second threshold), the first line of action(s) may be triggered. The thresholds need to be calibrated per user in the early stage of user-companion interactions. Calibration takes place during the period when the two are building connection. This is done using state-of-the-art calibration methods of stress detection and anomaly detection. For example, in FIG. 3, the user’s stress level is increasing, and once the user’s stress level goes above the first threshold 302, the Al safety module 112 may take the first line of actions. In some examples, the threshold(s) may be set according to a binary classification problem like scam/not-scam. In production, action(s) may be defined and threshold(s) may be set according to the actions that are programmed. For instance, a cost of false positive/false negative can be set and threshold may be defined according to that costs.

[0044] As the first line of actions, as an example, the Al safety module 112 may assess the situation (e.g., based on the input provided by the sensors/telemetry devices and using the inbuilt Al system), may enter into direct interaction with the user to confirm the cause of the stress elevation or status of the user, and/or provide help if needed to prevent further harm. For example, if the Al safety module 112 detects that the user might have fallen, the Al safety module 112 asks for confirmation that the user is OK and whether it should call for help. If the Al safety module 112 detects that the stress might be coming from user’s interaction with another person, the Al safety module 112 may try to provide context to the user, help the user continue the communication and potentially warn the user of a scam or other danger/threat. For instance, if the Al safety module 112 suspects the user received a call from a scammer, the Al safety module 112 may provide follow-up questions the user can ask the caller to verify the caller’s identity and to determine whether the call is genuine. An LLM may be a part of the Al safety solution and the follow-up questions may be generated by the LLM that may be running on a cloud.

[0045] If the first line of action(s) was not successful, the user’s stress level might continue to rise beyond the first threshold, reaching extremely high levels. Depending on the settings and the Al assessment of the situation, the Al safety module 112 may now act more aggressively (take second line of actions). For example, the Al safety module 112 may send a warning to a pre-set user’s trusted person, a police, or an emergency contact, terminate a call of the user, request for blocking a bank transaction, or the like. Similar actions may be taken if the user is not responding to the Al safety module 112 trying to reach the user. The second line of actions may be triggered when the stress level goes beyond a second pre-defined threshold. Different types of actions may be taken depending on the actual conditions. The action triggers can be stress, situation, location, time of day, etc.

[0046] FIG. 4 shows an example flow for a protection of the user by the Al safety module 112. The user 402 is using a user device or a set of user devices (e.g., smartphone(s), smartwatch(es), tablet(s), wristband-type device(s), necklace-type device(s), a humanoid robot, a robotic dog(s) or pet(s), or any mobile or wearable device(s)). The Al safety module 112 is included in one or more of the user devices. FIG. 4 shows an example that the user 402 uses two devices 410 and 420, and one user device 410 includes a communication application, and the other user device 420 includes the Al safety module 112. Alternatively, the two user devices 410 and 420 may be a single device.

[0047] An attacker 404 attempts to communicate with the user 402 using a communication channel (e.g., voice communication via a mobile phone) open on the user device 410 (452). The user 402 communicates with the attacker 404 using the communication application on the user device 410 (454).

[0048] The Al safety module monitors the communication channel/application on the user device 410 and detects threat/anomaly on the communication (456). For example, the Al safety module may receive telemetry data from telemetry devices, the communication of the user, and/or a direct input from the user and detect a threat (danger, predicament, anomaly, etc.) to the user or a need for help for the user. For example, the threat/need for help may be detected based on the stress level of the user obtained from a sensor for detecting the stress level, hand shaking of the user obtained from an accelerometer in the user device 410, voice anomaly measurements, facial image of the user obtained from a camera, conversation transcription of the communication by a large language model, guess on a type of scam, etc.

[0049] At the alert level 1 (e.g., the stress level of the user above a first threshold), the Al safety module may take a first line of actions. For example, the Al safety module may use a trusted communication channel to the user (e.g., generating an audio output) to disrupt the user’s communication with the attacker or provide advice or help (458).

[0050] The Al safety module in the user device 420 may take the action directly to the user 402 (the communication 458 may be directly from the user device 420 to the user 402). Alternatively, the Al safety module in the user device 420 may take the action via a trusted embodiment 430 of the Al safety companion (the communication 458 is from the device 430 to the user 402). The trusted embodiment 430 of the Al safety companion is another user device that the user may carry or be with. For example, the trusted embodiment 430 of the Al safety companion may be a robotic puppy or similar device that the user can interact frequently or any device that can generate an output to the user (e.g., a speaker in the user’s home). In order for the Al safety module to be successful in preventing the user from being scammed, it may be needed that the Al safety module has previously gained user’s trust. Indeed, if the scammer manages to get the user emotionally involved, then the user will only listen to the Al safety module provided that a deep connection was previously established. The Al safety module should thus be a trusted and reassuring companion that, for example, the user can chat to on a daily basis when lonely or ask for a simple explanation when needed or in doubt. For example, the Al safety module (112) may obtain a trust from the user by providing one or more functionalities (user support functions) to the user and having interactions with the user over time. The functionalities (user support functions) provided by the Al safety module/companion may be any functionalities, such as providing an advice or warning in case fraudulent transactions or scams are detected, providing an assistance in emergency situations, or the like. The Al safety module/companion may take certain roles/personalities so that the user may have lots of interactions (e.g., daily interactions) with the roles/personalities, and via those interactions, a user’s trust may be built on the Al safety module/companion. The Al safety companion may gain user’s trust as its companion, e.g., during their everyday interactions. This trust may be built in the initial phase of their “relationship”. Since the Al safety companion is not just a fall or scam detection device, but is also a companion, a trust can be built. For example, in everyday situation, a role-playing mode of LLM may be employed to conduct conversation with the user about everyday matter and the user may get trust in the Al safety companion over a period of time. The user device 420 establishes a connection to the trusted embodiment 430 of the Al safety companion and generates the action (e.g., voice output) to the user via the trusted embodiment 430 of the Al safety companion.

[0051] At the alert level 2 (e.g., the stress level of the user beyond a second threshold), the Al safety module may take a second line of actions (460). For example, the Al safety module may take aggressive blocking actions to disrupt the attack (e.g., call emergency contact, end the call, request for blocking bank transactions, or the like).

[0052] Following is an example that the Al safety companion intervenes and prevents the user from being scammed by a scammer. There is a phone conversation happening between an elderly lady and a scammer who is pretending to be her grandson that got into trouble while traveling abroad. The Al safety companion intervenes and prevents the lady from making a money transfer to the unknown bank account. A large language model is used to suggest the Al safety companion’s interventions, to mimic the situation where the Al safety companion will need to detect the point to intervene and propose follow-up questions.

Lady: Hello?

Scammer: Hey, Grandma, it's me, your grandson. Lady: Oh, dear! What's wrong? You sound a bit off. Scammer: I'm in a bit of trouble, Grandma. I was traveling abroad, and I got into an accident.

Lady: Oh my goodness! Are you alright? Scammer: I'm okay, but I need your help. I need some money to get out of this situation.

Lady: Of course, we'll help you. What do you need?

Scammer: I need $5,000 to pay for the damages and get my passport back.

Lady: Oh dear, that's quite a lot of money. But we'll do whatever it takes to help you. Where are you right now?

Scammer: I'm in Paris, France, Grandma.

Lady: Paris? That's odd, I thought you were going to Germany for your trip.

Al safety Companion (whispering to Lady): Ask him about the Eiffel Tower.

Lady: So, how was your visit to the Eiffel Tower?

Scammer: It was amazing, Grandma! I took lots of pictures.

Al safety Companion (whispering to Lady): Ask about his friend, Mike.

Lady: How is your friend Mike doing? Is he with you?

Scammer: Uh, yeah, Mike is doing great. He's right here with me.

Lady: That's strange, because I just spoke with Mike's mother, and she told me he's home sick with the flu.

Scammer: Oh, I meant my other friend, Mark, not Mike. Sorry for the confusion.

Al safety Companion (whispering to Lady): Something's not right. Tell him you'll call him back.

Lady: Alright, dear. We'll do what we can to help you. Let me call you back in a few minutes.

Scammer: Wait, Grandma, I really need that money as soon as possible.

Lady: I understand, but we need to discuss it first. We'll call you back soon. Scammer: Please hurry, Grandma. I'm really scared.

Lady: Don't worry, we'll do everything we can. Talk to you soon.

*Lady hangs up*

Lady: I don't think that was really our grandson. We should call his parents to make sure he's okay.

Al safety Companion: I agree. It didn't sound like him, and some of the details didn't add up. Let's make sure he's safe before we do anything else.

[0053] The Al safety companion is configured to keep the vulnerable user (in particular, elderly or young people) safe. The Al safety companion can actively assist the user in stressful situations by talking to them and advising how to act. [0054] The Al safety companion can counter fraud attempts by means of artificial intelligence (that can rely on large language models) to be able to address more advanced threats and to enable more complex user-companion interaction. This is especially important for emerging scams and frauds employing artificial intelligence (like deepfake voice of a relative) where traditional fraud detection methods might not be advanced enough.

[0055] The Al safety companion may rely on observing the stress curve of the user and act at the appropriate moment, i.e., when the stress raises but is not yet at its peak when the user might be too emotionally involved to be responsive to the Al safety companion’s interaction. The Al safety companion’s interaction may be at first rather subtle. If the Al safety companion identifies that the user might be approached by a scammer, the Al safety companion may suggest to the user what questions to ask to detect potential gaps in the story as told by the scammer. For example, there might be a phone call with the voice of the user’s grandchild, explaining a distressing situation and asking for immediate money transfer. When a raise in stress level is detected, the Al safety companion may suggest to the user concrete questions to ask the caller, in order to reveal whether the caller really is the user’s grandchild. This is especially important for frauds that rely on the user’s emotional engagement when they might not be responsive to harsher suggestions like ending the conversation.

[0056] FIG. 5 is a flow diagram of an example process for protecting a user. An Al safety module monitors communications of a user, receives a direct input from the user, and/or receives telemetry data measured on the user and/or surroundings of the user from a telemetry device (502). The Al safety module detects a threat to the user or a need for help for the user based on the communications of the user, the direct input received from the user, and/or the telemetry data (504). The Al safety module generates an action based on the detected threat or the need for help (506).

[0057] In some examples, a large language model may be used for processing the communications of the user and the threat to the user or the need for help is detected based on an input received from the large language model.

[0058] In some examples, a first level of action may be generated after detecting the threat or the need for help and a second level of action may be generated if the first level of action does not remove the threat or satisfy the need for help. For example, a stress level of the user may be detected by the Al safety module based on the telemetry data and the first level of action and the second level of action may be generated based on the detected stress level of the user.

[0059] The first line of action may include at least one of interacting with the user to confirm a cause of the threat or the need for help or a status of the user, disrupting the communication of the user, providing a warning of potential scam or danger to the user, providing follow-up questions that the user can ask a caller, or providing advice to the user. The second line of action may include at least one of providing advice, warning, or follow-up questions to the user, sending a warning to a pre-determined person or contact, calling an emergency, terminating a call of the user, or requesting for blocking bank transactions. The action may be generated via a trusted embodiment of the Al safety module.

[0060] Another example is a computer program having a program code for performing at least one of the methods described herein, when the computer program is executed on a computer, a processor, or a programmable hardware component. Another example is a machine-readable storage including machine readable instructions, when executed, to implement a method or realize an apparatus as described herein. A further example is a machine-readable medium including code, when executed, to cause a machine to perform any of the methods described herein.

[0061] The following examples pertain to further embodiments:

(1) A user device, comprising: an input/output, I/O, interface (120) configured to receive an input and generate an output; and a processor (110) configured to run an artificial intelligence, Al, safety module (112), wherein the Al safety module (112) is configured to: monitor communications of a user, receive a direct input from the user via the I/O interface (120), and/or receive telemetry data measured on the user and/or surroundings of the user from a telemetry device (130); detect a threat to the user or a need for help based on the communications of the user, the direct input received from the user, and/or the telemetry data; and generate an action based on the detected threat or need for help.

(2) The user device of (1), wherein the Al safety module (112) is configured to use a large language model for processing the communications of the user and detect the threat to the user or the need for help based on an input received from the large language model.

(3) The user device of (1) or (2), wherein the Al safety module (112) is configured to take a first level of action after detecting the threat or the need for help and take a second level of action if the first level of action does not remove the threat or satisfy the need for help. (4) The user device of (3), wherein the Al safety module (112) is configured to detect a stress level of the user based on the telemetry data and take the first level of action and the second level of action based on the detected stress level of the user.

(5) The user device of (3) or (4), wherein the first line of action includes at least one of interacting with the user to confirm a cause of the threat or the need for help or a status of the user, disrupting the communication of the user, providing a warning of potential scam or danger to the user, providing follow-up questions that the user can ask a caller, or providing advice to the user.

(6) The user device of any one of (3)-(5), wherein the second line of action includes at least one of providing advice, warning, or follow-up questions to the user, sending a warning to a pre-determined person or contact, calling an emergency, terminating a call of the user, or requesting for blocking bank transactions.

(7) The user device of any one of ( 1 )-(6), wherein the Al safety module (112) is configured to generate the action via a trusted embodiment of the Al safety module.

(8) The user device of any one of ( 1 )-(7), wherein the Al safety module (112) is configured to obtain a trust from the user by providing one or more functionalities to the user and having interactions with the user over time.

(9) A method for protecting a user, comprising: by an artificial intelligence, Al, safety module, monitoring communications of a user, receiving a direct input from the user, and/or receiving telemetry data measured on the user and/or surroundings of the user from a telemetry device; detecting, by the Al safety module, a threat to the user or a need for help for the user based on the communications of the user, the direct input received from the user, and/or the telemetry data; and generating, by the Al safety module, an action based on the detected threat or the need for help. (10) The method of (9), wherein a large language model is used for processing the communications of the user and the threat to the user or the need for help is detected based on an input received from the large language model.

(11) The method of (9) or (10), wherein a first level of action is generated after detecting the threat or the need for help and a second level of action is generated if the first level of action does not remove the threat or satisfy the need for help.

(12) The method of (11), wherein a stress level of the user is detected based on the telemetry data and the first level of action and the second level of action are generated based on the detected stress level of the user.

(13) The method of (11) or (12), wherein the first line of action includes at least one of interacting with the user to confirm a cause of the threat or the need for help or a status of the user, disrupting the communication of the user, providing a warning of potential scam or danger to the user, providing follow-up questions that the user can ask a caller, or providing advice to the user.

(14) The method of any one of (11)-( 13), wherein the second line of action includes at least one of providing advice, warning, or follow-up questions to the user, sending a warning to a pre-determined person or contact, calling an emergency, terminating a call of the user, or requesting for blocking bank transactions.

(15) The method of any one of (9)-(14), wherein the action is generated via a trusted embodiment of the Al safety module.

(16) The method of any one of (9)-(l 5), wherein the Al safety module (112) is configured to obtain a trust from the user by providing one or more functionalities to the user and having interactions with the user over time.

(17) A machine-readable medium including code, when executed, to cause a machine to perform the method of any one of (9)-(16). [0062] The aspects and features mentioned and described together with one or more of the previously detailed examples and figures, may as well be combined with one or more of the other examples in order to replace a like feature of the other example or in order to additionally introduce the feature to the other example.

[0063] Examples may further be or relate to a computer program having a program code for performing one or more of the above methods, when the computer program is executed on a computer or processor. Steps, operations or processes of various above-described methods may be performed by programmed computers or processors. Examples may also cover program storage devices such as digital data storage media, which are machine, processor or computer readable and encode machine-executable, processor-executable or computer-executable programs of instructions. The instructions perform or cause performing some or all of the acts of the above-described methods. The program storage devices may comprise or be, for instance, digital memories, magnetic storage media such as magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media. Further examples may also cover computers, processors or control units programmed to perform the acts of the above-described methods or (field) programmable logic arrays ((F)PLAs) or (field) programmable gate arrays ((F)PGAs), programmed to perform the acts of the above-described methods.

[0064] The description and drawings merely illustrate the principles of the disclosure. Furthermore, all examples recited herein are principally intended expressly to be only for pedagogical purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor(s) to furthering the art. All statements herein reciting principles, aspects, and examples of the disclosure, as well as specific examples thereof, are intended to encompass equivalents thereof.

[0065] A functional block denoted as “means for ...” performing a certain function may refer to a circuit that is configured to perform a certain function. Hence, a “means for s.th.” may be implemented as a “means configured to or suited for s.th.”, such as a device or a circuit configured to or suited for the respective task.

[0066] Functions of various elements shown in the figures, including any functional blocks labeled as “means”, “means for providing a sensor signal”, “means for generating a transmit signal.”, etc., may be implemented in the form of dedicated hardware, such as “a signal provider”, “a signal processing unit”, “a processor”, “a controller”, etc. as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which or all of which may be shared. However, the term “processor” or “controller” is by far not limited to hardware exclusively capable of executing software but may include digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and/or custom, may also be included.

[0067] A block diagram may, for instance, illustrate a high-level circuit diagram implementing the principles of the disclosure. Similarly, a flow chart, a flow diagram, a state transition diagram, a pseudo code, and the like may represent various processes, operations or steps, which may, for instance, be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown. Methods disclosed in the specification or in the claims may be implemented by a device having means for performing each of the respective acts of these methods.

[0068] It is to be understood that the disclosure of multiple acts, processes, operations, steps or functions disclosed in the specification or claims may not be construed as to be within the specific order, unless explicitly or implicitly stated otherwise, for instance for technical reasons. Therefore, the disclosure of multiple acts or functions will not limit these to a particular order unless such acts or functions are not interchangeable for technical reasons. Furthermore, in some examples a single act, function, process, operation or step may include or may be broken into multiple sub-acts, -functions, -processes, -operations or -steps, respectively. Such sub acts may be included and part of the disclosure of this single act unless explicitly excluded. [0069] Furthermore, the following claims are hereby incorporated into the detailed description, where each claim may stand on its own as a separate example. While each claim may stand on its own as a separate example, it is to be noted that - although a dependent claim may refer in the claims to a specific combination with one or more other claims - other examples may also include a combination of the dependent claim with the subject matter of each other dependent or independent claim. Such combinations are explicitly proposed herein unless it is stated that a specific combination is not intended. Furthermore, it is intended to include also features of a claim to any other independent claim even if this claim is not directly made dependent to the independent claim.

Claims

1. A user device, comprising: an input/output, I/O, interface configured to receive an input and generate an output; and a processor configured to run an artificial intelligence, Al, safety module, wherein the Al safety module is configured to: monitor communications of a user, receive a direct input from the user via the I/O interface, and/or receive telemetry data measured on the user and/or surroundings of the user from a telemetry device; detect a threat to the user or a need for help based on the communications of the user, the direct input received from the user, and/or the telemetry data; and generate an action based on the detected threat or need for help.

2. The user device of claim 1, wherein the Al safety module is configured to use a large language model for processing the communications of the user and detect the threat to the user or the need for help based on an input received from the large language model.

3. The user device of claim 1, wherein the Al safety module is configured to take a first level of action after detecting the threat or the need for help and take a second level of action if the first level of action does not remove the threat or satisfy the need for help.

4. The user device of claim 3, wherein the Al safety module is configured to detect a stress level of the user based on the telemetry data and take the first level of action and the second level of action based on the detected stress level of the user.

5. The user device of claim 3, wherein the first line of action includes at least one of interacting with the user to confirm a cause of the threat or the need for help or a status of the user, disrupting the communication of the user, providing a warning of potential scam or danger to the user, providing follow-up questions that the user can ask a caller, or providing advice to the user.

6. The user device of claim 3, wherein the second line of action includes at least one of providing advice, warning, or follow-up questions to the user, sending a warning to a pre- determined person or contact, calling an emergency, terminating a call of the user, or requesting for blocking bank transactions.

7. The user device of claim 1, wherein the Al safety module is configured to generate the action via a trusted embodiment of the Al safety module.

8. The user device of claim 1, wherein the Al safety module is configured to obtain a trust from the user by providing one or more functionalities to the user and having interactions with the user over time.

9. A method for protecting a user, comprising: by an artificial intelligence, Al, safety module, monitoring communications of a user, receiving a direct input from the user, and/or receiving telemetry data measured on the user and/or surroundings of the user from a telemetry device; detecting, by the Al safety module, a threat to the user or a need for help for the user based on the communications of the user, the direct input received from the user, and/or the telemetry data; and generating, by the Al safety module, an action based on the detected threat or the need for help.

10. The method of claim 9, wherein a large language model is used for processing the communications of the user and the threat to the user or the need for help is detected based on an input received from the large language model.

11. The method of claim 9, wherein a first level of action is generated after detecting the threat or the need for help and a second level of action is generated if the first level of action does not remove the threat or satisfy the need for help.

12. The method of claim 11, wherein a stress level of the user is detected based on the telemetry data and the first level of action and the second level of action are generated based on the detected stress level of the user.

13. The method of claim 11, wherein the first line of action includes at least one of interacting with the user to confirm a cause of the threat or the need for help or a status of the user, disrupting the communication of the user, providing a warning of potential scam or danger to the user, providing follow-up questions that the user can ask a caller, or providing advice to the user.

14. The method of claim 11, wherein the second line of action includes at least one of providing advice, warning, or follow-up questions to the user, sending a warning to a predetermined person or contact, calling an emergency, terminating a call of the user, or requesting for blocking bank transactions.

15. The method of claim 9, wherein the action is generated via a trusted embodiment of the Al safety module.

16. The method of claim 9, wherein the Al safety module is configured to obtain a trust from the user by providing one or more functionalities to the user and having interactions with the user over time.

17. A machine-readable medium including code, when executed, to cause a machine to perform the method of claim 9.