ES2638280B2 - Electronic method and system for comprehensive cognitive assistance - Google Patents

Abstract

Method and electronic system for comprehensive cognitive assistance. # Method and system for comprehensive cognitive assistance in everyday life, which improves the understanding, learning and memorization of the subject's environment, through the integration of an augmented virtual reality system, coordinated with a system of global memory improvement through sleep modulation and an intelligent expert system that controls both subsystems. The proposed invention improves the life of its users: it provides assistance in daily tasks and its continued use could reduce memory loss and learning inevitably associated with aging.

Description

Electronic method and system for comprehensive cognitive assistance

5 TECHNICAL SECTOR

The invention is framed within the technical sector of user assistance and data processing technology and more specifically to technology for improving cognitive capacity and enhancing memory in an integral manner, as well as helping aging

10 healthy.

BACKGROUND OF THE INVENTION

Memory is a fundamental part of the learning process and a necessary faculty

15 throughout life. Unfortunately, a characteristic of human aging is the progressive deterioration of cognitive abilities, including the deterioration of the long-term retention capacity of episodic memories, therefore learning capacity is impaired. Cognitive disorders are one of the growing problems in modern societies due to the increase in life expectancy.

20 In addition, in some cases, with age comes dementia, a medical term that describes the group of symptoms and behaviors observed in patients with Alzheimer's disease, degenerative disease that attacks the brain; and other memory disorders. A person with dementia loses the ability to remember and reason. Dementia progressively affects the memory, behavior and reasoning of the individual and as

As the process progresses, the symptoms become more severe until the patient's life is completely incapacitated.

The first symptoms vary from person to person, but they always cause a deterioration in the activities of daily living, and eventually they will need long-term care. He

The disease progresses through several stages, at different rates and intensity and its symptoms coincide with each other at each stage. Thus a milder first stage can be distinguished (lasting between 1 and 3 years) where the symptoms are, among others: moderate memory loss (mostly related to recent events), difficulty finding words (anomie), difficulty with relation to time, geographical disorientation, difficulty

35 to solve problems, depression ... Then there is a stage that could be described as moderate (lasting between two and 10 years) where severe memory loss occurs, 2

loss of the power of spoken expression, disorientation in time and place, difficulty with reasoning and judgment, personality and behavior changes ... Finally there is a more severe stage (which normally lasts from 8 to 12 years) where in addition to severe loss of Other symptoms may occur such as nonsense or mutism, inability to make decisions or solve problems, rigid and bent posture, incontinence ...

Cognitive changes associated with age occur in parallel with three prominent features of aging. The first is usually structural cerebral atrophy, more pronounced in the midline of the frontal lobe. The second is the difficulty of maintaining sleep, which precedes in years the establishment of cognitive impairment. Finally, the third is normally the disturbance of the electroencephalographic (EEG) features of the slow-wave sleep (SWS), mainly the progressive decrease in slow wave activity (SWA). Slow-Wave Activity ”). This is described for example in the document, "Age-related changes in slow wave sleep and REM sleep and relationship with growth hormone and cortisol levels in healthy men," JAMA, vol. 284, pp. 861-868, 2000 by E. Van Cauter, R. Leproult and L. Plat. In addition, recent evidence (described for example in the document "Prefrontal atrophy, disrupted NREM slow waves and impaired hippocampal-dependent memory in aging," Nat. Neurosci., Vol. 16, pp. 357364, by BA Mander, V. Rao, B. Lu, JM Saletin, JR Lindquist, S. Ancoli-Israel, W. Jagust and MP Walker) suggest that these three features of aging are causally related to age-related cognitive impairment, and are even more pronounced when Associates incipient dementia.

On the other hand, from the pioneering studies in neuroscience, the relationship between sleep and memory is maintained in the psychological and neuroscientific literature. For example, already at the beginning of the last century, it is described how the forgetfulness rate is reduced during sleep with respect to that of wakefulness. In addition, in recent years, advances in neuroscience have once again focused on learning, placing the relationship between sleep and memory consolidation at the center of academic debate in the most important journals worldwide.

There are two types of sleep: rapid eye movement sleep (REM sleep, "Rapid Eye Movement") and non-REM or slow wave sleep (SWS). Although both types of sleep have been related to memory consolidation, evidence is accumulating that points more to the relationship between SWS and semantic memory. For example, it has been found that the low cortical acetylcholine levels characteristic of SWS are

necessary for the consolidation of declarative memory or that, among the basic mechanisms involved, it is known that the SWS enhances the thalamocortical evoked response, through a calcium-dependent mechanism. In addition, it has recently been shown that SWS promotes the formation of new synapses after learning and there is scientific evidence that demonstrates that slow wave activity during sleep can help strengthen existing neuronal connections and provide long-term stable memories. .

Therefore, when slow wave activity decreases during sleep in older adults, the consolidation of declarative memory loses efficacy, that is, the absence or decrease of slow waves during sleep in older adults contributes to their long memories term be weaker. The reported results demonstrate that the age-related memory deficit can be reduced by artificially increasing the activity of slow waves during sleep (see for example the document "The memory function of sleep," Nature Reviews Neuroscience, vol. 11, pp. 114-126, 2010 of S. Diekelmann and J. Born). Therefore, manipulation by physical, pharmacological or psychological means of SWS could be a therapeutic target to lessen the effects of memory impairment associated with age.

Following this approach, it has been shown that the enhancement of SWS through extracranial electric fields or through auditory stimuli is capable of producing improvements in learning and memory tasks with positive and measurable effects. Thus, patent application US2014 / 0057232 A1 describes a device and system for modifying memory and / or cognitive functions by releasing a sensory stimulus associated with the learning material at appropriate physiological periods during sleep. A device that can be used to identify the appropriate periods of the sleep cycle is used to facilitate or interrupt memory consolidation. The device can record environmental sensory inputs during acquisition during waking periods in which it is studied; and replicate them during specific portions of the user's dream. The user can receive a stimulus from the device to affect brain activity during sleep to modulate the sleep phase, the quality of sleep, and / or the quality or intensity of brain rhythms during a particular phase of sleep.

Patent application US2015 / 0164361 A1 describes a sound stimulator for memory improvement. It is an atrial stimulator device comprising at least one atrial sensor, a control unit and an output transducer. The device is configured

to measure the brain electrical activity indicative of an EEG electroencephalogram of the user carrying the device. The control unit is configured to receive the electrical signal and detect the presence of a slow wave, indicative of the slow sleep cycle; In addition, this unit is configured to trigger the generation of at least one stimulus pulse in response to the detection of the presence of a slow oscillation.

As for the assistance in the daily life of the elderly who suffer some kind of cognitive disability, there are several solutions based on Augmented Virtual Reality (AVR). This approach, sometimes, consists of superimposing information, either in the form of animation, images or text, on an image captured by a digital camera. The NACODEAL project (Natural Communication Device for Assisted Living) is a natural communication device for assisted daily life that helps older people with the tasks of their daily lives so that slight memory losses are less decisive and can live longer time independently, that way the life of caregivers and family members is also improved, reducing the assistance they must provide. The device consists of three parts, an online server for caregivers, family members and doctors, a graphical interface adapted for the user and an augmented reality device for daily life aids. The online server is the node of union between the elderly user of the system and the rest of the elements. It allows family members, caregivers and doctors to upload information to the graphical interface or augmented reality device. These can create new notices, add an appointment with the doctor, send images, among other functions. The graphical interface allows interaction with the user as simple as possible and gives access to the private social network hosted on the server, also allows access to other services such as calendar, photographs, maps, games, etc.

The augmented reality device provides services to the user without the need for intervention. The system performs a complex calculation of indoor positioning and provides information to the user according to their needs, some of this information can be introduced by caregivers and family members. Some of the services provided by this system can be: Reminders of appointments with the doctor, birthdays, visualize images or videos, remember actions on fixed objects (Ex: "Take the keys"; "Turn off the gas"), remember to take medicines, information about the weather and suggestions on the type of clothing you should wear, video instructions for general use of household appliances, suggestions on the type of food you should eat or remind you to drink.

Another patent application in this field is US2012 / 0025999 A1 describes a system that provides assisted daily life messages with different levels of detail to a person inside a home, according to the proximity gradients. The system includes location data that can be used by the caregiver, located in a different place in the home, to specify different messages that will be presented to the person depending on their location.

However, although there are currently cognitive assistance techniques for patients in incipient stages of the disease and sleep modulation systems to reinforce learning, no solution has proposed the integration of an augmented virtual reality system and a global improvement system memory in a precise and functional way, managed through an intelligent expert system with the characteristics, versatility and advantages of the proposal of the present invention (a global solution that assists the subject in their daily lives and simultaneously improves the ability to learn and remember to prolong independence time).

DESCRIPTION OF THE INVENTION

The invention consists of a novel method and electronic system for comprehensive cognitive assistance in everyday life (called ACISTE), which improves the understanding, learning and memorization of the subject's environment, through the integration of a (sub) system of augmented virtual reality, coordinated with a (sub) global memory improvement system through sleep modulation and an intelligent expert system that controls both subsystems, in an efficient, precise way and using the minimum number of resources. ACISTE improves the lives of its users: it provides assistance in daily tasks and its continued use could reduce memory loss and learning inevitably associated with aging.

Assistance in everyday life is achieved through an augmented virtual reality module that collects information from the environment and the user, and displays, in real time, relevant data that vary according to needs, time of day and the place where the Username. The system records daily events and can display part of them to the user just before going to sleep. This summary is prepared automatically by an intelligent expert system. Memory improvement is carried out through the modulation of brain activity during slow sleep, through synchronized brain stimulation techniques. In this way, the daily events that the system

intelligent expert considered relevant to the user, are consolidated in declarative memory, prominent daily events become memories in the long-term memory. The long-term use of this system favors healthy aging, and may be relevant for the treatment of other pathologies such as mild cognitive impairment, neurodegenerative diseases such as Alzheimer's, or other sleep disorders such as sleep apnea or insomnia.

The present invention proposes a system for comprehensive cognitive assistance based, among other things, on synchronized stimulation with brain electrical activity. For simplicity and clarity, this system can be seen as a device that performs several functions or as a set of functional elements (modules or devices) each in charge of at least one function. These modules or functional elements (also called sub-systems) can be:

-

A virtual reality (augmented) module (device) that receives information from the outside world and performs various functions (such as object recognition, face recognition, voice recognition ...); it processes them and optionally encodes them to be used by another of the modules (called module or expert system).

-

A set of biosensors that record various physiological signals of the user.

-

A sensory processing module of the physiological signals collected by the biosensors and encoding them for use by the module or expert system.

-

A memory improvement module that modulates brain activity through synchronized phase stimulation with brain electrical activity. This module can be in charge of: recording the brain electrical signal or electroencephalogram (EEG) in real time; process said recorded signal, also in real time, to distinguish between different waking and sleeping states; synchronize the stimulus signals with those recorded; store the wake and sleep cycle properties; and produce non-invasive and synchronized phase stimulation with brain activity that is administered to the user for the improvement of memory (all these functions can be performed through appropriate subsystems for each function). Brain modulation can be achieved by synchronizing the activity.

brain with multistable oscillators that can be chaotic or not. The non-invasive stimulus can be sound, optical, mechanical, electrical and / or magnetic among others.

-

An agent of interaction with the user that is part of the physical interface between the user and the system and communicates the user with the system in a bidirectional manner by means of for example auditory, visual, audiovisual or mechanical means with vibrations. This agent can be installed in any electronic communication device such as a mobile phone, tablet, computer, laptop or any other.

-

An intelligent expert module (also called an expert system, decision module or controller module) as the core of the system that communicates with all other modules for decision making. The information received is analyzed and processed using a database that is constantly updated and, for example, together with the location coordinates of the open or closed space where the user is located, it is decided what information to display in the deployment system. The intelligent expert system generates a daily summary as a strategy to reinforce learning. The communication of the expert system with other modules or with the user can be done through communication modules of any kind, preferably wireless technology such as WIFI, Bluetooth, Infrared, Radio Frequency, GSM mobile telephony, UMTS ... or any other.

These modules can be implemented via hardware or software according to the system performance requirements. They can also be implemented as separate entities (and communicated by any communication technology) or some of them found together in the same physical entity. Thus, for example, the augmented virtual reality module may contain the biosensors and include the sensory processing module. It can also for example include an EEG record.

The expert module can consist of: a virtual assistant in real time, a subsystem for the prediction of daily life and detection of unusual behaviors; a mental fatigue detector, a sleep analyzer agent, a daily summary generating agent to generate (and through the virtual reality module present in a visual and / or auditory way) the most significant events of the day minutes before the user of having to sleep; and a daily learning examining agent to assess the progression of learning or the improvement in cognitive abilities.

The present invention also proposes a method for comprehensive cognitive assistance based, among other things, on synchronized stimulation with brain electrical activity. Said method may comprise the following phases (this is only an example of embodiment and not all phases are essential and essential for the method to be carried out):

-

A training phase, where the user's daily information is preloaded in a database (for example, from the expert system).

-

A phase of day cognitive assistance that facilitates daily tasks. In this phase, based on the knowledge that the expert system acquires daily, a task prediction is made and attempts to anticipate and provide pertinent and timely information to help the subject, as well as to avoid forgetting them.

-

A phase of memory consolidation during sleep in which the expert system prepares and shows a daily summary of the most outstanding events of the day before going to sleep using a daily summary generator. Subsequently, some characteristics of the various phases of sleep can be recorded and modulated by synchronized phase stimulation with brain electrical activity to improve and / or consolidate memory.

-

A phase of evaluation of the integral cognitive learning of the method and device in which a daily test can be carried out, which consists of a set of psychophysical tests that are elaborated from the material presented the previous day. The test is performed quickly and allows you to evaluate the learning achieved by the user. A periodic review strategy is also prepared, throughout the day, to strengthen the knowledge that the system has established as relevant.

ACISTE can include a daily summary generator that stores, for example, the voices and images of known people, as well as the environments in which the user has developed during the day. The information shown in the daily summary is chosen for example using the parameters collected during the day by the various modules of interaction with the user and the environment.

The proposed solution can interact with caregivers, family members and / or experts in the care of the user to report daily activities and behaviors 9 10

made during the day, likewise, generates warning signs in case of unusual behaviors or high risk. Communication is achieved through any communication network, preferably wireless technology such as WIFI, Bluetooth, Infrared, Radio Frequency, mobile telephony ...

The proposed method and system provides daytime cognitive assistance that includes displaying information to the user for assistance in daily activities. The information that is displayed by visual, auditory and / or audiovisual means to the user includes, for example: information of the people with whom it interacts (whether the system recognizes them for example by voice or by face), remember to the user the events and scheduled tasks, (such as taking medications or visits to the doctor) and in case of unusual activity can emit an alert signal through the wireless communication system. For this, the proposed device can communicate with the emergency services (for example, when the physiological signals received by the expert system are abnormal).

The proposed method and system also involves the improvement of nocturnal memory by modulating brain activity during slow sleep, so that everyday events become part of declarative memory or long-term memory.

Finally, in another aspect of the invention there is a computer program comprising instructions executable by computer to implement the described method, when executed in a computer (or a digital signal processor, an application-specific integrated circuit, a microprocessor , a micro controller or any other form of programmable hardware). Said instructions may be stored in a digital data storage medium.

The scope of the present invention is defined by the independent and dependent claims appended. For a more complete understanding of the invention, its objects and advantages, reference may be made to the following specification and the accompanying drawings.

BRIEF EXPLANATION OF DRAWINGS

To complement the description that is being made and in order to help a better understanding of the characteristics of the invention according to preferred examples of practical embodiments thereof, it is accompanied as an integral part of

This description drawings where, for illustrative and non-limiting purposes, it has been represented:

Figure 1 schematically shows a block diagram of the solution 5 proposal for Comprehensive Cognitive Assistance Based on Synchronized Stimulation with Brain Electric Activity (ACISTE), according to an embodiment of the invention.

Figure 2 schematically shows a block diagram of the proposed sub-system or memory improvement module (MemBoost), according to an embodiment of the

10 invention.

Figure 3 schematically shows a block diagram of the proposed sub-system or Smart Expert module (EXPI), according to an embodiment of the invention.

15 DETAILED DESCRIPTION OF THE INVENTION

Figure 1 schematically shows a block diagram with some of the modules of the proposed system for Comprehensive Cognitive Assistance Based on Synchronized Stimulation with the proposed Brain Electric Activity (ACISTE), of

20 according to an embodiment of the invention.

The present embodiment of the invention proposes a method and system (called ACISTE) that allows comprehensive cognitive assistance in the daily life of the user (11), which improves the understanding, learning and memorization of the subject's environment

25 For simplicity and clarity, the ACISTE system can be seen as a set of functional elements (also called modules, devices or subsystems) each in charge of at least one function. These modules would be (this is just a preferred embodiment example, some of these modules being optional, that is, not all modules are

30 essential for the realization of the invention):

-

Augmented Virtual Reality (RVA) module or device (12): This device collects all information from the outside world (for example, audio and / or video) in real time. You can also determine and record the user's location in

35 open spaces and, optionally in enclosed spaces. In addition to receiving this external information, this module can perform various functions such as 11

object recognition, face recognition, voice recognition ...

(processing and coding faces, voice or recognized objects). Optionally

This system can inform the user of the people or objects recognized.

5

To accomplish all this, this module would comprise an audio input and / or

video (cameras, microphones or any other means known for this function), a

outdoor positioning tool (system) (for example, a system of

GPS global positioning or any other known location system), a

indoor positioning system (the same GPS if used indoors, a

10

beacon location system or any other known system that allows

location in enclosed spaces) and an information display system to

user also called user interface (that is, a system that allows

provide information to the user, for example a display and / or a speaker or any

visual, auditory and / or mechanical device that allows the user to give information).

fifteen

In addition it could also incorporate a system of recognition, processing and

face coding, and / or objects and / or voice, using any of the technologies

known; This recognition system would receive the video and / or audio input

recorded by the audio and / or video input and even the location detected

by the tool of positioning in exteriors or interiors (since sometimes the

twenty

recognition made may also depend on the user's location).

In a preferred embodiment, this module would be implemented in glasses, for

example in virtual reality glasses (although you could also use any

another device that could perform the functions mentioned above).

25

-

Biosensors module (called BioSense module): A set of sensors

(biosensors) that records the biological (physiological) signals (parameters) of the

user such as: heart rate, temperature, movement,

acceleration, response / galvanic resistance of the skin or any other signal

30

biological that might be needed. These biological signals are recorded with

different electronic, electromechanical, optical, magnetic or

any other type All these sensory parameters obtained by the module

BioSense (which are often analog signals), can be processed (this

It can be done by a sub-module, external or internal to the Biosense module, called

35

sensory processing module and may comprise for example digitizing,

integrate and optionally encode these signals) for sending and processing

12

Suitable by intelligent expert system. Even, in one embodiment, they are sent to

intelligent expert system the signals of the sensors and it is this system that

process

5

This module can be independent or incorporated into the virtual reality module

increased In other words, the RVA can incorporate biosensors that record

biological signals and they are sent (before or after being processed) to the system

intelligent expert

10

- Memory Improvement Module (called MemBoost) (15): A block diagram

showing the different elements (blocks or sub-systems) of this module it

included in figure 2. The main function of MemBoost is to modulate the activity

brain by stimulation preferably synchronized (in phase) with the

brain electrical activity The modulation of brain activity is carried out

fifteen

through a stimulus (preferably non-invasive) that modifies some or some

of the characteristics of brain electrical signals. This stimulus must of

be in phase with brain electrical signals what is achieved with different

synchronization systems Synchronization can be carried out using

any known method (for example, in a preferred embodiment would be used

twenty

multi-stable chaotic oscillators). Thus, synchronization would be carried out by

the theory of coupled oscillators that stipulates that two systems weakly

trailers will end up oscillating with perfect synchrony, in this way if coupled

the brain electrical signal to an oscillator, the latter will end up synchronized with the

brain activity This characteristic allows to emit a stimulus in phase with the

25

brain activity to modulate wakefulness or sleep states. The stimulus no

Invasive can be sound, optical, mechanical, electrical and / or magnetic, among others. The

synchronization of the brain signal with the stimulus that is carried out through

chaotic multistable oscillators can be implemented by hardware or

software.

30

To accomplish all this, this module would comprise a (sub) registration system

(acquisition) of the brain electrical signal or electroencephalogram (EEG) in time

real (21). These signals are processed (preferably also in real time)

to distinguish between different states of wakefulness and sleep (22); a (sub) system of

35

synchronization (23) (eg chaotic oscillators); a (sub) system of

storage (24) of the wake and sleep cycle properties (for example, of

13

daily way); and a (sub) system to produce the non-invasive stimulus (25) and deliver it (stimulate) to the user (26).

-

User Interaction Agent (called AGIS) (14): This module together with the

5

Augmented Virtual Reality system, are the physical interfaces between the user and the

system (ACISTE). AGIS communicates the user with the system in a bidirectional way

(for example, you can communicate the daily summary of activities to the generated user

by the EXPI, perform tests and collect the user's response to the tests, provide

User information of any kind ...). In a preferred embodiment,

10

this agent can be implemented in any electronic device of

communication such as a mobile phone, tablet, computer, laptop or

any other. The interaction with the user can be done by auditory means,

visual, audiovisual, mechanical (vibrations) or any other type.

fifteen

- Module (system or subsystem) Intelligent Expert (EXPI) also called module

processor, controller or decision (13): A block diagram showing the

Different elements (blocks or sub-systems) of this module are included in the figure

3. It is the core of the ACISTE system, it communicates with all other modules

for decision making. EXPI receives information from the RVA module that includes the

twenty

registered faces and objects (found in the field of vision of the

RVA video input device), also receives voice information

that the audio system has been able to register. All this information is sent

(normally processed and encoded, even if processed and encoded (32)

can be done in the EXPI) from the RVA module to EXPI using any type of

25

communication network (preferably wireless). This information is recorded in

a database (31) that is constantly updated in the Expert System

Intelligent and serves as a basis (for example along with other previous information

stored in this or another EXPI database and along with the coordinates of

location of the open or closed space where the user is located) to

30

determine what information to display in the display system of

RVA module information.

The signals (normally processed and encoded, even if processed and

coding can be done in the EXPI) of the biosensors module are sent

35

also to the EXPI through any type of communication network (preferably

wireless) The biosensors information, together with the module information

RVA (and optionally together with other previous information stored in the EXPI and together with the location coordinates of the open or closed space where the user is located) are used to detect what type of activity the user is doing and the level of fatigue (35 ) (mental and / or physical) and to determine what information will be displayed in the daily summary by the AGIS User Interaction Agent.

MemBoost also communicates with EXPI (via any type of preferably wireless communication network) by sending information about the user's sleep properties (for example, the waking or sleep states detected), to determine the quality and quantity of the subject's sleep ( 36) (user). This information can be used to plan the user's daily activities, depending on what he slept (how rested he is) that day or on previous days.

From all these data, EXPI can determine the activities that the user has performed daily, as well as predict unusual behaviors

(3. 4)

(when comparing the task that the user is performing with the usual tasks that are performed and that are stored in the EXPI or, for example, when comparing the current location of the user with the usual location at that time of day, stored in the EXPI). In addition, from all this information received during the day, EXPI can generate a daily summary to present the most significant events of the day visually and / or auditively to the user periodically (for example, every day minutes before the user gets ready to sleep or at any other time). The agent (subsystem) that generates this daily summary

(33)

You can have a memory in which you store relevant information during the day to decide what to show in the daily summary. This information is selected throughout the day depending on various parameters, such as its relevance in the EXPI database or by the physiological signals recorded during the occurrence of the event. Finally, EXPI has a Daily Learning Examining Agent (37) to assess whether the ACISTE system is meeting the stated objectives (this is done, for example, with periodic user tests carried out through the User Interaction Agent).

As we have indicated, EXPI is communicated with the rest of the modules of the system (through any type of communication network, preferably wireless). EXPI 15

You can also, through a communication module (preferably wireless) connect to external devices or a web page to share and update information remotely and to report daily activities and behaviors performed during the day. Similarly, through a communication module, the system can notify emergency medical services and / or family members (by connecting to a communications network) in emergency situations. For example, when the signals registered by the biosensors module are abnormal (they are within the normal range stored in the system which denotes an emergency situation of the user) or for example, when the user's location is outside a predetermined area (for example the user's house).

The information received from the rest of the modules and / or generated by the EXPI can be stored in the EXPI database (31) (or externally) for use in future decisions. This information can be the faces, voices or objects detected, the user's location, the daily activities and the user's daily agenda, sleep properties, physiological indicators ...

In summary, it can be said that the expert module can perform the following functions among others or, in other words, includes the following functional elements: real-time virtual assistant (32) to send and receive information from the RVA, prediction of daily life and detector of unusual behaviors (34); Mental fatigue detector (35), sleep analyzer (36), daily summary generator (33) and daily learning examiner (37) to assess learning progression or improvement in cognitive abilities.

This module can be a separate independent module or be incorporated (be in the same physical entity) as other modules, such as the RVA module.

In the operation of the proposed ACISTE system, several stages can be distinguished. These operating steps may be (this is only a preferred embodiment example, some of these optional stages being, that is, not all stages are essential for carrying out the invention):

-

Training or pre-preparation: Before the device is used as

cognitive assistant can go through a previous preparation stage (or

training), where it is preloaded into a system database (which

normally belongs to the EXPI module) daily information (11) of the user,

5

such as the activities that you perform daily, people's faces and voices

with which it interacts regularly, values of physiological constants (for

example in the form of ranges) of the user when the activities are carried out

everyday (so when physiological values are out of these ranges you can

determine that there is an anomalous situation and notify emergency services or

10

default contact persons), daily schedule of user activities and,

in general, any type of user information that is considered interesting

for system operation. It can also be included in the database,

the scheduled places, that is, the places where the user should be at

throughout the day (so when the user's location does not match the location that

fifteen

should you have at a certain time of day you can determine that there is a

anomalous situation and notify emergency services or contact persons

default). During system operation you can register

information received such as the tasks that are performed, the people with whom

find the user, the places where he is and decide (for example, if

twenty

they repeat a certain number of times) if those records become part of the

daily user information. Therefore, it can be said that this is a system

intelligent that is learning or training with daily functioning.

-

Day Cognitive Assistance: ACISTE acts during the day as an assistant

25

cognitive that facilitates daily tasks. Assistance may include for example

that, from the knowledge that the system has of the tasks that the subject

(user) usually performs, as well as the places he frequents and people and

objects with which it interacts, the cognitive assistant makes a prediction of the

tasks that the subject must perform and try to anticipate and provide information

30

relevant and timely to help the subject in the performance of their activities

usual, as well as to avoid forgetting them. For example, if the system knows

that at a certain time the subject must perform a task (for example, take

a certain medication), you can notify the subject at the determined time that

you have to perform this task (for example, take that medicine), through

35

the different user interfaces (for example, the RVA display or the AGIS agent).

This knowledge that the system has can be based on the data preloaded in the system during the pre-preparation phase and / or on the data that is being recorded in the system (that is, knowledge that the system is acquiring) during its operation. .

5

10 15 20

In addition, ACISTE can perform many other tasks such as providing real-time information of the people with whom it interacts (which it has recognized through the face and / or voice recognition system) giving information about them visually or audibly ( through one of the interfaces with the user) such as the name and relationship with the user. Finally, the system can interact with professionals related to the care of the elderly and / or family to inform daily activities and behaviors performed during the day. Likewise, the system generates warning signals in case of unusual behaviors (for example, that at a certain time it is in a different location than the one scheduled for that time), high risk, or when the physiological signals are abnormal and notify emergency medical services (and / or other previously stipulated persons and whose contact details have been uploaded to the system) through the communication module of the expert module.

-

Memory consolidation during sleep: The Memory Improvement Module, MemBoost, during user sleep, modulates brain activity through

synchronized phase stimulation with brain electrical activity. Modulation

25 30

of brain activity is carried out by means of a stimulus that modifies some or some of the characteristics of brain electrical signals. This stimulus, in a preferred embodiment, is in phase with the registered brain electrical signals, which is achieved with different synchronization systems. As indicated, MemBoost employs in a preferred embodiment a predictive system based on multi-stable chaotic oscillators. This feature allows a phase stimulus to be emitted with brain activity to modulate sleep states. The non-invasive stimulus can be sound, optical, mechanical, electrical and / or magnetic among others. With this, it is possible to modify some characteristics of the various phases of sleep to improve memory.

35

18 Before the Memory Improvement Module comes into operation, the EXPI can prepare and show the user before he goes to sleep, a daily summary

(or of the period of time that is considered opportune) of the most outstanding events of the day. For this task, EXPI uses the Daily Summary Generator that stores, for example, the voices and images of known people (with whom it has interacted during that day or period of time considered),

5 tasks performed as well as the environments in which the user has developed during the day. The system also decides what additional information (for example, collection of biosensors) is necessary to store in the EXPI database for this purpose.

10 - Evaluation of Cognitive Learning: Once the user has woken up, the Expert System, EXPI, can perform a comprehensive evaluation of ACISTE, examining the user's daily learning. For this, a periodic test (for example, daily) is performed on the user through the User Interaction Agent. The test can comprise a set of psychophysical tests

15 that are made from the material presented the previous day (the daily summary shown to the user), which the user performs quickly and allow to evaluate the learning achieved. You can also prepare a strategy of periodic reviews, throughout the day, to strengthen the knowledge that the system has established as relevant, in addition to strengthening those that the subject has not

20 learned properly.

In a physical implementation of the system according to a preferred embodiment (which has already been indicated throughout the present description but which we collect here for clarity), ACISTE can be integrated into an Augmented Virtual Reality glasses that contains the biosensors, the EEG brain activity recording system, a synchronization system with brain activity, a stimulation management system for sleep modulation and an audiovisual system to assist the user. The user interaction agent can be integrated into a mobile phone, tablet, computer or any other electronic communication device. In this preferred embodiment, in the memory improvement module, synchronization of the brain signal with the stimulus is carried out through chaotic multistable oscillators that are implemented with hardware to improve system performance and reduce processing time. . The wireless communication of the EXPI expert system with the rest of the modules and with devices external to the system can be done through various communication modules, using any type of communication technology such as WiFi, infrared, RF, bluetooth, GSM, 3G mobile telephony, 4G ... or any other type. This is just a

example of embodiment and, of course, the system is not limited to this embodiment but can be implemented using other types of devices and technologies that are suitable for performing the functions described.

5 As a summary, we can say that the advantages of this invention with respect to the state of the existing technique are, among others, the following:

The system consists of a cognitive assistant combined with an automatic maker of daily summaries and a memory reinforcement system to attend the

10 subject in their daily life and simultaneously improve the ability to learn and remember, thus prolonging the time of independence and their performance in daily life.

The core of the invention is an intelligent expert system that coordinates assistance

15 cognitive daytime, based on augmented virtual reality; and the nocturnal improvement of memory, through slow sleep modulation. This is done efficiently and minimizing the resources used. Cognitive reinforcement is achieved through the daily summary that is presented to the subject just before sleep and in-phase stimulation with brain electrical activity during the slow sleep phase.

20 The augmented virtual reality system employs the recognition of faces, voice and objects to provide real-time information about the subject's interaction environment and assist him during daily tasks.

25 The memory enhancement system employs a stimulation system synchronized with brain electrical activity using multistable chaotic oscillators, which can be implemented via hardware or software, which makes the system more precise and functional.

30 The intelligent expert system includes a daily learning examining agent to fully evaluate the system and quantify the user's cognitive improvement. In addition, the test results allow you to make a daily planning of tasks to be performed to improve everyday learning.

Some preferred embodiments of the invention are described in the claims that follow. twenty

In this text, the term "comprises" and its derivations (such as "understanding", etc.) should not be understood in an exclusive sense, that is, these terms should not be construed as excluding the possibility that what is described and defined can include more elements, stages, etc.

5 Describing sufficiently the nature of the invention, as well as the manner in which it is carried out in practice, it should be noted that its different parts may be manufactured in a variety of materials, sizes and shapes, and may also be introduced into its constitution or procedure, those variations that the practice advises, always and

10 when they do not alter the fundamental principle of the present invention. The description and drawings simply illustrate the principles of the invention. Therefore, it should be appreciated that those skilled in the art will be able to devise various provisions that, although not explicitly described or shown herein, represent the principles of the invention and are included within its scope. In addition, all

15 examples described in this document are provided primarily for pedagogical reasons to help the reader understand the principles of the invention and the concepts contributed by the inventor (s) to improve the technique, and should be considered as non-limiting with respect to such examples and conditions specifically described. In addition, everything stated in this document related to the principles, aspects and

20 embodiments of the invention, as well as specific examples thereof, encompass equivalences thereof.

Although the present invention has been described with reference to specific embodiments, those skilled in the art should understand that the foregoing and various other changes,

Omissions and additions in the form and detail thereof can be made without departing from the scope of the invention as defined by the following claims.

Claims (18)

1. Electronic system for comprehensive cognitive assistance to a user that includes:

-

an augmented virtual reality device (12) comprising:

or at least one audio and / or video input; 10 or a user positioning subsystem, and; or a user information display subsystem; 15 -one or more biosensors configured to record the user's physiological signals;

-

a memory improvement module (15), to improve the user's learning and memorization processes, configured to modulate sleep 20 by synchronized brain stimulation during slow sleep, which

understands: or a recording subsystem (21) of the user's brain electrical signals; or a subsystem (25) configured to generate and provide the user with stimuli of the user's brain activity, and; or a synchronization subsystem (23) of the stimuli with the user's brain electrical signals 30;

-

a user interaction device (14) configured to provide information to the user and receive user interactions;

-

a controller module (13) configured to:

•

receive information from the augmented virtual reality device, the biosensors and the memory improvement module;

•

from this information received and / or data stored in the system, determine information about the activity that the user is carrying out and provide information about said activity to the user through the user information display subsystem or through another user interface;

•

from this information received and / or data stored in the system, determine if the user's activity is abnormal and, if it is determined to be abnormal, send an alarm message to an external device through a communications network and / or the user through the user information display subsystem or through another user interface.

2.

Electronic system according to claim 1, wherein the augmented virtual reality device (12) also comprises a face and / or sound and / or object recognition subsystem based on the information collected by at least one audio and / or video input , and where the augmented virtual reality device (12) is configured to send to the controller module the faces and / or sounds and / or recognized objects.

3.

Electronic system according to any of the preceding claims, wherein the user interaction device (14) is a mobile phone, a tablet, a personal computer or a laptop.

Four.

Electronic system according to any of the preceding claims, wherein the controller module (13) is also configured to create a daily summary (33) of the most outstanding events of the day according to the information recorded and display said daily summary before sleeping ( 33) to the user through the user information display subsystem or through the user interaction device (14).

5. Electronic system according to any of the preceding claims, wherein the positioning subsystem includes an outdoor location system and an indoor location system.

5. Electronic system according to any of the preceding claims, wherein the controller module (13) is also configured to provide information to the user about the activities to be carried out at any time according to a calendar of activities stored in the system.

10. Electronic system according to any of the preceding claims, wherein the information on the activity that the user is performing includes information on the state of mental fatigue of the user.

8. Electronic system according to any of the preceding claims, wherein the The memory improvement module (15) also comprises a sleep state detection subsystem (36) of the user from the registered brain electrical signal of the user. 9. Electronic system according to claim 8, wherein the memory improvement module (15) is configured to:

-

record the brain electrical signal in real time;

-

process said recorded signal, also in real time, to distinguish between different waking and sleeping states;

-

synchronize the stimulus signals with those recorded;

-

store the wake and sleep cycle properties, and;

-

produce non-invasive and synchronized phase stimulation with brain activity that is administered to the user for memory improvement.

10. Electronic system according to any of the preceding claims, wherein the user information display subsystem is audio or video or audio / video. 11. Electronic system according to any of the preceding claims, wherein the augmented virtual reality device (12) are virtual reality glasses. 5 12. Electronic system according to any of the preceding claims, wherein the synchronization subsystem (23) comprises chaotic oscillators by means of which is synchronized with the user's brain electrical signals. 13. Electronic system according to any of the preceding claims, wherein 10 the stimuli of the user's brain activity are sound and / or optical and / or mechanical and / or electrical and / or magnetic signals. 14. Electronic system according to any of the preceding claims, wherein the Communication between the controller module (13) and the rest of the system is wireless via WiFi, Bluetooth or mobile telephony. 15. Method for comprehensive cognitive assistance to a user, carried out by an electronic system, where the method comprises the following steps: 20 - record and store information, in real time, about the user's activity, where the recorded real-time information is one or more of the following: audio and / or video signals in the user's environment; user location; values of physiological parameters of the user; faces, objects or sounds recognized in the user's environment; 25 -from the real-time information recorded and / or information previously stored in the system, determine the activity that the user is performing and, through a user interface, provide information to the user about said activity being performed and / or determine a 30 activity that the user must perform and, through the user interface, provide information to the user about said activity to be performed;

-

from the real-time information recorded and information previously stored in the system, determine if the activity of the user 35 is anomalous and, if determined to be anomalous, send a

alarm message to an external device through a communications network and / or to the user through the user interface;

-

record the user's brain electrical signals during sleep;

-

provide the user with stimuli of the user's brain activity in a synchronized manner with the user's brain electrical signals.

16. Method according to claim 15, further comprising creating a summary 10 diary of the most outstanding events of the day according to the information recorded and show before sleeping said summary to the user through an electronic communications device. 17. Method according to claim 16, wherein said most notable events 15 comprise voices and / or images of known persons recorded during the day, the activities that the user has determined to have performed during the day as well as the locations in which the user has been during the day. 18. Method according to any of the preceding claims 16-17 which further 20 comprises performing a daily test to the user through a user interface, on the daily summary presented previously to the user. 19. Method according to any of the preceding claims 15-18, wherein determining whether the user's activity is abnormal comprises comparing the 25 current location of the user with the intended location of the user stored in the system and / or compare the value of physiological variables of the user with ranges of normal values for the user stored in the system. 20. Method according to any of the preceding claims 15-19, wherein 30 prior to the first step of the method, the user's daily information is stored in a database of the system, where the user's daily information is one or more of the following: activities performed daily by the user, faces and / or voices of people with which the user interacts regularly, ranges of usual values of the user's physiological parameters, agenda 35 daily user activities. 21. Method according to any of the preceding claims 15-20, wherein said stimuli of the user's brain activity are sonorous and / or optical and / or mechanical and / or electrical and / or magnetic. 22. Method according to any of the preceding claims 15-21, wherein the information to the user about the activity that he is doing or should perform consists in information of the people with whom it interacts, which the system recognizes by voice or by face. Method according to any one of the preceding claims 15-22 wherein determining if the user's activity is anomalous comprises, from values of physiological parameters of the user, determining if said values are within a range of normal values stored in the system. . Method according to any of the preceding claims 15-23 wherein providing information about an activity that the user is performing comprises providing the user with real-time information of the people with whom he interacts, detected by a face recognition subsystem and / or sounds 25. Computer program product comprising computer executable instructions for performing the method according to any of claims 15-24, when the program is executed on a computer. 26. Digital storage medium for storing the computer program product according to claim 25.  Figure 1    Figure 2   Figure 3