FI124369B - Method and Arrangement for Evaluating Activity and Function Based on Physiological Signals of Interaction - Google Patents

Description

Method and Arrangement for Evaluating Activity and Function Based on Interaction and Physiological Signals

The invention relates to a method for monitoring the activity and performance of a person or persons. The invention also relates to an arrangement for monitoring and evaluating activity and activity and to a server for monitoring activity and activity and a computer program for monitoring activity and activity.

The share of the aging population in industrialized countries is growing rapidly. Often an aging person lives in a place where relatives or friends do not have enough time to assess the person's health or performance. In such a case, relatives and friends may have a great deal of anxiety about coping with the loved one in their daily routine.

In order to overcome this situation, various IT solutions have been developed that can monitor roughly the daily activities of one or more individuals. For example, an apartment occupied by a person may be equipped with various door or room sensors to indicate the use of a door or the movement of a person in a room. In this case, the movement of the person can be discovered over a period of time. If no signals are received from the sensors about a person's movement either at certain times or at certain time intervals, the arrangement will cause an alarm to be transmitted over the data network. Examples of such tracking systems are disclosed in JP 2007299121 and CN 101324662.

Solutions are also known in which a person has, for example, a device on his wrist that monitors vital signs or which can send an alert to an outsider in situations where the person feels that his or her functional capacity is impaired. The alarm can be transmitted, for example, by radio, telephone or the Internet.

Owning and caring for pets is also becoming more common. Caring for Lemmik-30 tongues is an element of the hustle and bustle of busy people. The presence of a pet has also been found to be beneficial to human mental health and even to lower blood pressure. Playing with a pet maintains both the physical and mental fitness of the person.

35 Various computerized pet monitoring and guidance tools are known. In particular, the emergence and development of the Internet as a central 2 communication channel has facilitated the development of various pet monitoring and control systems. Such tracking and control tools can be used to monitor your pet's behavior, whether by video or audio, even when away from home. From the video and / or the sound of the pet, it can be deduced what type of functional state the pet is in at that moment.

The information network can also be used to keep track of the pet's movement and stay. Pet tracking at home can be accomplished, for example, by video surveillance, RFID tag readers, GPS, infrared detectors, acoustic positioning methods, ultrasonic positioning, radio positioning, or payroll based on acceleration measurement.

At least part of the monitoring of your pet's daily activities can be provided to a suitably programmed home computer. In such a system, the home computer 15 simulates the owner for at least part of the day. One such arrangement is described in US 2006/0011144. The described arrangement includes real-time locating of the pet (dog) by various methods, control of feeding and beverage dispensers, monitoring of the dog's condition with an accelerometer (stationary or moving), monitoring of the dog's body temperature, dog bark tracking, 20 different creating and presenting animations for the dog. If an abnormal behavioral pattern is detected at any point in time, the computer will send an alert to the dog owner. Thus, the solution described in the reference may alert if the dog's behavior is momentarily different from the behavior stored in the computer's memory. 25

The arrangement disclosed in US 2006/0011144 is incapable of drawing conclusions in situations where the activity of the dog changes unexpectedly, although the activity remains within normal limits, but the movement of the dog is either temporarily or slightly different from normal movement. In Example 30, the dog moves as much as before, but the range of movement has changed in the habit. One possibility for the pet's movement to change as described is a sudden change in the condition of the person staying in the apartment, such as an illness.

It is an object of the invention to provide an arrangement that utilizes information obtained from long-term follow-up on the activity and physiological measurements of the subject and the interaction between that person and his / her companion, for example a pet or a partner of the subject. In particular, the physiological measurement data of the subjects being observed, as well as the follow-up data related to interactions, such as the distance between locations and the interdependence of movements, shall be utilized. Comparison of short-term follow-up results with long-term follow-up results reveals changes in the subject, the individual and their companion and / or pet, physiological measurements, and behavioral changes that are not solely apparent from the short-term follow-up results.

The objects of the invention are achieved by a system whereby instantaneous movement, vital signs and behavioral data obtained by means of motion measurement and 10 other sensors connected to the person being monitored and the companion or pet are compared with the corresponding long-term measurement data of the monitored person and companion or pet. .

15

An advantage of the invention is that the combination of the vital signs measurement data of the subject being monitored, his companion and the pet with changes in the behavior, movement of the person, companion and pet, indicates a situation that does not appear solely in the individual monitoring object.

20

A further advantage of the invention is that, through the computer network, for the purpose of analyzing the problem, it is possible to visualize the movement of the person being monitored, the companion and the pet in the apartment during a time interval chosen by the follower. In motion visualization, the location of a person's, companion's, and pet's resting place can be used as a starting point, a kind of calibration information when a person, companion, or pet leaves after resting.

A further advantage of the invention is that all other activities and vital signs measurement data detected during the movement of the person, companion and pet can be visualized at their location. This helps to avoid misleading conclusions and unnecessary contacts when interpreting measurement data. For example, if the subject of observation is in the middle of the day for a prolonged period of time, but the tracking system shows him to be in his own bed and the companion's activity and vital functions are normal, it can be concluded that the object of observation is day-35 dreams. Instead, if the target is stationary in an unusual location and the companion's activity and vital signs have accelerated, it is advisable to contact the telephone by asking for the condition of the person being monitored.

4

A further advantage of the invention is that changes in the activities of the person being observed, the companion and the pet can be inferred from the occurrence of a companion or pet-related event, such as a subject's illness or injury.

5

The method of evaluating a person's activity and activity according to the invention, which measures the activity information of a person being monitored on a first wireless monitor, further comprises measuring at least one activity and activity information assessment parameter - comparing the short term measurement data of both the subject and the companion physiological assessment parameter with the corresponding measurement parameters of the long term; and - transmitting, by comparison, the change in the person's activity and performance to the at least one data processing device if the physiological assessment parameter threshold is not met.

20

The personal activity and activity monitoring system of the invention, which measures the activity information of a person being monitored on a first wireless monitor, further comprises a 25 second wireless monitoring device, comprising means for means for storing companion geographic and physiological status data in a database accessible through the data transmission network 30 means for selecting at least one person and companion physiological status assessment parameter for use in assessing a person's and companion's short-term measurement of physiological status m it-35 background data and means for transmitting, by comparison, the change in activity and performance of the detected person to at least one data processing device if the threshold value set for the physiological evaluation parameter is not met.

5

The server according to the invention is characterized in that the server comprises means for receiving activity information transmitted from a person's companion tracking device, which comprises companion up-to-date location information and at least one companion physiological status information means for at least one person and companion means for comparing short-term measurement data of physiological status of both the person and the companion with corresponding long-term background data, and means for transmitting, based on the comparison, changes in the activity and performance of the identified person to at least one data processing device, if the threshold is not met.

A computer program product according to the invention for use in the evaluation of a person's ability and activity, comprising computer program code means stored on a computer readable medium, the code means being arranged to select at least one activity information and -parameter - compare short-term measurement data of both the person and the companion physiological assessment parameter to the corresponding measurement parameter measurement data; and 25 - send, based on the comparison, a change in the person's activity and performance to at least one data processing device if the threshold for physiological assessment parameter is not met.

Certain preferred embodiments of the invention are set forth in the dependent claims.

The basic idea of the invention is as follows: The person being monitored, the companion and the potential pet are provided with a device capable of transmitting various information about the presence and movement of the person and the pet to a home computing device. The device carried by the person, companion and pet preferably comprises a processor unit and a associated memory unit, a transceiver, at least one 3D motion sensor and a microphone. The motion sensor may comprise 3D acceleration, compass, and gyro sensors, or part 6 thereof. The person and pet device may also include an audio or ultrasound sonar, a loudspeaker, an RFID tag or reader, a thermometer, and measuring sensors that monitor the life of a person, companion, or pet.

5 As the person, companion, and pet move, the 3D motion sensor will calculate the position of the person, companion, and pet after a specified time interval. Each location is first stored in the memory unit of the person, companion and pet. From time to time, location information is sent to a home PC. Location data can also be sent continuously to the PC in real time. Measurement data of the ant-10s can also be sent to the PC, whereby the location data are calculated only on the PC. For example, a transceiver connected to a USB port (Universal Serial Bus) on a PC may serve as a base station.

All other measurement data describing the behavior or physical condition of the individual and the pet are also transferred to a home PC. Examples of such information are the sleeping, feeding, and staying time of the person and companion, as well as various vital signs measurements. Similarly, the pet's resting and resting time, noise, body temperature, heart rate, and eating and drinking events are transferred to the aforementioned PC.

20

Preferably, the measurement data obtained from the home PC is transmitted to a server on the Internet. Through the server, information on the performance of the monitored person, companion and pet can be provided to an outside person, i.e. a follower, who has the permission and / or obligation to monitor the monitored person's performance and changes thereto. The information contained in the server can be used to present the behavior, status, and behavior of the person being monitored, the companion, and the pet to a third party for a specified period of time. By utilizing the long-term tracking data contained in the server, it is possible to identify interactive changes in the behavior of the person being monitored, the companion, and the pet, which are not only apparent from the real-time tracking information for one item. This kind of tracking gives you an idea of the abnormalities in the physical activity and interaction of the person being monitored, the companion and the pet. When a server-side application detects an abnormal trait in physiological measurements or interactions between a person, a companion, and a pet, it sends an alert to a follower or other specified outside person or entity.

7

The invention will now be described in detail. Reference is made to the accompanying drawings, in which Figure 1 illustrates a device arrangement for monitoring a person's activity and performance in accordance with an embodiment of the invention; Figure 2 illustrates a preferred embodiment of a subject, companion, and pet tracking device; a preferred embodiment, Figure 4a shows an example of presenting pet tracking information on a follower device, Figure 4b illustrates an example of presenting a subject tracking information on a follower device, Figure 4c illustrates an example of a subject, companion, and pet moving over a time slot. Figure 6 shows an example of the main functions performed by a person, companion or pet being watched the main steps used in presenting an observer, companion, and pet tracking data as a flowchart 20 and Figure 7 illustrates an exemplary flowchart of how the tracker, companion, and pet tracking data is utilized to provide an alarm.

The embodiments in the following description are exemplary only and one of ordinary skill in the art may implement the basic idea of the invention in a manner other than that described in the description. Although the specification may refer to one embodiment or embodiments at multiple locations, this does not mean that the reference is directed to only one embodiment described, or that the feature described is useful in only one embodiment described. The individual features of two or more embodiments may be combined to provide new embodiments of the invention.

8

Fig. 1 shows an example of a person 3, a companion 3a and a pet 1 tracking arrangement 10 according to the invention. The tracking arrangement 10 may be installed, for example, in an apartment used by the person being observed. An example of apartment 40 is shown in Figures 4a, 4b and 4c. Preferably, the exemplary 5 kinetic person tracking arrangement 10 shown in Figure 1 includes a wireless tracking device 4 carried by the person being watched 3, a wireless tracking device 4a carried by a companion, a wireless tracking device 2 with an antenna 11a, a wireless communication network 15 and a computer 17. Preferably, the monitoring environment 10 may also include one or more RFID (Radio Frequency IDentification) 12c readers, IR detectors 12b, video cameras 12a, microphones 12d, pressure sensors 12e, and door switches 12f.

The monitoring devices 2, 4 and 4a according to the invention shown in Figure 1 are connected via wireless radio link 15 to at least one base station 11 belonging to the monitoring arrangement 10. The base station 11 can be, for example, a device connected to a USB port of a computer. The wireless radio link 15 may be, for example, an infrared (IR) link, a Bluetooth link, a ZigBee link, an UVVB link (Ultra WideBand), a WLAN link (Wireless Local Area Network), or a cellular network link. Because of the short distance between the tracking devices 2, 4 and the base station 11 of the wireless radio network, the transmission powers of the tracking devices 20, 4 and 4a are also small. This allows a long operating time for the monitoring devices 2, 4 and 4a.

By means of the monitoring arrangement 10 according to the invention, "remote treatment" or "" remote monitoring "of the person 3 to be monitored can be achieved. Through the arrangement, the vital signs of the monitored person 3 can be remotely monitored in various interactions with the companion 3a as well as a possible pet 1. The invention enables the monitoring of the activity and ability of the subject 3 to be monitored by measuring the vital functions of the person 3, the companion 3a and the pet 1 and by monitoring and analyzing the interaction between the various parties. In the arrangement of the invention, the relative or friend does not have to go to the physically watched person.

The person to be followed 3 and the companion 3a shown in the example of Figure 1 may be spouses and any pet 1 may be, for example, a dog. The tracking system according to the invention is particularly well suited to situations where the spouse is in a weaker position to follow and the better person to be a companion. In the exemplary tracking system 10 illustrated, the person 3, the companion 3a and the pet 1 are provided with at least one tracking device 2, 4 and 4a according to the invention. If the pet is a dog, the tracking device 2 can preferably be shaped as a collar that the dog carries all the time.

Preferably, all of the tracking devices 2, 4 and 4a shown in Figure 1 comprise a 3D-5 motion sensor, which preferably comprises a 3D accelerometer, a 3D gyro sensor, and a 3D magnetometer sensor. By integrating motion sensor data over time, the movements of the target, companion and dog to be monitored, the magnetometer's direction of motion and even the rapid changes in motion directions of the gyro sensor can be combined to calculate the path of each of the above objects

Advantageously, the 3D motion sensor data can also be used to draw a '' map view '' of an apartment observer 3 or a dog 1 using motion tracking device 2 or 4 by moving the 3D accelerometer by the owner throughout the apartment. 15

For example, a floor plan of a room can be provided as follows. The tracking device 2, 4, or 4a is taken to the specified "home" in the apartment. At the home location, motion tracking is triggered on the tracking device 2, 4 or 4a. After that, the tracking device is hand-moved from the home office, for example clockwise, to the nearest corner of 20 rooms. The tracking device is held in a substantially horizontal position near a corner (for example, about 10 cm from both walls or if it is an outer corner about 10 cm from the corner). At the corner, the position measurement is re-registered, for example, by pressing a button on the tracker. After that, we move on clockwise to the next corner and proceed as described above. To get the floor plan of the apartment, the entire apartment is rotated following the walls. Each time you turn in a new direction at an angle or at a right angle, position measurement is recorded.

Upon finally returning to the origin, i.e. home, the tracking device 2, 4 30 or 4a is deactivated by a suitable function associated with the tracking device 2, 4 and 4a.

The tracking device 2, 4 or 4a is then connected to the computer and the resulting floor plan is accepted. If needed, you can edit the image, add things like furniture and furniture, and move places around walls or corners.

The 3D accelerometer is an example of a motion sensor that can determine the position of a target object, such as a person, companion or pet, in 3D. Other positioning technologies may also be utilized in the tracking system of the invention. The tracking device 2, 4 or 4a may also include a compass or gyro sensor that can be used to determine motion directions. Examples of other positioning technologies include RSSI (received 5 signal strength indication) and time of flight (TOF) applied to wireless sensor networks, and RFID-based positioning. The position information computed by the 3D accelerometer can be calibrated and refined from time to time, for example, by means of fixed RFID readers 12c or IR detectors 12b.

Preferably, the position information of the observer 3, the companion 3a, or the pet 1 can be calibrated, for example, by means of a sleeping / resting position. The sleeping position and sleeping position are often constant from day to day. The sleep or rest of a person, companion, or pet is indicated by the fact that no new acceleration data is available within a certain time. When a person 3, a companion 3a, or a pet 1 finally sets off for 15 movements, the probable direction of motion can be determined using the specified sleeping position and acceleration measurement data. If a person or pet appears to be walking "through the wall" on departure, the tracking system will know that the tracking device's 2, 4, or 4a coordinate system has turned during sleep or rest. In this case, the tracking system inverts or moves the coordinate system of the floor plan of the apartment so that the detected Path is adaptable to the floor plan of the apartment. In addition, in such cases, other positioning techniques may be utilized as the person or pet moves to accurately determine the location at a given time. Once accurate positioning has been completed, acceleration measurement alone can once again be used to track motion. Such a tracking system may not necessarily require compass direction measurement at any point.

In the tracking system according to the invention, the movements of the subject 3, the companion 3a and the pet 1 are monitored and recorded continuously. This allows the movement of the person 3, the companion 3a and the pet 1 in the apartment to be advantageously recorded over a longer period of time. The stored information can be processed statistically and / or utilizing a neural network. Information about the movement so processed and the interactions between the person, companion and pet can be used to send possible alarms. For example, the abnormally active movement and / or vocalization of the companion 3a and / or the pet 1 in a situation in which the subject 3 is stationary indicates that the subject 3 is impaired for some reason or another.

11

Advantageously, the movement time can be encoded with different colors or line shapes, or it can be associated at appropriate intervals with the drawn flow path pattern. For example, bright red can depict activity history from the last 15 minutes, yellow history from 15 to 60 minutes, green history from 1 to 3 hours, etc.

5

The tracking system 10 preferably comprises means for analyzing the path of a person 3, a companion 3a and / or a pet 1 by calculating correlation with, for example, the routes of previous days. For example, the last passage of the monitored person 3, the companion 3a, and / or the pet 1 can be compared to, for example, the average of the last 10 daily passage routes over a given day. The information used for comparison may include, for example, the average number of minutes a person 3, a companion 3a, and / or pet 1 spend at bedtime, at the front door, in the kitchen, at a person's 3 resting place, etc. Combining this activity information with vital signs measurements person's status.

Alternatively, a neural network may be used to interpret pathways, vital signs, and interactions.

The SOM neural network (Self-Organazing Map) is another neural network that can be utilized in the invention. In the SOM neural network, the statistical relationships between the elements of a multidimensional input data set are converted into simple geometric relationships.

25 The SOM neural network is updated by the following algorithm (1): || x (tk) - mi (tk) || = mini {|| x (tk) - mi (tk) ||}, (1) where x (tk) is a multidimensional data vector received by the SOM neural network and 30 m {tk) is an artificial neuron or weight vector. Time is expressed in fo.

As a rule for updating the weight vector, formulas (2) and (3) can be used: rrii (tk + 1) = rrii (tk) + a (tk) [x (tk) - mM, ie Nc (2) 35 rrii (tk + 1) = rrii (tk), otherwise. (3) 12

The parameter a is a "forgetting term", the magnitude of which depends on how much of the old neuron value remains in the update. It also controls network stability. Nc is a topological neighborhood, that is, a set of neurons that are in the network mainly the neuron performing the minimum operation.

5

The map update control means that the neurons m nearest to the data vector x are shifted towards the data vector x. Thus, neurons in the SOM neural network learn / excite through the input quantities they receive.

In the monitoring system of the invention, the SOM neural network learns or is trained to know the interactions of the person being monitored 3 with companion 3a or pet 1 at different times of the day. Simultaneously, the measurement data from the vital signs measurement is recorded and combined with this interaction event. If the detected interaction event does not match the interaction event known to the SOM-15 neural network and the associated vital signs measurement results, then an alert message is sent to the observing party.

Preferably, the neural network can be trained to provide an alarm when one or a particular combination of thresholds for evaluation parameters is exceeded, lowered, or a combination thereof is an unwanted combination. Using a neural network, an alarm can be triggered, for example, when a particular combination of person and pet paths (or measurement data) occurs abnormally. For example, a person-to-pet interaction event has become more active than usual. Also, abnormal data from vital signs measurement combined with a specific known interaction event will cause an alarm.

In the monitoring arrangement of the invention, it is possible to investigate where the person 3, the companion 3a and any pet 1 have moved around the apartment, where 30 and at what time of day interaction has been observed and how long the interaction event has lasted. Likewise, what kind of physiological measurement data from these interactions have been measured will be discovered.

Also, the computer program on computer 17 may alert an individual 3, 35 companion 3a, and / or pet 1 based on abnormal movement, noise, or abnormal physiological measurement of a subject being monitored. For example, if the pet 1 stays for a certain amount of time next to the resting place of the person 3 and, for example, bark repeatedly at a predetermined barking limit, the monitoring family or friend can preferably send an SMS alert or directly call Skype to listen and / or watch in the apartment happens. Similarly, if the heart rate, respiratory rate, or sound level and pitch of the companion 3a suddenly change in the vicinity of the person being scanned 5, the tracking system will preferably send an alarm. Such a situation may mean, for example, that the companion has found the person being monitored as having a medical condition but is unable to act immediately on the condition.

In one embodiment of the invention, the person 3, the companion 3a and / or the pet 1 are provided with a plurality of different tracking devices. For example, the dog 3D tracking device 2 monitors the movement of the dog and another tracking device (not shown in Figure 1) that is wired or wirelessly connected to the actual tracking device 2 can monitor, for example, 15 dog tails and / or vital signs such as body temperature and heart.

For person 3 or companion 3a, there may be, for example, six different sensors. It is advantageous to attach the actual tracking device 4 to the breast at a height of, for example, a band extending over the torso. The actual tracking device 4, 4a handles communication to the nearest base station 11 in the system. Some examples of other sensor locations include a headband or spectacles, and sensors on the wrists and ankles.

The headband may preferably be used to indicate, for example, head movement, eye and ear movement, or even to measure brain electrical functions.

Figure 2 illustrates a preferred embodiment of the person 3, companion 3a and / or pet 1 tracking device 4, 4a or 2. In the following description 30, the tracking device is referred to only by reference numeral 2. The tracking device 2 preferably comprises an energy source 26, such as a battery. The power source 26 is preferably rechargeable, for example, by connecting to a USB port on a computer. The electrical components contained in the tracking device 2 receive the energy they need to operate from this energy source 26. The 3D motion sensors 22 have at least one in the tracking device 2. The 3D accelerometer used for the Si-35 preferably has a measuring range of ± 10 g.

The measurement data of the 3D motion sensor 22 is processed at the measurement event in the central processing unit 21 (CPU) of the tracking device 2. The processing unit 14 is connected to the memory 23. The memory 23 is used to store the computer programs required for processing the measured values according to the invention. Also, at least temporarily, all values of the variables measured by the tracking device 2 are stored in the memory 23. The results calculated from the measured variables in the tracking device 2 are also stored in the memory 23.

An example of the variable computing performed by the processing unit 21 is position data computation using measured acceleration data. The payroll calculator according to the invention is stored in the memory 23. The computer program comprises computer program instructions by means of which the processing unit 21 calculates from the 3D acceleration measurement data the displacement of the tracking device 2 in three dimensions between two successive acceleration measurements.

The processing unit 21 also communicates with the communication component 25. This communication component 15 establishes a communication link 15 to the wireless communication network base station 11 included in the monitoring system. The communication component 25 preferably supports at least one communication method. Some of the preferred methods of data transmission are infrared (IR) technology, Blue-tooth technology, ZigBee technology, UWB technology, WLAN technology, and various time or code division communication technologies used in cellular networks. Through the communication component, the measurement data stored in the memory 23 of the tracking device 2 is transmitted over the wireless communication link 15 to the base station 11. From the base station, the measurement information is transmitted through the communication link 16 for storage to the computer 17. The communication link 16 may be either a wireless connection or a cable connection, or a combination thereof.

Preferably, the communication component 25 also comprises communication means capable of communicating with the RFID reader. Using the information of the RFID reader 12c, the position information of the tracking device 2, calculated from the data of the 3D accelerometer, can be verified in computer 17, for example, where a person 3 or a pet 1 appears to pass through a wall. If necessary, the position information calculated from the acceleration measurement data is preferably corrected to correspond to the position information obtained from the RFID reader 12c.

Advantageously, the tracking device 2 may comprise measuring sensors 24 other than the acceleration sensor 3D-35 22. The other sensors may be either part of the tracking device 2, 4 or 4a, or may be separate sensors across the body of a person or pet. The measurement data of these sensors is transmitted to the tracking device via either a wired or wireless communication link 27.

15

Examples of other sensor types include a compass, gyro sensor, and barometric sensor that can measure relative height change. Sensors that measure physiological properties may also be included in the monitoring device 2 of the invention. Examples of such sensors include a body temperature sensor, a respiratory function sensor, a muscle motion sensor, a head motion sensor, a heart rate monitor, or a speech sensor, and a bark sensor. The data of these measuring sensors is also processed and stored at least temporarily by the processing unit 21 in the memory 23. During the recording, the processing unit 21 combines the data of the other measuring sensors 10 with the position information obtained from the acceleration sensor 22. The combined measurement data is stored in memory 23, from which it is periodically transferred to computer 17 for further processing.

When person 3, companion 3a, or pet 1 is out of bed, it is possible to list 15 that the tracker 2, 4, or 4a coordinate system has rotated during the rest of the person, companion, or pet. Because of the rotation of the coordinate system, it may appear that a person, companion or pet is moving '' through the wall ''. In such a case, in one embodiment of the invention, the compass or gyro sensor measurement data can be utilized to determine the direction of movement of the person, companion, or pet 20 when the objects are lying down. From this measurement data it is possible to find out the actual direction of movement of the object being tracked, which advantageously corrects the direction information provided by the acceleration measurement.

In another embodiment of the invention, the known resting / home position and sleeping position of person 3, companion 3a and 25 pet 1 can also be utilized in calibrating the coordinate system of the tracking device 2, 4 and 4a. If person 3, companion 3a, and pet most often sleep in the same position / direction, the tracking system preferably updates / calibrates the sleeping position coordinates from this location information.

30

In a preferred embodiment of the invention, the tracking system learns to make standard corrections automatically after it has made the same "walk through the wall" correction several times and has found that "walking through the walls" ends with the said coordinate system corrections.

35

Measurement data from the tracker 2, 4 or 4a can be used to track the movement of an object or objects from one location to another. The information can also be used to deduce the mode of movement, such as walking, running or crawling. The measurement data can also be used to deduce the position of the object being observed, such as standing, sitting, lying down or in an abnormal position. In addition, sensors attached to the wrists and ankles can be used to measure the movement of the limbs and any tremors that may occur.

5 The tracker 2, 4, or 4a can also track the subject's sound pattern, such as volume, pitch, duration, repetition, and unusual noises. For example, speech, crying, crying, yawning, breathing sounds and whining can be distinguished from human sounding. Barking, barking, growling, purring, moaning, moving sounds, breathing sounds, yawning and wheezing can advantageously be distinguished from pet noise.

Combining all of the measurement data described above provides a picture of the interaction between the person 3 and the companion 3a and / or the pet 1 and the current level of their vital functions. Once the system is taught the interaction and levels of vital signs associated with normal life, the system then distinguishes abnormal levels of vital functions in various interaction situations, for example, by statistical computation methods or by utilizing neural network techniques.

20 Figure 3 shows an example of the use of tracking data in the Internet environment. The pet 1 has a tracking device 2 which at least continuously measures the position of the pet. The monitored person 3 has a tracking device 4 which continuously measures the location of the monitored person. Figure 3 does not show separately the companion 3a and his tracking device 4a. The tracking data of all the objects is transferred to the computer 17 via the communication network 16,15. The measurement data is stored on the computer 17 at least temporarily.

From the computer 17, a communication connection 18 to the Internet 30 can be established. Also, the server 31, which is advantageously utilized for utilizing the tracking information, is connected 30 via the communication connection 36 to the Internet. A communication link 30a between the computer 17 and the server 31 can thus be established over the Internet. This arrangement allows the measurement data obtained from the tracking device 2 and 4 to be transferred from the computer 17 to the server 31 for further analysis and possible presentation. The transmitted tracking device measurement data is stored in database 33 via data link 32 35.

Reference 34 discloses another data processing device capable of establishing a communication link 35 to the Internet 30. Through the communication link 30b established to the Internet, a data analysis and presentation request may be made from the data processing device 34 to the server 31 for monitoring devices 2, 4 and 17 4a. The server 31 checks whether the service request made by the data processing device 34 is allowed or not. If the service request is allowed, the server 31 retrieves from the database 33 information for the monitoring devices 2, 4, and 4a defined in the service request for the time period specified in the service request. The server May 31 processes the measurement data, and transmits the processed data to the data processing device 34. The data processing device 34 may be a data processing device or a third-party data processing device having the right to follow the person uses three functional relative or friend.

Figure 4a shows an example of a dog movement pattern shown in a tracking system according to the invention. The floor plan of apartment 40 is created by some prior art graphic drawing program. For example, by moving the dog tracking device 2 in the apartment 40, a measuring sensor can be used to calibrate the dimensions of the floor plan to the measuring data of the tracking device.

15

In the example of Figure 4a, the tracking of a pet 1, for example a dog, begins at point 41. Line 43 depicts the movement of the dog in the apartment 40 for the selected time interval. Preferably, line 43 is modified in such a way that the movement of the dog at different times can be represented. For example, the movement time can preferably be encoded with different colors or line shapes. For example, bright red can depict activity history from the last 15 minutes, yellow history from 15 to 60 minutes, green history from 1 to 3 hours, etc. Using coding procedures gives a clear picture of the dog's movements at different times.

25 Reference 42 describes the location of the last measurement. Reference 44 may be used, for example, to describe locations where the dog has barked or otherwise sounded for one reason or another. Reference 45 describes the place where the dog has rested. If the measured resting position 45 is conventional for that dog, the known resting position 45 can be used to calibrate the dog acceleration measurement data from the 3D accelerometer as the dog moves after resting. This avoids any possible measurement error caused by the turning of the 3D sensor and the inaccuracy of the acceleration measurement during dog rest.

Figure 4b shows an example of a person 3 moving pattern shown in the tracking system 35 of the invention in the same apartment where the pet movement is shown in figure 4a.

18

In the example of Figure 4b, the tracking of person 3 is started at point 45. The line 43b depicts the movement of person 3 in the apartment 40 for the selected time interval. Preferably, line 43b is modified in such a way that the movement of a person at different times can be represented as described above.

5

Reference 47 describes the person's visit to the toilet. Reference 48 describes a person's activity in the kitchen. Reference 49 describes a place where a person has rested or otherwise spent long periods of time. Position 49 may be, for example, a chaise lounge.

By combining the movement and interaction points of Figures 4a and 4b, a near real-time view of the functions and operational condition of the person 3 is obtained. The server 31 belonging to the monitoring system is able, using the tracking data, to determine whether the performance of the person 3 is reduced to such an extent that it necessarily requires the location of someone.

15

Figure 4c depicts the time movement of the person 46 being followed, the companion 46a, and the dog 41 as a function of time in the apartment 40 shown in Figures 4a and 4b. The flow chart in Figure 4c can advantageously record time on the person 43, companion 46a, and pet 41. In the example of Figure 20, person 46 has left the room at 10.07.10. At the time of 10.07.40 person is at 48b located in the kitchen. The passage 43b is depicted as a continuous line. Similarly, companion 46a has awakened to movement of person 46 at 10.07.10. The companion 46a is currently in the kitchen at 10.07.140, monitoring the activities of the person 46. Similarly, dog 41 has awakened to movement of person 41 at 25.07.10. Also, dog 41 is in the kitchen at 10.07.40, watching the activities of person 41.

In the example of Figure 4c, the movement of companion 46a in the kitchen between 10.07.30 and 10.07.40 is chaotic. If the vital signs measured by the simultaneous companion 3a and 30 of the monitored person 3 are also abnormal, there is a high probability that this will require an alert from an outside party. Preferably, the recorded sequences of motion of a tracked person 3, a companion 3a and a pet 1 can be viewed as easily reproducible animations.

Preferably, when using neural network computation, information such as the current and previous position coordinates, motion directions and velocities of the person 46, companion 46a and / or pet 41, movement directions and velocities, physiological measurement data, similarities and differences. By gradually allowing the neural network to organize (learn), for example, for a month to recognize normal daily routines, it becomes sensitive to abnormal situations and can thus trigger alarms.

5

Figure 5 illustrates, by way of example, a flow diagram of the main functions performed by the tracking device 2, 4 or 4a during tracking. In the following description, reference numeral 2 is used only when referring to the tracking device, although the tracking device 4 or 4a is also concerned. In step 50, the tracking device 2 is activated. The tracking device 2 10 can be activated, for example, by switching the tracking device 2 on by a switch on the tracking device. As a result of the activation of the tracking device 2, at least the 3D acceleration measurement is started in 51 tracking devices. During activation, the position of the tracking device 2 in the used floor plan coordinate system of the apartment is preferably calibrated. Thereafter, the tracking device 2 continuously measures the accelerations in three dimensions perpendicular to the two actuators.

In step 52, the processor unit 21 of the tracking device 2 calculates, by means of two successive 3D motion measurements, how much and in what direction the tracking device 2 has moved between these measurements. In step 53, the computed position information is stored in the memory 23 of the tracker 2.

Step 54 checks whether the stored measurement data is transmitted to base computer 11 via base station 11. If it is determined in step 54 that no recording is being made at this stage, proceed to step 55 to check whether any other 25 persons 3, 3a or pet 1 are to be measured. Such other measurement may be some measurement following vital signs. If it is determined in step 55 that no other measurements are performed, the process returns to step 51, after which a new acceleration measurement is performed.

30 If it is determined in step 55 that measurements other than 3D motion measurement are made, the process proceeds to step 58. In step 58, other specified measurements are made and their results stored in memory 23 of tracking device 2. Examples of such other possible measurements on person 3 or 3a are heart rate, ECG, EEG, and EMG, respiratory tones, and blood oxygen saturation. The position of the subject, such as standing, sitting or lying, can also be measured. Other types of measurement results include the type of sound the subject makes, such as speech, crying, crying, yawning, loudness, pitch, duration of sound, repetition of sound and howling.

20

Measuring data describing the gestures and expressions of a person 3 or 3a, as well as movement data of the limbs, can also be measured and recorded.

During recording, a temporal link is created between the other measured measurement data and the 5 nearest motion measurement data. By doing so, the computed position data and other measurement data can be presented in correct temporal proportions later. After recording, the tracking process returns to step 51 where a new 3D motion measurement is performed.

If it is found in step 54 that the measurement data stored in the memory 23 of the tracking device 2 is transferred to the computer 17, the process is branched to step 56. The data transmission may be initiated at certain intervals, when storage 23 memory is full or by a command from the computer 17. In step 56, the measurement data stored in the memory 23 of the tracking device 2 is transmitted to the computer 17 via the tracking arrangement base station 11 15.

After the data transfer has been made, step 57 checks whether the monitoring is still active or not. If the measurement is still active, the process returns to step 51 where a new 3D motion measurement is performed.

20

If it is determined in step 57 that it is no longer necessary to perform monitoring, then the tracking device 2, 4 or 4a is closed in step 59. Preferably, the closing can be accomplished by sending a command from the computer 17 to the tracking device 2, 4 or 4a. Preferably, the received shutdown instruction is executed after the first communication step 56 25 which is executed after the monitoring end command sent by the computer 17.

Figure 6 illustrates, by way of example flow diagram, the main steps of the computer 17 and server 31 tracking process during object tracking. In step 60 30, the tracking program on the computer 17 is activated. In active mode, the computer 17 can receive and transmit data from either the server / server 31 or the tracking device / tracking device 2, 4, or 4a.

In step 61, the measurement data 35 in the memory 23 of the tracking device 2, 4 or 4a is transferred to the computer 17. The communication may be initiated either by the computer 17 or by the tracking device.

21

In step 62, the measurement data received from the tracking device 2, 4 or 4a is stored in the computer memory. After the received measurement data is stored, in step 63 it is checked whether or not the stored measurement data is transferred to the server 31. If check 63 results in the measurement data not being transferred to the server 5 31 at this stage, the computer returns to step 61 to await the next transmission of measurement data from the tracking devices 2, 4, and 4a.

If the check in step 63 results in the transfer of the measurement data to the server 31, then in step 64 the computer 17 establishes a communication link over the Internet 10 to the server 31.

In step 70, the computer 17 sends the last stored measurement data to a server 31 which receives the measurement data. In step 71, the server 31 stores the measurement data in the database 33. The server then preferably checks if there is a tracking request 15 for the stored measurement data. If no tracking request is made, the server process goes to step 78, where the stored measurement data is neither processed nor displayed to outsiders.

If, at step 72, it is found that at least one tracking request has been made for the tracking devices 2, 4 and 4a from which the measurement data was received, the process proceeds to step 73. In step 73, the evaluation parameters are checked or set. Examples of evaluation parameters include the monitoring interval, position information, and a physiological measurement result, such as body temperature or heart rate.

In step 74, the server retrieves from database 33 measurement data for the selected monitoring devices 2, 4, and 4a and for the selected evaluation parameters. In step 75, the server 31 processes the measurement data with a suitable computer program such that the measurement data retrieves information about selected parameters. If necessary, the positioning and orientation calibration 30 is performed on the measurement data on the basis of the last observed resting position before sending the measurement data to the data processing device 34.

In step 76, the processed tracking information is sent over the Internet to the data processing device that has made an approved tracking request. The processed tracking information is transferred in a ready-made format to the communication processor 34 requesting the tracking request.

After processing and presenting the information, next, in step 77, it is checked whether any tracking request 22 for the tracking devices 2, 4 and 4a is still valid. If there is no unresponsive tracking request, the process moves to step 78, where the measurement data in database 33 is not currently being utilized.

Fig. 7 is an exemplary flowchart of Fig. 6, in step 75 5, the main software functions performed by the server in a case where it is desired to compare the latest metering data with long-term average data, for example, to generate an alarm

In step 751, a check is made to see if any evaluation parameter comparison te-10 kernel is provided. The evaluation parameter can be, for example, monitoring of interactions in the apartment based on measurement data or monitoring of physiological measurement data or a combination thereof. If no comparison request is made, in step 752, the measurement data retrieved from the database 33 is processed in a form suitable for the data processing device 34. In step 753, the processed data is sent to a data processing device 34 for presentation.

If at step 751 a request to compare at least one evaluation parameter describing the interaction and at least one physiological measurements is detected, then proceed to step 754. Step 754 compares the last 20 monitoring data of the selected evaluation parameter, for example, with a long-term average or conclusions.

In step 755, a decision is made as to whether or not the observed difference between the latest position data and the physiological measurement data and the long-term tracking causes an alarm observer. If the difference value resulting from the comparison meets a preset threshold, no alarm is sent. The threshold can be met either by exceeding or below the threshold set for a particular evaluation parameter. If several different physiological variables are being monitored, it is possible that one of the measured quantities will cause an alarm to be exceeded, and another measurement to cause a lower value to cause an alarm. In a preferred embodiment of the invention, only a predetermined simultaneous failure of a plurality of measurable variables triggers an alarm.

In the event of an alarm, the process proceeds to step 753, where the existing measurement-35 information can be sent to the requesting data processing device 34. Once the measurement information has been sent, the process proceeds to step 76 of Figure 6.

23

If the neural network is utilized in decision-making in step 755, the neural network in use can be trained to provide an alarm in step 755 when a certain combination of threshold values set for the evaluation parameters is not met. Fulfillment of a threshold may mean exceeding or falling below a predetermined threshold of physiological measurement, or various combinations of said thresholds in a given interaction.

If, as a result of the comparison in step 755, it is determined that the difference value obtained is above or below the threshold value set for at least one measurement parameter, the alarm ge-10 is generated. In step 756, the measurement data to be sent to the data processing device 34 requesting the tracking request is accompanied by information which evaluation parameter describing the interaction or physiological measurements does not meet predefined thresholds. Once the alarm is connected to the measurement data to be transmitted, the process proceeds to step 76 of Figure 6.

15

All process steps of the interactive activity monitoring process shown in Figures 5-7 may be implemented by computer program instructions executed in a suitable general purpose processor or special processor. The computer program instructions may be stored on computer readable media, such as 20 data disks or memory, from which the processor may retrieve and execute said computer program instructions. References to computer-readable media may also include, for example, special components such as programmable USB Flash drives, logic networks (FPLAs), client-specific integrated circuits (ASICs), and signal processors (DSPs).

25

Some preferred embodiments of the method and apparatus of the invention have been described above. The invention is not limited to the solutions just described, but the inventive idea can be applied in numerous ways within the scope of the claims. Further, the invention is not limited to monitoring the interaction between humans and one pet, but may also replace two or more pets whose interaction with the individual is monitored individually or together. Likewise, there may be two or more companions whose interactions with the person being monitored are monitored together or separately.

Claims (16)

A method for evaluating a person's (3) activity and functionality via a data transmission network (30), in which method 5 - measuring (50-56) with a first wireless monitoring device (4) activity information for a person (3) which is monitored in real-time, stores (56, 61-64, 70-71) the activity information about the person in a database (33), which is accessible via a data transmission network, and 10 processes (73- 75) a description of the person's activity that can be displayed via the data transmission network from the activity information stored in the database, which description is displayed (76) on the screen of a data processing apparatus (34), characterized in that in order to evaluate the functional capacity of the person (3), the method comprises: 15 meters (50-56) with a second wireless monitoring device (4a) activity information for the person (3) companion (3a) monitored in real time - selects (751) at least one in the activity information included physiological evaluation parameter for the person (3) and the companion (3a) to be used in the evaluation of activity and functional ability 20 - compare (754) short-term measurement information of the physiological evaluation parameter for both the person (3) and the companion (3) 3a) with corresponding long-term measurement information of the monitoring parameters ooh - transmits (756) information on the basis of the comparison regarding change in the person's (3) activity and functional ability to at least one data processing apparatus (34), if the tolerance value set for the physiological evaluation parameter. c3 2. A method according to claim 1, characterized in that, as an evaluation parameter for the person's (3) activity and functional ability, at least one of the following monitoring parameters included in both the person's (3) and the follower's activity's (3a) monitoring information is used: the person's (3, 46) lane (43b), companion (3a, 46c) lane (43c), rate of movement, heartbeat, ECG, EEG, and EMG curves, body temperature, blood oxygenation, breathing sound, speech power, speechless - the height of the sound, the duration of the sound and the repetition. ° 35 CM °° Method according to claim 2, characterized in that the measurement values of the evaluation parameter for both the person (3) and the companion (3a) from the selected measurement period are compared with the corresponding long-term average values for the evaluation parameters. Method according to claim 2, characterized in that the measurement values of the evaluation parameter 5 for both the monitored person (3) and the follower (3a) from the selected measurement period are analyzed with a neuronal network, which has been learned corresponding physiological measurement signal values related to the person's (3). and the usual interaction (3a) of the companion (3a). 10 Method according to claim 3 or 4, characterized in that the measurement values of the evaluation parameter are also compared for a pet (1) from the selected measurement period with the corresponding long-term average values for the evaluation parameters. 6. Evaluation arrangements for a person's (3) activity and ability, comprising - an at least partially wireless data transmission network (15, 16, 18, 30, 36) - a first wireless monitoring device (4) comprising means for defining a person's ( 3) real-time movement and physiological states, and means for creating a data transmission connection to a wireless data transmission network (15) - means (2, 17, 31) for storing the person's (3) location information and physiological state information in the database ( 33) which is available via the data transmission network (30) and - means (31) for processing information stored in the database (33) for activity information for the person (3), which can be displayed via the data transmission network, which information is arranged to be displayed on the data transmission network. a screen of a data processing apparatus (34), characterized in that the evaluation arrangement thereto comprises c3 - a second wireless monitoring device (4a), comprising means for defining companion (3a) movement and real-time physiological states, and means for creating a data transmission connection to a wireless data transmission network (15) x means (2, 17, 31) for storing the companion (3a) location information and physiological state information in the database (33) accessible via the data transmission network (30) and means for selecting at least one evaluation parameter describing the person's ( 3) and the physiologic condition of the companion (3a) to be used in the evaluation of the person's (3) activity and ability 30 - means for comparing short-term measurement information of the evaluation parameters describing both the person (3) and the companion (3a) physiologist. similar condition with corresponding long-term measurement information and means for transmitting information found on the basis of the comparison, which describes a change in the person's (3) activity and functional ability, at least at least one data processing apparatus (34), if the tolerance value set for the physiological evaluation parameter is not met. Evaluation arrangement for activity and ability according to claim 10, characterized in that the evaluation parameter for the activity (3) of the person (3) is at least one of the following monitoring parameters included in the activity information of the person (3) and the companion (3a): the person (3). 46) lane (43b), companion (3a, 46c) lane (43c), movement speed, heartbeat, ECG, EEG, and EMG curves, body temperature, blood oxygen saturation, breathing sound, voice power, voice voice height, sound duration and repetition. Evaluation arrangement for activity and function according to claim 7, characterized in that the measurement values of the evaluation parameters for both the supervised person (3) and the companion (3a) from the selected measurement period are arranged to be compared with the corresponding long-term average values for the evaluation parameters. Evaluation arrangement for activity and function according to claim 25, characterized in that the measurement values of the evaluation parameters for both the monitored person (3) and the follower (3a) from the selected measurement period ^ are arranged to be analyzed with a neuronal network, which has been learned corresponding c3 physiological measurement signal values that relate to the usual interaction events of both the person (3) and the follower (3a). in oo 30 (M Evaluation arrangement for activity and ability according to claim 8 or 9, characterized in that the measurement values of the evaluation parameters for the c animal (1) from the selected measurement period are arranged to be compared with o corresponding long-term average values for the evaluation parameters. ° 35 CM °° A server (31) comprising - means for receiving activity information transmitted from the monitoring device (4) of the person (3), which activity information comprises the person's real-time location information and at least an information describing the person's physiological state - means for storing the received activity information in the database (33) - means for processing activity information stored in the database (33) into activity information for the person (3), which can be displayed via the data transmission network, which information is arranged to be displayed on a data processing apparatus (34). ) screen, characterized in that the server (31) comprises - means for receiving activity information transmitted from the monitoring device (3a) of the person (3) monitoring device (3a), which activity information comprises the real-time location information of the companion (3a) and at least one information describing the companion's physiological state - means for a to select at least one evaluation parameter describing the physiological state of the person (3) and companion (3a) to be used in the evaluation of the person's activity (3) and functional means for comparing short-term measurement information of the evaluation parameters describing the person's ( 3) and the physiologic state of the companion (3a) with corresponding long-term measurement information and means for transmitting information found on the basis of the comparison, describing a change in the person's (3) activity and ability, to at least one data processing apparatus (34), if the tolerance value is set for the physiological descriptive evaluation parameter is not met. Server according to claim 11, characterized in that the evaluation parameter for the activity (3) of the person (3) is at least one of the following monitoring parameters included in the activity information of the person (3) and the companion (3a): the path of the person (3, 46). 43b), the pathway of the companion (3a, 46c), (c), speed of movement, heartbeat, ECG, EEG, and EMG curves, body temperature, blood oxygen saturation, breathing sound, speech strength, speech height, io duration of sound and repeat. in oo 30 (M Server according to claim 11, characterized in that the measurement values of the evaluation parameters for both the monitored person (3) and the companion (3a) from the selected measurement period are arranged to compare in the server with the corresponding long-term average values for the evaluation parameters. ° 35 CM °° Server according to claim 11, characterized in that the measurement values of the evaluation parameters for both the monitored person (3) and the companion (3a) from the selected measurement period are arranged to be analyzed with a neuron network included in the server, which has learned corresponding physiological measurement signal values that relate to the usual interaction events of the person (3) and the companion (3a). Server according to claim 13 or 14, characterized in that the measurement values of the evaluation parameters for the pet (1) from the selected measurement period are also arranged to be compared with the corresponding long-term average values for the evaluation parameters. 16. A computer program product, characterized in that it comprises computer program code means stored in a storage device that can be read with a computer, which code tools are arranged to perform all steps in any method defined in claims 1-5 where said program is run in the computer. δ {M CD O oo {M X cn CL δ {M in o δ {M