DE102022104499A1 - Synchronization of movements of a service robot with those of a person - Google Patents

Abstract

The invention relates to a system and method for detecting a person's movements and/or vital parameters and synchronizing them with the movements or signals of a service robot. For this purpose, vital parameters and/or movements of the person are recorded by the service robot and their frequency is evaluated. Based on the determined frequency, movements and signaling parameters of the service robot are adjusted, which can also include embodiment elements of the service robot.

Description

field of invention

The invention includes mobile service robotics, medical technology and here in particular the recording of vital data (mainly contactless) and human-technology interaction.

Background of the Invention

More and more tasks are now being taken over by technical systems. This also includes mobile service robots. They can train with people without necessarily having a supervisor present. If such mobile service robots are used, for example, for exercises with people who have only limited resilience due to heart problems, who have psychological problems, suffer from high levels of nervousness, etc., this can lead to the people being heavily stressed during the exercises, mentally through high stress or physically into an overload up to the point of collapsing. The inventive task is to equip a service robot in such a way that it monitors a person first, either in combination with external sensors on the person to be monitored, or - preferably with its own sensors for contactless acquisition of data - so that for stress or overload ideally, relevant parameters can be continuously monitored and, secondly, an intervention by the service robot takes place in such a way that the monitored person is brought out of the area of high stress. This is about reducing the risk of injury to the person. For example, the person is not even aware of the problems of their own situation, so they cannot consciously articulate themselves, for example. It is therefore about technical means to make human-machine interaction safer by detecting a person using sensors, evaluating the recorded data and converting them to system states of the robot that enable risk-reducing human-machine interaction.

The service robot described in this document addresses this problem by the service robot taking over diagnoses and/or exercises, for example, or simply interacting with people. In one aspect, the service robot can record and evaluate a person's movements and/or a person's vital parameters. These include, for example, blood pressure, heart rate, respiratory rate and/or other cardiovascular parameters of a person. Apart from direct diagnostic and/or exercise purposes, such information can also be used to control your own hardware and software components, for example to influence the person or to communicate with him directly or indirectly. As a result, the service robot can, for example, reduce its speed in order to reduce the stress on the person. Alternatively or additionally, the service robot can also adjust the frequency of signaling, e.g. an oscillating light or another embodiment element, in such a way that the oscillation frequency is set to a frequency that is, for example, below the breathing rate of the person detected. Since people tend to adapt their behavior to other people, primarily indirectly, this can result in the person's breathing rate being reduced or not increasing further, for example. In this way, for example, the burden on the person can be reduced.

State of the art

Various types of service robots in the health and geriatric field are known to those skilled in the art. at CN106671105 is a mobile service robot for the care of elderly people. The service robot uses sensors to monitor body parameters such as temperature and facial expressions. The service robot also recognizes whether the person has fallen and can alert help accordingly via a network.

are stored similarly CN104889994 and CN204772554 , in which a service robot from the medical field recognizes the heart rate, supplies patients with oxygen and has voice recognition and a multimedia module for entertainment purposes. The oxygen determination in the blood is also carried out in CN105082149 . CN105078445 refers to a service robot that makes it possible to record an electrocardiogram and measure the level of oxygen in the blood, especially in the elderly. similarly located CN105078450 with an electroencephalogram measurement.

Some of the health robots explicitly refer to carrying out exercises with patients or tests. In CN108053889 a system is described in a relatively abstract manner that carries out exercises with a patient based on stored information. CN108039193 describes a system for automatically generating health reports that is used in a robot. The recording of movements/fitness exercises with the help of a robot, the recording and storage of the same data for analysis purposes and the transmission to external systems are CN107544266 described. At the same time, this robot is able to monitor the intake of medication through various sensors.

CN106709254 describes a robot for the medical diagnosis of a patient, which at the same time, based on the diagnosis, also creates a treatment plan. To do this, the robot evaluates voice and image information and compares it with information stored in memory. A neural network is used for this.

In CN106407715 a service robot is described that uses speech processing and image recognition to take a patient's medical history. In addition to queries via voice input and voice output devices via a touchpad, a photo of the tongue taken by a robot camera is also used for the anamnesis.

CN105078449 describes a service robot with a tablet computer as a communication device, which is used, among other things, for cognitive function training or a cognitive-psychological test to detect Alzheimer's in patients. For this purpose, the tablet records a phone call between the patient and a child that follows a specific process and uses the course of the conversation to determine whether the patient suffers from Alzheimer's disease.

The service robot is configured in such a way that the service robot can use its sensors to collect additional parameters, including blood pressure in a contactless manner, for example using a camera. The state of the art for determining blood pressure using a camera-based analysis is mostly in the research stage: Zaunseder et al. (2018) gives an overview of those methods that carry out a color evaluation of the blood flow. The review article by Rouast et al. (2018) goes even further. For example, Karylyak et. al (2013) or Wang et al. (2014), while McDuff et al. (2014), for example, devoted to determining the timing of the systolic and diastolic pressure, while Bai et al. (2018), for example, checked the effectiveness of a new signal filter. General approaches to determining the blood pressure from measured values can be found, for example, in Parati et al. (1995). Liu et al. (2018) deal with concrete implementations of the color evaluation and also compare different sub-regions of the face, similar to Verkruysse et al. (2008), while Lee et al. (2019) describes a concrete implementation based on facial movements. Unakafov (2018), on the other hand, compares different methods on the basis of a freely available data set. For example, the approach of Pasquadibisceglie et al. (2018), who integrates a method of color evaluation into a mirror. In contrast, Luo et al. (2019) a smartphone for recording the color data. More specifically, the color data mapping approach by Wei et al. (2018), which already has the character of a clinical study. On the other hand, the approach of Ghijssen et al. (2018). Here, a laser is used to send light through a finger and a sensor detects it on the opposite side, with the emitted light showing speckle patterns that allow the rhythmic vascular blood flow to be detected on the one hand and, as with the approaches described above, on the other , to capture the rhythmic vascular expansion of the vessels.

Sources:

Zaunseder et al. Cardiovascular assessment by imaging photoplethysmography - a review. Biomed. Eng.-Biomed. tech 2018; 63(5): 617-634, DOI: 0.1515/bmt-2017-01.

Kurylyak et al. Blood Pressure Estimation from a PPG Signal, 2013 IEEE International Instrumentation and Measurement Technology Conference (I2MTC). DOI: 10.1109/I2MTC.2013.6555424.

McDuff et al. Remote Detection of Photoplethysmographic Systolic and Diastolic Peaks Using a Digital Camera. IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 12, DECEMBER 2014, DOI: 10.1109/TBME.2014.2340991.

Bai et al. Real-Time Robust Noncontact Heart Rate Monitoring With a Camera, IEEE Access VOLUME 6, 2018. DOI: 10.1109/ACCESS.2018.2837086.

Pasquadibisceglie et al. A personal healthcare system for contact-less estimation of cardiovascular parameters. 2018 AEIT International Annual Conference. DOI: 10.23919/AEIT.2018.8577458.

Wang et al. Cuff-Free Blood Pressure Estimation Using Pulse Transit Time and Heart Rate. 2014 12th International Conference on Signal Processing (ICSP). DOI: 10.1109/ICOSP.2014.7014980.

Luo et al. Smartphone-Based Blood Pressure Measurement Using Transdermal Optical Imaging Technology. Circular Cardiovascular Imaging. 2019;12:e008857. DOI: 10.1161/CIRCIMAGING. 119.008857.

Wei et al. Transdermal Optical Imaging Reveal Basal Stress via Heart Rate Variability Analysis: A Novel Methodology Comparable to Electrocar diography. Frontiers in Psychology 9:98. DOI: 10.3389/fpsyg.2018.00098.

Parati et al. Spectral Analysis of Blood Pressure and Heart Rate Variability in Evaluating Cardiovascular Regulation. hypertension. 1995;25:1276-1286. DOI: 10.1161/01.HYP.25.6.1276.

Rouast et al. Remote heart rate measurement using low-cost RGB face video: a technical literature review. Front. computers Sci., 2018, 12(5): 858-872. DOI: 10.1007/s11704-016-6243-6.

Lee et al. Vision-Based Measurement of Heart Rate from Ballistocardiographic Head Movements Using Unsupervised Clustering. Sensors 2019, 19, 3263. DOI: 10.3390/s19153263.

Liu et al., Transdermal optical imaging revealed different spatiotemporal patterns of facial cardiovascular activities. Scientific Reports, (2018) 8:10588. DOI: 10.1038/s41598-018-28804-0.

Unakafov. Pulse rate estimation using imaging photoplethysmography: generic framework and comparison of methods on a publicly available dataset. Biomed. physics Closely. Express 4 (2018) 045001. DOI: 10.1088/2057-1976/aabd09.

Verkruysse et al. Remote plethysmographic imaging using ambient light. 22 December 2008 / Vol. 26 / OPTICS EXPRESS 21434. DOI: 10.1364/OE.16.021434.

Ghijssen et al. Biomedical Optics Express Vol. 9, Issue 8, pp. 3937-3952 (2018). DOI: 10.1364/BOE.9.003937.

The solution according to the invention for risk-reducing human-technology interaction aims to determine the condition of a person, in particular by means of vital data acquisition (including respiratory and heart data), and to control the service robot in an appropriate manner (e.g. to initiate speed reductions or to control certain positions), on the other hand through adjustment the frequency of signals to affect the condition of the person, so as to reduce the risk to the person.

For example, a computer-implemented method for recording a person's vital parameters is stored on the service robot. This includes the recording and tracking of the person, optionally the recording and tracking of a body region of the person on or via which the vital parameters are collected, the recording of the vital parameters, the comparison of the recorded vital parameters with at least one stored threshold value and the triggering of a Event when the threshold value is exceeded or not reached. In this case, the method includes various possible functions in order to fulfill the task according to the invention. In one aspect, the event is a speed reduction of the service robot. In an alternative and/or supplementary aspect, the event involves driving to a target position, in which case the target position can be a seat, for example. Said threshold value of vital parameters is determined dynamically, for example from previously recorded vital parameters, in one aspect by averaging recorded vital parameters over a defined time interval. The vital parameter detection (by the service robot) takes place without contact in one aspect, in an alternative and/or supplementary aspect by a vital parameter sensor attached to the person to be monitored. In addition to the recording of vital parameters, body movements of the person are optionally recorded and the recorded vital parameters are evaluated by comparing the recorded body movements. In this case, for example, vital parameters are recorded as a function of the recorded movements of the person. The vital parameters can be the pulse rate, pulse rate variability, heart rate, heart rate variability, the systolic and/or diastolic blood pressure and/or the respiratory rate. In a further, optional step, the person's risk of falling can be determined from the vital parameter(s) recorded, which is, for example, the acute risk of falling. In one aspect, the vital signs are recorded while a test and/or exercise is being performed on the person. The invention includes not only this computer-implemented method, but also an apparatus for carrying out the same.

The invention also includes a system for recording a person's vital parameters, with a computer, a memory and at least one sensor for contactless recording of the person's movements over time and a vital parameter evaluation module, the sensor for recording the movements of the person is a camera, a LIDAR, an ultrasonic and/or radar sensor and a unit based on the data from the vital parameter evaluation module notifies a system via an interface, outputs via an output unit (e.g. a display or a Loudspeaker), a change in speed of the system itself and / or a control of a target position of the system can initiate. The system further includes, in one aspect, a body region detection module and a body region tracking module for tracking the vital signs acquisition region and a vital signs acquisition unit recording of the person's vital parameters. It can also include, for example, an application module with rules for carrying out at least one exercise or one test. Alternatively and/or additionally, the system may also include an interface and a vital sign sensor attached to a person.

As an alternative and/or complementary to the system, the device and the method described above, the invention also includes the following possible configurations: A system for synchronizing the movements and/or vital parameters of a person with the movement and/or lighting frequency of stylized embodiment elements of a service robot, comprising a person detection and tracking unit, a pulse-breath evaluation unit, and an initialization unit, with the movements occurring at a frequency or the light frequency corresponding to a frequency that lies within a defined bandwidth around the determined frequency of the movements or vital parameters of the person. For example, the stylized embodiment elements can be implemented in hardware and/or software, e.g. stylized embodiment elements implemented in software displaying at least one stylized face or facial element on a display, at least one stylized face, facial element or a torso, chest or heart , Or, for example, include at least one lamp. The stylized embodiment elements are stimulated by the initialization unit, for example, to imitate the respiratory rate, pulse rate and/or heart rate. In one aspect, the system also includes a movement signal detection unit and a processing unit for detecting and evaluating detected movements of the person. Said pulse-breath evaluation unit and/or the movement signal detection and processing unit can also be located in a portable sensor, for example.

In a further aspect, the invention includes a computer-implemented method for synchronizing the movements and/or vital parameters of a person and a service robot, comprising detecting and tracking a person and their movements and/or vital parameters, determining the frequency of the movements and /or the vital parameters of the person, and the variation of a movement frequency and/or a lighting frequency of stylized embodiment elements of the service robot with a frequency that lies within a defined bandwidth of the determined frequency of the recorded movements and/or vital parameters of the person. In one aspect, the movement of stylized embodiment elements mimics the movement of a face, facial element, torso, chest, or heartbeat and/or breathing of the sensed person. In an alternative or supplementary aspect, the frequency difference between the movements of the stylized embodiment elements and the determined frequency of the movements and/or vital parameters of the person is kept approximately constant over time by the initialization unit. The recorded movements and/or vital parameters of the person are, for example, the pulse rate, heart rate and/or respiratory rate. The movements of the stylized embodiment elements are set, for example, by the initialization unit to a frequency that is lower than the frequency of the recorded movements and/or the person's vital parameters. In one aspect, the frequency of the stylized embodiment element(s) may be controlled by the initialization unit to slow down over time. In addition to the computer-implemented method, the invention also includes a device for carrying out the same, as has been described above.

character list

• 1 Skizze einer möglichen Ausführung des Serviceroboters;
• 2 Architektursicht
• 3 Managementsystem für den Serviceroboter
• 4 Ablauf Vitalparameterermittlung
• 5 System zur Vitalparameterermittlung
• 6 Interferenzreduktion
• 7 System zur Synchronisation von Bewegungen zwischen einer Person und einem Serviceroboter
• 8 Ablauf der Bewegungssynchronisation
• 9 Vitaldatenerfassung und Triggern eines Ereignisses

Show it:

• 1 Sketch of a possible version of the service robot;
• 2 architectural view
• 3 Management system for the service robot
• 4 Vital parameter determination process
• 5 System for vital parameter determination
• 6 interference reduction
• 7 System for synchronizing movements between a person and a service robot
• 8th Sequence of motion synchronization
• 9 Vital data acquisition and triggering of an event

Detailed description of the invention

Service robots and system landscape

The term user is understood to mean a person who uses the service robot 17 and who in this case is primarily evaluated by the service robot 17 using the apparatus described. These can be people of advanced age with whom the service robot 17 carries out a geriatric test, people in medical facilities, athletes during training, or other people who interact with the service robot 17 .

1 represents the mobile service robot 17. This has a large number of sensors and devices that are typical for a service robot, which is why the exact representation of the general robotic components, modules and functions is only to be understood as an example. The service robot 17 has a laser scanner (LIDAR) 1 for scanning the area surrounding the service robot 17. Other sensors are also possible as an alternative and/or in addition, e.g. a camera (2D and/or 3D) 185, an ultrasonic and/or radar sensor 194 .

The service robot 17 has at least one display 2, which is a touchpad in one aspect. in the in 1 aspect shown, the service robot 17 has two of the touchpads. The touchpads in turn have a microphone 193 and a loudspeaker 192, for example, which allow acoustic communication with the service robot 17. Furthermore, the service robot 17 has at least one sensor 3 for the contactless three-dimensional acquisition of the movement data of a person. In a non-limiting example, the sensor is a Microsoft Kinect device. Alternatively, an Orbecc Astra 3D camera can also be used. Such 3D cameras usually also have an RGB camera for color recognition. In an alternative aspect, a conventional mono camera can be used. Technologies that can be used in 3D cameras are the stereo principle, time-of-flight sensors (ToF) or speckle sensors.

A pressure-sensitive bumper 4 is located around the outer shell of the service robot 17 at a distance of e.g. 5 cm above the ground, at least in the areas that lie in a possible travel direction of the service robot 17 . The motor controller 191 is connected to the pressure-sensitive bumper 4 and detects collisions of the service robot 17 with an object. In the event of a collision, the drive unit 7 is stopped immediately. The service robot 17 also has an energy source 8 such as an accumulator to supply the drive and computer unit 9, the sensors (laser scanner 1, sensor 3, and bumper 4) and the input and output units 2. The energy source 8 is a battery or a Accumulator.

The computer unit 9 has at least one memory 10 and at least one interface 188 (such as WLAN) for data exchange. In an optional aspect, this includes (not shown) a device for reading out a mobile memory (e.g. a transponder/RFID token). In a further aspect, this mobile memory can also be written to. In one aspect, this or another interface 188 (such as WLAN) allows for wireless communication with a network. The service robot 17 has rules for carrying out evaluations, which are described in the further course of this document and are stored in the memory 10 . Alternatively and/or additionally, these rules can also be stored in the memory of a cloud 18, which the service robot 17 accesses via the at least one interface 188 (e.g. WLAN). This does not have to be explicitly mentioned elsewhere, but is included in the disclosure.

The sensor 3 recognizes a person and their actions and creates a skeleton model based on the person's movements. Here, in one aspect, the sensor 3 is also capable of recognizing walking aids/crutches (UAGS). The service robot 17 also optionally has one or more microphones 193, which can be implemented independently of the touchpads in order to record the person's speech and evaluate it in a computer unit.

2 illustrates that the service robot 17 is connected to the cloud 18 via an interface 188 . A trainer has the option of using a terminal 13 to access a personal administration module 160 stored in the cloud 18 with a computer 161 , which in turn is connected to a memory 162 .

Personnel can store personal data in the person administration module 160 or, in one aspect, import this personal data from other systems via an interface 188 (such as WLAN). These other systems primarily include hospital management systems (HIS) and/or personal data management systems. In addition to the name and possibly the room number of the person, the personal data also includes information on the general state of health, etc. The computer 161 in the person administration module 160 generates an ID for each person, which is stored with the personal data in the memory 162 . The staff can define the tests and/or exercises to be carried out. The management system is connected via the cloud 18 to a set of rules 150, consisting of a computer unit 151 and a memory 152. The set of rules 150 provides rules for carrying out and evaluating the tests and/or exercises, which can match those of the service robot 17 and e.g maintained centrally in the set of rules and then distributed to a number of service robots 178 .

The classification of objects and movements is stored in the set of rules 150, but also the combination thereof in order to make the observations in the Evaluate the meaning of the test/exercise. For example, the position of the legs, upper body, arms, hands, etc. is stored on the basis of a skeleton model. In addition, objects that are to be evaluated as part of the test/exercise can be recognized. The set of rules 150 can initially be created based on templates with the help of experts, ie limit values are defined for individual limbs. Fuzzy algorithms can also be used for the limit values. Alternatively, individual images or image sequences, which, for example, can be translated into a skeleton model with a view to recordings of a person, can be labeled by staff and classifications that map the limit values are defined with the help of machine learning algorithms including neural networks. In addition, methods of imitation learning can also be used to define rules.

In one aspect, there is also a cloud-based navigation module 170 with a navigation computer 171 and a navigation memory 172.

The service robot 17 can be connected to a cloud application in the cloud 18 . The trainer can assign a portable storage device, such as a token, to the person who is to conduct the test/exercise. The token contains the person ID and/or another token ID assigned to the person or his/her ID. The person can identify himself at the service robot 17 with this token or the serial number and/or the ID. Identification is also possible in other ways, e.g. by entering login data in a screen-guided menu, but also by means of biometric features such as a face scan or software on a mobile device that has a code ready is entered or read into the service robot 17 . The service robot 17 now downloads from the cloud 18 via an interface 188 (such as e.g. WLAN) according to the tests, diagnostic procedures, exercises etc. stored by the staff - the assignment is made via the personal ID. After completing the test/exercise, the service robot 17 loads the data of the test/exercise in encrypted form into the person administration module 160—the assignment takes place via the person ID. The data is only decrypted in the person administration module 160 (cf. below). The staff can then evaluate the data.

In a further aspect, the staff transfers the instructions for carrying out a test/exercise or a sub-component thereof to a storage medium (e.g. transponder in the form of an RFID tag), which the person receives in order to identify themselves on the service robot 17, for which purpose this has an RFID interface 183 . In this case, for example, the data is transmitted from the storage medium to the service robot 17 , including the person ID that was specified by the person administration module 160 . After completing the test/exercise, the service robot 17 transfers the data back to the storage medium, so that the staff can transfer the data to the person administration module 160 when reading the storage medium. In a supplementary and/or alternative aspect, the data can also be transmitted to the person administration module 160 in encrypted form via a wireless or wired interface 188 (such as WLAN). Combinations of the approach described above and data exchange via storage medium (e.g. transponder) are also possible.

In one aspect, the service robot 17 is equipped with a wearable sensor 201 via a cloud or directly, which serves to record a person's vital signs. This can, for example, be a smartwatch that records pulse rate, respiration, etc.

3 illustrates the architecture view, which, however, hides applications described in the course of this document. Various modules with basic functions of the service robot 17 are found at the software level. Various modules are contained in the navigation module 101, for example. Below this is a 2D or 3D surroundings detection module 102 which, for example, evaluates surroundings information based on various sensor data. The path planning module 103 allows the service robot 17 to determine its own path, which it covers. The movement planner 104 uses, for example, the results of the path planning from the path planning module 103 and calculates an optimal route for the service robot, taking into account or optimizing various cost functions. In addition to the data from path planning, data from obstacle avoidance, a preferred direction of travel, etc., such as, for example, an expected direction of movement of a monitored person, also function as cost functions. Aspects of driving dynamics also play a role here, such as speed adaptation, e.g. in curves, etc. The self-localization module 105 allows the service robot 17 to determine its own position on a map, e.g. 3D environment detection with environment parameters stored in a map from the map module 107, etc. The mapping module 106 allows the service robot 17 to map its environment. Maps that have been created are stored, for example, in the map module 107, which, however, can also contain other maps than just self-created maps. The loading module 108 for automatic loading. In addition, there can be a database with spatial data 109, which contains information, for example, in which room an evaluation is to be carried out with a person, etc. A movement evaluation module 120 contains, for example, a movement sequence extraction module 121 and a movement sequence evaluation module 122. These each include the rules for Movement evaluation, which are described in more detail later in this document. The person recognition module 110 contains, for example, a person identification module 111, which contains rules, for example, in order to determine from the acquired sensor data whether it is a person or something else. A visual person tracking module 112 for visual person tracking is based, for example, primarily on camera data as input variables, the laser-based person tracking module 113 uses LIDAR 1 accordingly classified insofar as it is one that was previously tracked. A seat detection module 115 makes it possible to detect a chair, for example, using stored patterns, for example. The service robot 17 also has, e.g /or tests with people.

On the hardware level 180 there is an odometry unit 181, e.g. an interface for communication with RFID transponders, a camera 185, operating elements 186, an interface 188 such as WLAN, a charging control 190 for the energy supply, a motor control 191, loudspeaker 192, at least one Microphone 193, e.g. a radar and/or ultrasonic sensor 194, a detector 195, which is described in more detail elsewhere, also e.g. a spectrometer 196 and e.g. a projection device 920. LIDAR 1, display 2 and drive 7 have already been described .

With the camera 185, a LIDAR 1 and radar and/or ultrasonic sensor 194, the service robot has sensors that can not only be used for navigation purposes, but also, for example, for person detection and tracking, which is why these sensors, together with corresponding software modules, are hardware-based form a person detection and tracking unit 4605, in which case further sensors can also be used, for example in conjunction with an inertial sensor 5620, which is located on the person to be detected and/or tracked. With regard to the detection of persons and the tracking of persons, a person recognition module 110 can be used in a first step, which recognizes a person from sensor data and can have various sub-modules. This includes, for example, a personal identification module 111 that allows a person to be identified. For example, characteristic features of the person can be stored. The person re-identification module 114 makes it possible to recognize the person, e.g. after the person tracking has been interrupted, which can be done by a visual person tracking module 112 (e.g. evaluating data from a camera 185) or a laser-based person tracking module 113 (e.g. evaluating data from a LIDAR 1). The person can be recognized in the person re-identification module 114 by means of a pattern comparison, the patterns resulting, for example, from the stored person characteristics. A motion evaluation module 120 allows various movements to be evaluated. Detected movements can first be pre-processed in the movement sequence extraction module 121, i.e. features of the movements are extracted, which are classified and evaluated in the movement sequence evaluation module 122, e.g. to identify a specific movement. With a view to capturing and evaluating a person's movements, a skeleton model can be created in the skeleton creation module 5635, which determines joint points on the person's joints and direction vectors between the joint points. A feature extraction based on hinge points takes place in the skeleton model-based feature extraction module 5460, for example. A number of specific feature extraction modules are listed in the document, as well as several feature classification modules that can, for example, be based on these feature extraction modules. In one aspect, these include the gait feature extraction module 5605, which also uses data from the skeleton creation module 5635, a gait feature classification module 5610, and a gait sequence classification module 5615.

With regard to threshold value comparisons, there is sometimes talk of exceeding a threshold value, which then leads to a specific assessment of a situation. Various calculations can be used, some of which could lead to a contrary interpretation of the evaluation results. An example is a comparison of two patterns that are used to recognize people. If, for example, a similarity coefficient is calculated here, for example a correlation, a high correlation that is above a threshold value means that, for example, two people are identical. However, if there is a difference between each value, a high difference value means the opposite, ie one high dissimilarity. However, such alternative calculations are regarded as synonymous with, for example, the first calculation via the correlation.

The use of machine learning methods can, for example, make the definition of explicit threshold values, e.g. for movement sequences, superfluous in favor of a pattern evaluation. This means that instead of threshold value comparisons, e.g. for dedicated distances of a joint point from a skeleton model, pattern comparisons take place that evaluate several joint points at the same time. If a threshold value comparison is discussed below, in particular with regard to a movement sequence, a way for a pattern comparison can also be found in the case of the use of machine learning algorithms. As a basis for such a pattern comparison, for example, body poses of a movement sequence, whether correct or incorrect, can be recorded over time and evaluated coherently. A classifier can be created on the basis of extracted features such as joint points, which then compares with other recorded body poses that are specified as correct or incorrect and the courses of the joint points derived from this.

Determination of blood pressure and determination of other cardio parameters

The robot 17, in one aspect, is further equipped with a system that senses repetitive motions of the human body that correlate with the heart's output of blood into the great blood vessels at each heartbeat. In this way, changes are detected which are caused on the one hand by movements of the large blood vessels and propagate in waves in the body, for example, and/or which result from movements of arteries in the skin. The latter are more robust to fluctuations in the lighting of the body part and/or different tones of the skin. Alternatively and/or additionally, changes in the blood volume or the blood flow in the skin, eg over time, are recorded, which correlate with the heartbeat. 4 below shows the process of data acquisition and evaluation.

A body region and multiple subregions are identified at step 3005 . For example, the body region is the face, with the evaluation being carried out using the camera 185, for example. The system uses state-of-the-art algorithms to record the face (alternatively other body regions), e.g. the OpenCV, OpenPose or dlib framework, with the evaluation being carried out, e.g. by means of the visual person tracking module 112 and/or the laser-based person tracking Module 113 takes place. At least the forehead, the cheeks or the chin are recorded as partial regions, e.g. several body regions together, which are then evaluated individually and/or separately according to the steps described below. Here, for example, candidate regions can be selected, i.e. partial regions of the face that are relevant for the evaluation, for which purpose known segmentation methods (such as RANSAC) can be used. Like the body region as such, these partial regions are tracked over time in step 3010 by the frameworks or modules mentioned.

In an optional step 3015, the camera 185 is aligned as parallel as possible to the region that is to be tracked. For this purpose, the angle of coverage of the face, which results from an axis that is perpendicular to the camera capturing the face and an axis that is perpendicular to the sagittal plane of the face, can be minimized. Here, the system determines, for example, a plane that runs through the face and that is essentially parallel to the top view of the face and that corresponds, for example, to the sagittal plane. Based, for example, on histograms of gradients (HoG), the system has a classification that describes the deviations from this plan view in order to record the inclination of the face in three-dimensional space. In one aspect, the system uses this approach to evaluate a face looking into the system's camera 185 based on how parallel the face is to the camera lens. In the event of deviations, the system can, in one aspect, adjust the inclination of the camera 185 in three-dimensional space by means of a corresponding mechanism such as an optional tilting unit 5130, e.g. by controlling two tilting axes arranged orthogonally to one another, which are driven by servomotors. In this document, the term tilting unit is therefore understood to mean a tilting unit with at least two axes, which on the one hand allows the horizontal plane to be inclined and on the other hand allows rotation about the vertical axis. Alternatively and/or additionally, the wheels of the service robot 17 are controlled in such a way that the service robot 17 turns in the direction of the person in order to reduce the determined deviation of the camera plane from the face plane. In an alternative and/or supplementary aspect, such a deviation triggers the voice output of the service robot 17 in order to instruct the person to align their face accordingly. For example, rules are stored that require alignment in the XY plane if deviations in the XY plane have been determined. Alignment of the face inclination and control of the tilting mechanisms, the orientation of the service robot 17 and/or the voice output with a view of the person are carried out, for example, until the Angles between the camera plane and the face plane have reached a minimum. Alternatively and/or additionally, the angle of coverage of the camera 185 can be minimized in comparison to the sagittal plane and possibly also the transverse plane, which also includes driving maneuvers of the robot 17, for example.

In an optional aspect, the system is configured to illuminate the body part such as the face in step 3020 . That is, during a photograph by the camera 185, the face (or other body part) is irradiated by an exposure lamp 5120, which illuminates the body part of the person to be photographed. At least one exposure light 5120 is used, which is located close to the camera 185, for example. Ideally, this is below and/or above the camera 185, i.e. it is vertically offset by this camera 185. For example, the emitted light is scattered in order to ensure that the area to be recorded is illuminated as homogeneously as possible. Depending on the position of the face and its dimensions, a lateral arrangement of the camera 185 can possibly lead to a shadow being cast by the nose, which affects the cheeks, which are located to the side of the nose and the recording of which provides an above-average signal-to-noise ratio, and thus possibly worsen the evaluation quality, but would be possible.

The camera 185 used provides at least one color channel for the evaluation, including e.g. at least the green color channel, since the light emitted here is particularly well absorbed by hemoglobin. In another aspect, the camera 185 also provides a color channel for the hue of orange and/or cyan. The color depth per channel is at least 8 bits, for example, and the frame rate is 30 frames per second. In one aspect, the camera 185 can also be an RGB-D camera 185, which provides depth detection in addition to color detection, for example based on time-of-flight sensors or speckle patterns, in order to find out more to record rhythmic, vascular blood flows and rhythmic, vascular expansions.

In the first step 3025 a signal extraction takes place. For this purpose, a selection of the input signal is first made on the basis of the video signals of the tracked regions, which can either be a movement caused by the pumping rhythms of the heart and/or a color change with regard to the flow of blood, especially hemoglobin, whereby said rhythmic , vascular blood flows and/or rhythmic, vascular expansion can be recorded.

In the second step, with a view to the raw data, a color channel evaluation takes place, including known information as to which features are imaged by which color channel, provided that the blood flow is recorded. This means in particular a weighting of the channels in step 3030 . The green color channel can primarily be evaluated, the green and red color channel (e.g. as a difference consideration of the green and red channel), the combination of green, cyan and orange, etc. For the detection of the movements, the spatial resolution is determined alternatively or additionally. This means that the vertical and/or horizontal movements of recorded features of the face are tracked, e.g. the position of the face and its sub-regions are recorded and evaluated over time. This includes both the movements of the head and the individual parts of the face.

The subsequent signal determination uses, for example, at least one filter in the first partial step (preprocessing 3035). This includes trend correction (e.g. with scaling and/or normalization); a moving average view, high-pass filtering; a band-pass filter, possibly configured as an adaptive band-pass filter; an amplitude selective filtering; a Kalman filtering, and/or a continuous wavelet transform. Alternatively and/or additionally, a linear polynomial approximation of the least squares can also be used.

This is followed by a signal separation process 3040 to improve signal-to-noise performance and reduce the number of feature dimensions to consider. Here, for example, a principal component analysis or an independence analysis can be used, in one aspect also methods of machine learning.

Signal processing 3045 includes determining the pulse rate and, if necessary, other variables as part of a Fourier transformation (fast, e.g. discrete Fourier transformation, in particular for determining the maximum power spectrum density), autoregressive models (e.g. using Burg's method), the use of bandpass filters in connection with the Detection of maxima, for example a peak detection, a continuous wavelet transformation, and/or machine learning models, in particular unsupervised learning. Alternatively and/or additionally, a discrete cosine transformation can also be used.

As part of post-processing 3050, methods can also be used to compensate for errors caused by head movements, etc., for which Kalman filters, (adaptive) bandpass filters, outlier detection, moving averages, Bayesian fusion and/or Machine learning methods can be used.

The processing steps that have taken place up to this point already reflect some of the relevant parameters, such as the pulse rate, pulse rate variability, pulse transit time, pulse waveform, etc. A further calculation of relevant parameters in step 3055, for example, is based on various approaches described in the prior art, which For example, the systolic and diastolic blood pressure can be determined, for which linear or non-linear prognosis methods can be used.

The machine learning methods mentioned, e.g. neural networks such as convolutional neural networks, are able to recognize hidden and partly unknown features in the data and to take them into account in the evaluation, e.g. in the context of cluster analyses. For example, weights for the classifications or linear or non-linear forecast models are generated on the basis of training data, which are then used in productive operation as part of the process described. For example, a multi-layer feed-forward-backward propagation network can be used as a neural network with each input parameter as a neuron, one neuron for each output, e.g. for the systolic and diastolic blood pressure, which uses e.g. two hidden levels, the first Level at least 30, the second at least 15 neurons.

In one aspect, the determined values for the pulse rate, pulse rate variability and possibly other values, possibly after post-processing has taken place, are compared with values stored in memory 10 in step and the pulse rate, pulse rate variability and other variables are determined based on this, in particular also the systolic and diastolic blood pressure.

The filtering as part of the pre- and post-processing depends on the variables to be detected. The pulse amplitude can, in one aspect, use a bandpass filter covering the 0-6 Hz spectrum, e.g. at least 0.7-4.5 Hz. In one aspect, the signal of the pulse can also be more narrowly sampled in this frequency range , e.g. with a window of 0.1 Hz. Smoothing can then be carried out using a low-pass filter. The pulse rate, heart rate or pulse frequency or heart frequency can be processed, for example, by means of a bandpass filter that has a width between 0.7 and 4 Hz. In order to determine the pulse rate variability, a bandpass filter with a window between 0 and 0.4 Hz can be used, sampled with intervals of 0.02 Hz in one aspect Bandwidth between 0.5 and e.g. 6 Hz can be evaluated, in one aspect sampled at intervals of 0.1 Hz. By comparing the values of several regions, the pulse wave propagation time can be determined. The pulse shape results from an unsampled course in the spectral window of approx. 0-6 Hz, characterized for example by the area under the curve, the height and/or width. The pulse energy results from the first derivative of these values.

For example, the blood pressure can be determined using a linear model from the transit time of the pulse wave and the pulse or heart rate as well as the previous blood pressure value, for which linear regression models or neural networks can be used. Instead of a previous blood pressure value, the shape of the measured pulses can also be evaluated, e.g. by determining the differences between the pulse curve and the vertical line running through the maximum value.

The system for determining blood pressure is, as in 5 set out, illustrated as follows: The system for determining cardiovascular parameters of a person, in one aspect a service robot 17, comprises a computer 9, a memory 10 and a camera 185 (e.g. a 2D and/or 3D camera) and also a body region detection module 4810 for detecting body regions, a body region tracking module 4815, a face recognition module 5005, a face candidate region module 5010, and a cardiovascular movement module 5110 for detecting movements attributable to cardiovascular activity. The camera 185 (identical in one aspect to the sensor for non-contact detection of a person 3) provides at least the 8-bit green color channel. The system also includes an exposure light 5120 to illuminate the face during recording by the camera 185 located, for example, above and/or below the camera 185. The system has a blood pressure determination module 5125 for determining the systolic or diastolic blood pressure 5125 and/or, for example, a tilting unit 5130 in order to minimize the angle of coverage of the camera 185 compared to the sagittal plane. For this purpose, the system has rules, for example, to place a vertical line between the eyes of the person detected, so that the head is divided into two halves. The face is segmented, with histograms of gradients being placed over the individual segments. If these show a (mirror-inverted) similarity that is below a threshold value, the face is perceived as vertical. The camera 185 can now be connected to an optional tilting unit 5130 be controlled in such a way that during the control this comparison of the mirror-inverted halves of the face is carried out via the histograms-of-gradients and the camera is positioned in such a way that these threshold values of the histograms-of-gradients are not reached. In one aspect, the system has a person recognition module 110, person identification module 111, a visual person tracking module 112, movement evaluation module 120, optionally a skeleton creation module 5635 and optionally a skeleton model-based feature extraction module 5640 and/or a movement planner 104

In contrast to an optical detection of the vital parameters, it is also possible to detect the vital parameters using radar. In one aspect and as described above, a body region is identified and tracked, for example, in which the vital data is recorded,

Triggering embodiment elements based on cardiovascular indicators

The service robot 17 calculates the pulse rate or pulse frequency, heart rate or another cardiovascular parameter of a person with whom the service robot 17 interacts, in that the service robot 17 uses the system described to record and evaluate the parameter by recording and evaluating the face and the cardiovascular movements reflected therein the facial surface and/or the head and/or vascular blood flows under the skin are determined, e.g. as described above. Alternatively, the service robot receives vital data and/or movement data from the portable sensor 201, which can also create an electrocardiogram (ECG), for example.

As an alternative and/or in addition to the at least one cardiovascular parameter, the service robot 17 can use the camera 185 or the ultrasonic and/or radar sensor 194 to identify and track a person's chest, which can be done, for example, using a framework such as OpenPose, OpenCV, etc. can be done, e.g. by means of the visual person tracking module 112 and/or the laser-based person tracking module 113. For example, skeleton model joint points from the skeleton creation module 5635 can be used to identify body regions that are then to be evaluated. For example, the upper body can be identified via the shoulder joint points and the hip joint points by determining the area that results from these points and then defining the upper area of this area, e.g. the upper 50%, as the chest area. Alternatively or additionally, this approach can be supplemented by evaluating depth information that reflects the outer shell of a person. Camera 185 and the two person tracking modules 112 and 113, but possibly also other sensors such as LIDAR 1, are also referred to as person detection and tracking unit 4605. With this person detection and tracking unit 4605, the service robot 17 can detect movements of a person over time, which reflect breathing, for example. In the event that the person should be located approximately frontally in front of the service robot, this detection includes both movements in the horizontal direction and in the depth. These movements can be determined, for example, using a bandpass filter with a window of 0.005 to 0.125 Hz, at least 0.05 to 0.08 Hz, and subsequent fast Fourier transformation. This can be used to determine the respiratory rate.

If the person's chest area is covered by clothing, making direct detection of the chest difficult, for example, the respiratory rate can be determined, for example based on Martinez et al. 2017, "Breathing Rate Monitoring during Sleep from a Depth Camera under Real-Life Conditions, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 24-31 March 2017 (DOI: 10.1109/WACV.2017.135). In this case, an interference reduction 2980 is carried out with regard to the detection of movements that are covered by fabric, that is to say primarily of the upper body/stomach. Interferences, which are caused by different phases of the recorded movements of the substance and make the actual respiratory movements more difficult to detect, are eliminated by determining the power density spectrum, which makes it easier to record the movements of the chest. The power density spectrum for each pixel recorded over time in the three-dimensional space is determined, for example using a fast Fourier transformation (FFT) in step 2982, then the power density spectra for all pixels are aggregated in step 2984 and the maximum is determined using quadratic interpolation in step 2986, where the position of the maximum indicates the respiratory rate in step 2988. This is then monitored 2990 with a view to frequency changes. The subsequent feature classification in step 2992 is based on a classification that was generated by recording breathing movements, for which purpose standard classification methods and/or neural networks can be used. A description of these processes can be found in 6 .

The determined pulse or respiratory rate (possibly also heart rate or other cardiovascular parameters) of the person detected by the robot are used to synchronize movements or signals from the service robot 17 with the named parameters of the person. In one aspect This synchronization can be used, for example, to calm down the person whose parameters are mirrored by the service robot 17 .

The movements and/or signals of the service robot 17 take place in one aspect via embodiment elements, i.e. elements that at least partially reproduce a person, such as a stylized head or parts of it such as stylized eyes, mouth, etc. For example, blinking can be more stylized eyes to be adjusted. Stylized means, for example, that with regard to eyes, eyes can be implemented in hardware, for example spheres with circles and hemispheres printed on them, which can mechanically cover the circles printed on the spheres. Alternatively, the eyes can also be shown on a display, for example in the form of circles, etc. Looking at a mouth, this can be defined by a line, for example like a smiley, which can assume different orientations and/or curvatures. Alternatively and/or additionally, if the service robot 17 has a stylized and movable chest, its movement frequency can also be adjusted. In an alternative or supplementary aspect, the embodiment elements are implemented, for example, by a flashing light 200 of the service robot 17, for example an LED, which in one aspect can be red and which thus stylizes a heart. This lamp 200 is located, for example, inside the service robot 17 or on its surface, whereby it does not necessarily have to be located in the area that a human being must see as the chest area of an embodied service robot. It can therefore also be located in the upper or lower area of the service robot 17 . In addition to the heartbeat, it can also stylize breathing. If it is inside the service robot 17, its light can be seen from the outside. For this purpose, the surface of the service robot 17 can be at least partially transparent, for example. In an alternative and/or supplemental aspect, the hue or brightness of a display is varied to stylize a heartbeat or breathing.

In this case, movement synchronization means that the service robot 17 adapts the described movements and/or signals to those of the person by the signaling frequency of the stylized embodiment elements approximately corresponding to the measured frequency of the vital parameters such as respiration, pulse, heartbeat, etc. This occurs, for example, not only statically, but dynamically, i.e. over time. Alternatively, movement synchronization can also mean that the service robot 17 does not choose exactly the same frequency that was determined for the person, but a lower frequency. This frequency can be tracked over time, i.e. the difference is kept approximately constant over time. The service robot can also reduce the frequency of movements / signaling over time. This can be the case, for example, if the person has an above-average pulse or breathing rate, which indicates, for example, high levels of nervousness (alternatively, other determined parameters that indicate high levels of nervousness). By selecting a movement frequency or lighting frequency (oscillation frequency) of, for example, the stylized eyes, chest, lamp 200 or other elements, the service robot can choose a frequency that is lower than that of the person, and which the service robot 17 reduces over time. The service robot 17 can thus contribute to calming down the person. The service robot 17 determines, for example, the pulse rate of the person over time and, if necessary, reduces its movement frequency or lighting frequency until the person also has a normal pulse rate, in which case the normal pulse rate can be stored in the memory 10 of the service robot 17. The difference between the detected pulse frequency and the movement frequency/lighting frequency of the service robot 17 can remain approximately constant. This is done by determining a frequency bandwidth around the determined movement frequency of the person, which includes, for example, the respiratory rate, heartbeat or pulse rate.

The pulse rate or pulse rate, heart rate or heart rate and/or respiration or respiratory rate is detected by means of a movement frequency determination unit 4606, which, for example, from the camera 185 and computer-implemented methods for determining the pulse rate or pulse frequency described elsewhere in this document, Heart rate or heart rate and/or respiration or respiratory rate, although other movements of the person are also conceivable. The parameters for pulse, heart and/or respiration are specifically recorded and evaluated by a pulse-respiration evaluation unit 4615 . Breathing in particular is recorded and evaluated via a movement signal detection and processing unit 4620, which distinguishes background signals from the body from those from the clothing. Details are given elsewhere in this document with a view to signal processing. Said stylized faces or facial elements, heads, trunks or chests, heart, lamp(s) 200 are also referred to as stylized embodiment elements 4625. These are moved or controlled by an initialization unit 4607 with a specific frequency. Depending on the type of stylized embodiment element, this can take place in different ways. Eye movements on a display, for example, can be implemented purely based on software, while other stylized embodiment elements 4625, for example, servomotors require, for example, move eyelids or move a stylized chest. In one aspect, the system also has a person recognition module 110, person identification module 111, tracking module (112, 113) and/or motion evaluation module 120. In an optional aspect, the system has a wearable sensor 201 such as a smartwatch, e.g is worn by the robot sensory detected person. In one aspect, this can include the pulse-breath evaluation unit 4615 and/or the movement signal acquisition and processing unit 4620 . An overview of the components of the system is in 7 set forth. In one aspect, the system can also be used together with a skeleton creation module 5635, skeleton model-based feature extraction module 5460, possibly. gait feature extraction module 5605, gait feature classification module 5610, and gait progression classification module 5615 may also be employed to monitor the frequency of the subject's parameters during walking or other fitness exercise.

The process follows 8th as follows: Movements and/or vital parameters of a person are recorded (step 6000), which includes, for example, pulse rate, heart rate or respiratory rate. The recorded movements and/or vital parameters of the person are tracked, ie recorded over time (step 6005). In an optional aspect 6010, the detected parameters are bandpass filtered and the detected parameters are Fourier transformed 6015 . (Thus) the frequency of the movements and/or vital parameters is determined (step 6020). There is an optional determination of a bandwidth 6025 around the determined frequency, for example based on a table stored in a memory (eg 10) which assigns determined frequencies to bandwidths. The bandwidth of the frequency initiated by the initialization unit (4607) from the determined frequency of the person can, for example, move downwards and/or upwards in an interval of 50% or downwards and/or upwards in an interval of less than 15%, e.g. less than 5% down and 0% up. Furthermore, there is a movement and/or activation 6030 of stylized embodiment elements, e.g. with a frequency that lies within the determined bandwidth, which corresponds to the determined frequency, etc. This can include, for example, the oscillation of a lamp 200, the movement of the eyes of a service robot 17, etc. In an optional aspect, the control frequency is lowered 6035, for example down to a threshold value that is stored in a memory (eg 10). In this case, in one aspect, the difference between the determined and the controlled frequency can be kept constant (step 6040). Alternatively and/or additionally, the activation frequency can be kept lower than the recorded frequency of the movements and/or vital parameters of the person.

Example 1 Method for non-contact recording of a person's vital parameters

In one aspect, the service robot 17 has rules stored in the memory 10 for carrying out a computer-implemented method for triggering events based on the observation of a person and vital data determined by this person. For this too 9 : In a first step 9000, a person is recorded and tracked. In a subsequent step 9010, a body region is recorded and tracked from which vital data are to be recorded, which are then recorded in the subsequent step 9020. In parallel, in an optional step 9015, body movements of the person are recorded (e.g. by evaluating data from a skeleton model via the skeleton creation module 5635) and an optional comparison of the recorded vital data and the recorded body movements takes place in the optional step 9025 . This combination of data makes it possible, for example, to identify whether the person is overloaded by evaluating a person’s movement (e.g. during a test or exercise) on the one hand, and vital data such as the pulse on the other. By correlating this data, possibly with a time delay (i.e. first determining movements over time and then subsequently an increase in the person’s pulse), the service robot 17 can draw the conclusion that the increase in pulse was caused by the intensity of movement of the person. This can lead, for example, to the optional step 9050 of determining the person's risk of falling. This is, for example, an acute risk of falling, caused, for example, by a sudden drop in blood pressure. Based on this, an event can be triggered (step 9060), whereby this event can also be reached without triggering step 9050. In an optional aspect, the recorded vital parameters from step 9020 are evaluated in optional step 9030 such that a threshold value is determined based on recorded parameters. In one aspect, this can be done in such a way that vital parameters are recorded over time, i.e. dynamically, and a threshold value for a defined gradient is determined from the rise in the curve that represents the recorded parameters over time, with the threshold value then being in the unit of determined vital parameter is indicated. Alternatively and/or additionally, the vital parameter can also be averaged over a defined period of time and this value can then be used as a threshold value, or this average value can be compared with the threshold value. In a further step 9040, recorded vital parameters are included compared to the threshold value (step 9040) and if the threshold value is exceeded or not reached, a fall risk is optionally determined (step 9050), followed by the triggering of an event (step 9060).

The events described from step 9060 can, for example, involve reducing the speed of the service robot 17 or driving to a position where, for example, there is a seat for the person whose vital data are being recorded, so that the person can put there. However, another system can also be notified via an interface 188, for example an external alarm system. The vital data is recorded either contactlessly via sensors of the service robot 17 or via a sensor 201 located on the person. The vital parameters are, for example, the pulse rate, pulse rate variability, heart rate, heart rate variability, the systolic and/or diastolic blood pressure and/or or the respiratory rate.

• ABEV1. Computer-implementiertes Verfahren zur Steuerung eines Serviceroboters (17) unter Einbezug von Vitalparametern einer Person, umfassend
  • • Erfassung und Tracking der Person;
  • • Erfassung und Tracking einer Körperregion der Person, auf bzw. über die die Vitalparameter erhoben werden;
  • • Erfassung der Vitalparameter und
  • • Vergleich der erfassten Vitalparametern mit mindestens einem hinterlegten Schwellwert und
  • • Auslösen eines Ereignisses bei Über- oder Unterschreiten des Schwellwert.
• ABEV2. Computer-implementiertes Verfahren nach ABEV1, wobei das Ereignis eine Geschwindigkeitsreduktion umfasst.
• ABEV3. Computer-implementiertes Verfahren nach ABEV1, wobei das Ereignis das Ansteuern einer Zielposition umfasst.
• ABEV4. Computer-implementiertes Verfahren nach ABEV3, wobei es sich bei der Zielposition um eine Sitzgelegenheit handelt.
• ABEV5. Computer-implementiertes Verfahren nach ABEV1, wobei der mindestens eine Schwellwert aus zuvor erfassten Vitalparametern dynamisch ermittelt wird.
• ABEV6. Computer-implementiertes Verfahren nach ABEV5, wobei der dynamisch ermittelte Schwellwert basierend auf einer Mittelwertbildung erfasster Vitalparameter über ein definiertes Zeitintervall erfolgt.
• ABEV7. Computer-implementiertes Verfahren nach ABEV1, wobei die Erfassung der Vitalparameter berührungslos erfolgt.
• ABEV8. Computer-implementiertes Verfahren nach ABEV1, wobei die Erfassung der Vitalparameter durch einen an der Person befestigten Vitalparameter-Sensor (5425) erfolgt.
• ABEV9. Computer-implementiertes Verfahren nach ABEV1, weiter umfassend die Erfassung von Körperbewegungen der Person und eine Auswertung der erfassten Vitalparameter unter Abgleich der erfassten Körperbewegungen.
• ABEV10. Computer-implementiertes Verfahren nach ABEV1, wobei die Erfassung von Vitalparametern in Abhängigkeit der erfassten Bewegungen der Person erfolgt.
• ABEV11. Computer-implementiertes Verfahren nach ABEV1, wobei die erfassten Vitalparameter die Pulsrate, Pulsratenvariabilität, Herzrate, Herzratenvariabilität, den systolische und/oder diastolischen Blutdruck und/oder die Atemfrequenz umfassen.
• ABEV12. Computer-implementiertes Verfahren nach ABEV1, weiter umfassend die Ermittlung des Sturzrisikos aus den erfassten Vitalparametern.
• ABEV13. Computer-implementiertes Verfahren nach ABEV12, wobei es sich beim Sturzrisiko um ein akutes Sturzrisiko handelt.
• ABEV14. Computer-implementiertes Verfahren nach ABEV1, wobei die Erfassung der Vitalparameter während der Durchführung eines Tests und/oder eine Übung mit der Person erfolgt.
• ABEV15. Vorrichtung zur Durchführung eines Verfahrens nach ABEV1-ABEV14.
• ABEV16. Serviceroboter (17) mit einem Rechner (9), einem Speicher (10) und mindestens einem Sensor zur berührungslosen Erfassung der Bewegungen der Person im Zeitablauf, und ein Vitalparameter-Auswertungs-Modul (5420), wobei es sich beim Sensor zur Erfassung der Bewegungen der Person um eine Kamera (185), einen LIDAR (1), einen Ultraschall- und/oder Radarsensor (194) handelt und wobei das Vitalparameter-Auswertungs-Modul (5420) eine Benachrichtigung eines externen Systems über eine Schnittstelle (188), eine Ausgabe über eine Ausgabeeinheit (2 oder 192), eine Geschwindigkeitsänderung des Serviceroboters 17 und/oder eine Ansteuerung einer Zielposition des Systems initiiert.
• ABEV17. Serviceroboter (17) nach ABEV15, weiter umfassend ein Körperregions-Detektions-Modul (4810) und ein Körperregion-Tracking-Modul (4815) zum Tracking der Erfassungsregion der Vitalparameter und eine Vitalparameter-Erfassungs-Einheit (5415) zur Erfassung von Vitalparametern der Person.
• ABEV18. Serviceroboter (17) nach ABEV15, umfassend ein Applikationsmodul (125) mit Regeln zur Durchführung mindestens einer Übung oder eines Tests.
• ABEV19. Serviceroboter (17) nach ABEV15, umfassend eine Schnittstelle (188) und einen an einer Person befestigten Vitalparameter-Sensor (5425).

The non-contact recording of vital parameters is characterized by the following aspects ABEV1 to ABEV20:

• ABEV1. Computer-implemented method for controlling a service robot (17) using vital parameters of a person, comprising
  • • Registration and tracking of the person;
  • • Recording and tracking of a body region of the person on or via which the vital parameters are collected;
  • • Registration of vital signs and
  • • Comparison of the recorded vital parameters with at least one stored threshold value and
  • • Triggering of an event when exceeding or falling below the threshold value.
• ABEV2. Computer-implemented method according to ABEV1, where the event includes a reduction in speed.
• ABEV3. Computer-implemented method according to ABEV1, wherein the event comprises driving to a target position.
• ABEV4. Computer-implemented method according to ABEV3, where the target position is a seat.
• ABEV5. Computer-implemented method according to ABEV1, wherein the at least one threshold value is determined dynamically from previously recorded vital parameters.
• ABEV6. Computer-implemented procedure according to ABEV5, whereby the dynamically determined threshold value is based on an averaging of recorded vital parameters over a defined time interval.
• ABEV7. Computer-implemented procedure according to ABEV1, whereby the vital parameters are recorded without contact.
• ABEV8. Computer-implemented method according to ABEV1, in which the vital parameters are recorded by a vital parameters sensor (5425) attached to the person.
• ABEV9. Computer-implemented method according to ABEV1, further comprising the detection of body movements of the person and an evaluation of the recorded vital parameters by comparing the recorded body movements.
• ABEV10. Computer-implemented method according to ABEV1, whereby vital parameters are recorded as a function of the recorded movements of the person.
• ABEV11. Computer-implemented method according to ABEV1, wherein the recorded vital parameters include the pulse rate, pulse rate variability, heart rate, heart rate variability, the systolic and/or diastolic blood pressure and/or the respiratory rate.
• ABEV12. Computer-implemented procedure according to ABEV1, further comprising the determination of the risk of falling from the recorded vital parameters.
• ABEV13. Computer-implemented method according to ABEV12, where the fall risk is an acute fall risk.
• ABEV14. Computer-implemented method according to ABEV1, in which the vital parameters are recorded while a test is being carried out and/or an exercise is being carried out on the person.
• ABEV15. Device for carrying out a method according to ABEV1-ABEV14.
• ABEV16. Service robot (17) with a computer (9), a memory (10) and at least one sensor for contactless recording of the movements of the person over time, and a vital parameter evaluation module (5420), the sensor for recording the movements being the person is a camera (185), a LIDAR (1), an ultrasonic and / or radar sensor (194) and wherein the vital parameter evaluation module (5420) a notification of an external system via an interface (188), a Output via an output unit (2 or 192), a change in speed of the service robot 17 and/or activation of a target position of the system.
• ABEV17. Service robot (17) according to ABEV15, further comprising a body region detection module (4810) and a body region tracking module (4815) for tracking the acquisition region of the vital parameters and a vital parameter acquisition unit (5415) for acquiring vital parameters of the person .
• ABEV18. Service robot (17) according to ABEV15, comprising an application module (125) with rules for carrying out at least one exercise or one test.
• ABEV19. Service robot (17) according to ABEV15, comprising an interface (188) and a vital parameter sensor (5425) attached to a person.

Reference List

1

LIDAR

2

screen

3

Sensor for non-contact detection of a person

4

pressure sensitive bumper

5

jockey wheel

6

drive wheel

7

drive unit

8th

energy source

9

calculator

10

Storage

17

service robot

13

terminal

18

cloud

100

software level

101

navigation module

102

2D or 3D environment detection module

103

path planning module

104

motion planner

105

Self-Location Module

106

mapping module

107

map module

108

loading module

110

People Recognition Module

111

Personal Identification Module

112

visual people tracking module

113

laser-based people tracking module

114

Person re-identification module

115

seat detection module

120

motion evaluation module

121

Trajectory Extraction Module

122

Trajectory Evaluation Module

130

Human-Robot Interaction Module

131

graphical user interface

132

language evaluation module

133

speech synthesis unit

150

set of rules

151

Calculator of the rules

152

Storage of the ruleset

160

People administration module

161

Computer of the person administration module

162

Storage of the people administration module

170

Navigation module in the cloud

171

navigation computer

172

navigation memory

180

hardware level

181

odometry

183

RFID

185

camera

186

controls

188

interface

190

charge control

191

engine control

192

speaker

193

microphone

194

Radar and/or ultrasonic sensor

195

detector

196

spectrometer

200

lamp

201

Wearable sensor

4605

Person detection and tracking unit

4606

movement frequency determination unit

4607

initialization unit

4615

Pulse-breath evaluation unit

4620

Motion signal detection and processing unit

4625

stylized embodiment elements

4630

excitation unit

4810

Body Region Detection Module

4815

Body region tracking module

5005

Face Recognition Module

5010

Face Candidate Region Module

5110

cardiovascular movements module

5120

exposure light

5125

Blood pressure determination module

5130

tilting unit

5415

Vital Signs Recording Unit

5420

Vital parameters evaluation module

5425

Vital signs sensor

5605

Gait Feature Extraction Module

5610

Gait feature classification module

5615

Gait Classification Module

5620

inertial sensor

5635

Skeleton Creation Module

5640

skeletal model-based feature extraction module

QUOTES INCLUDED IN DESCRIPTION

This list of documents cited by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent Literature Cited

• CN106671105 [0004]
• CN104889994 [0005]
• CN204772554 [0005]
• CN105082149 [0005]
• CN105078445 [0005]
• CN105078450 [0005]
• CN108053889 [0006]
• CN108039193 [0006]
• CN107544266 [0006]
• CN106709254 [0007]
• CN 106407715 [0008]
• CN105078449 [0009]

Claims (15)

System for synchronizing the movements and/or vital parameters of a person with the movement and/or lighting frequency of stylized embodiment elements (4625) of a service robot (17), comprising a person detection and tracking unit (4605), a pulse-breath evaluation unit (4615), and an initialization unit (4607), the movement occurring at a frequency or the luminous frequency corresponding to a frequency that lies within a defined bandwidth around the determined frequency of the movements or vital parameters of the person. system after claim 1 , further comprising a movement signal detection and processing unit (4620) for detecting and evaluating detected movements of the person. system after claim 1 , wherein the stylized embodiment elements (4625) are implemented in hardware and/or software. system after claim 3 wherein stylized embodiment elements (4625) implemented in software comprise the display of at least one stylized face or face element on a display (2). system after claim 3 wherein hardware-implemented stylized embodiment elements (4625) include at least one stylized face, face element, or a torso, rib cage, or heart. system after claim 3 wherein stylized embodiment elements (4625) implemented in hardware include at least one light (200). system after claim 1 , wherein the stylized embodiment elements are stimulated to imitate the respiratory rate, pulse rate and/or heart rate by the initialization unit (4607). system after claim 1 and/or 2, wherein the pulse-breath evaluation unit (4615) and/or a motion signal detection and processing unit (4620) are located in a portable sensor (201). Computer-implemented method for synchronizing the movements and/or vital parameters of a person and a service robot (17), comprising • Capture and tracking of an individual and their movements and/or vital signs; • determining the frequency of movements and/or the vital parameters of the person; · Variation of a movement frequency and/or a lighting frequency of stylized embodiment elements (4625) of the service robot with a frequency that lies within a defined range of the determined frequency of the recorded movements and/or vital parameters of the person. Computer-implemented method claim 9 wherein the movement of stylized embodiment elements (4625) mimics movement of a face, facial element, torso, chest, or heartbeat and/or breathing of the person. Computer-implemented method claim 9 , the frequency difference between the movements of the stylized embodiment elements (4625) and the determined frequency of the movements and/or vital parameters of the person being kept approximately constant over time by the initialization unit (4607). Computer-implemented method claim 9 , wherein the detected movements and/or vital parameters of the person are the pulse rate, heart rate and/or respiratory rate. Computer-implemented method claim 9 , wherein the movements of the stylized embodiment elements (4625) are adjusted by the initialization unit (4607) to a frequency that is lower than the frequency of the detected movements and/or the vital parameters of the person. Computer-implemented method claim 9 , wherein the frequency of the stylized embodiment element(s) (4625) is controlled by the initialization unit (4607) to become slower over time. Device for carrying out the method according to claims 8 - 14 .

Images