RU2781179C2 - Computer-implemented method for analysing man-related emergency data - Google Patents

Abstract

FIELD: computing technology.

SUBSTANCE: technical result is achieved through the analysis of man-related emergency data (10) of one or more users of the vehicle (3), wherein the emergency data (10) includes at least the data of images from the video footage of the user of the vehicle (3) during the emergency. The method includes at least the first stage (S1) of recognising the pattern (M(x)) based on the image data and at least the second stage (S2) of comparing the recognised pattern (M(x)) with a number of presaved patterns (M(1)…M(n)), wherein the presaved patterns (M(1)…M(n)) correlate with the medical diagnostic data (D(1)…D(p)), respectively, wherein the method further includes the third stage (S3) of outputting the medical diagnostic data (D(j)) correlating with the recognised pattern (M(j)).

EFFECT: higher degree of recognition of trauma caused by a traffic accident.

10 cl, 3 dwg

Description

The present invention relates to a computer-implemented method for analyzing human-related emergency data of one or more vehicle users, as well as an apparatus for performing computer-implemented analysis of human-related emergency data of one or more vehicle users.

As a result of a traffic accident, the driver of the vehicle or other users of the vehicle may suffer various injuries. As a result of an emergency, in particular, a collision with an obstacle, the cervical spine (due to the sudden and unforeseen movement of the head relative to the fixed trunk), the head and chest itself, as well as the trunk and limbs are especially susceptible to injury. Various shaking movements of the head relative to the torso in an emergency when held by a seat belt occur, for example, for a very short time in the range of hundreds of milliseconds. The resulting injuries and/or medical symptoms are referred to as whiplash (also Cervical Spine Distortion or Whiplash Syndrome) and can range in severity from overstressed ligaments to spinal fractures or fatal injuries.

However, adverse impact angles can also cause the seat belt functions to fail, resulting in unlimited multiple injury that can be proven on pictures. However, most injuries that occur cannot be displayed, or only with difficulty, in conventional imaging medical techniques, such as x-ray diagnostics, which, among other things, can make it difficult for medical diagnosis and the provision of evidence for long-term damage insurance claims.

On the other hand, with many injuries, it is important to provide timely, injury-oriented medical treatment in order to heal the patient as well as possible and to prevent or at least reduce possible long-term consequences.

Patent application EP18169880.4, unpublished as of the filing date of the present application, describes a system and method for recording human-related emergency data in a vehicle. Meanwhile, during a vehicle emergency such as a motor vehicle, image data of the user or users of the vehicle is recorded and captured by the high-speed camera. The image data can then be used to analyze the accident from a medical point of view and thus allow conclusions to be drawn about possible injuries to vehicle users.

It is an object of the present invention to provide an alternative or even an improvement to a computer-implemented method for analyzing human-related crash data, and an apparatus for performing computer-implemented analysis of human-related crash data of one or more vehicle users.

This problem is solved by a computer-implemented method for analyzing human-related emergency data of one or more vehicle users, in accordance with claim 1, a device for performing computer-implemented analysis of human-related emergency data of one or more vehicle users. according to paragraph 11 of the claims and the emergency assistance system according to paragraph 13 of the claims. In this case, the devices can also be improved on the basis of the features of the method listed below or in the dependent claims, or vice versa, or the features listed below or in the dependent claims can also be used together for further improvement.

The computer-implemented method of the invention is used to analyze human-related crash data of one or more vehicle users, wherein the human-related crash data includes at least video image data of the vehicle user at the time of the crash. The method includes at least one first step of pattern recognition based on image data and at least one second step of comparing the recognized pattern with a number of previously stored patterns, preferably a plurality of previously stored patterns.

By "near" in the context of this annex should be understood "one or more", while the term "many" means several, i.e. more than one.

In this case, the "pattern" or even "template of an emergency" refers to the characteristics extracted from the image data, and characterize the circumstances of the accident in relation to the user of the vehicle. Such characteristics can be, in particular, kinematic data.

In this case, the video sequence preferably includes a sufficient number of images, i. e. enough image data to be able to detect a pattern from it, as described below. For this, the footage may, for example, be provided with a high-speed camera, in which the high-speed camera is designed to record a large number of frames per second (fps), such as at least 700 frames per second or at least 1000 frames per second. In this case, the video sequence preferably includes a sufficiently long recording time to record the entire sequence of movement of one or more vehicle users during an emergency, and, if necessary, a short period of time before and/or during an emergency. For example, a video sequence may correspond to a recording time of 10 or 15 seconds.

The result of the comparison carried out in the second step may be, for example, that the recognized pattern matches a previously stored pattern, where the previously stored pattern may in particular be based on image data recorded in previous accidents. In this case, that is, if the pattern matches, additional information can be provided that was determined from the previous alarm. Therefore, for example, you can use empirical values, such as, for example, injuries resulting from previous accidents. Since pattern detection and pattern matching are automatic, better results can be achieved than human case analysis, for example, if the first assistant and/or attending physician analyzes the provided visual sequence only visually or without computer support.

The registered template can, for example, also be used to improve evidence for insurance policy payments, for example in connection with long-term damages.

The coincidence of the previously saved and registered template in this case should not have a 100% correlation. In this case, it may be only a partial match, for example, within a previously established confidence interval, such as 95% or 90%. In addition, a comparison unit may, for example, be configured to recognize a pattern that respectively partially matches two previously stored patterns, and to provide any data or information messages that match those two previously stored patterns.

Preferably, the recognized and/or previously stored pattern includes the motion sequence of the vehicle user during the emergency, in particular the temporal sequence of acceleration processes of the head and/or neck and/or chest and/or torso and/or limbs of the vehicle users and /or time sequence of movement of the head and/or neck and/or torso and/or chest.

Such a motion sequence is provided by a time sequence of the individual images of the footage, in which the individual images are evaluated and combined into a corresponding motion sequence. This movement sequence may, for example, refer to individual or several body parts of the vehicle user (eg head, neck, chest, torso and/or limbs) or also to the relative movements of individual or several body parts to each other. In this way, individual images can, for example, be evaluated in terms of the respective position of the respective body part.

Under the acceleration or even the process of acceleration, in the context of this presentation are understood as positive, i.e. increasing speed, and negative, i.e. decelerating speed, acceleration or acceleration processes.

Because crash injuries are essentially caused by one or more of these motion sequences, such a pattern that is based on or contains motion sequences can be a good indication of emerging injuries. Thus, automatic evaluation of image data, i.e. Pattern recognition based on image data can, for example, allow an assessment of potential damage prior to or in coordination with the initial investigation. As a result, for example, initial treatment can be tailored as best as possible to the injury or injuries and/or the risk of missing any injuries can be reduced and/or injuries or even different parts of the body can be treated, treated or observed over time depending on the severity. injuries (expected). Therefore, for example, it is generally possible to provide better medical (initial) care for a person involved in an emergency.

Preferably, the previously stored templates are respectively associated with the medical diagnostic data, and the method further includes a third step of outputting the medical diagnostic data that is associated with the recognized template. Furthermore, preferably, the medical diagnostic data includes at least a medical diagnosis and/or medical long-term consequences information and/or medical diagnostic data constituting an indication of the degree of possible injury severity of the vehicle user. For example, with the aid of such medical diagnostic data, possible injuries to the user of the vehicle can be detected and/or assessed in a simple manner, in particular in advance, i. before the initial examination. For example, an injury severity metric value can be provided for different body regions and/or body parts, and a timely injury sequencing for an emergency victim can be established using the corresponding metric values. With the method, diagnostic data can be provided quickly and in a timely manner. This may be especially important for life-threatening injuries. In addition, the medical diagnostic data can be used, for example, as support for conventional medical diagnostic methods, in particular image-providing diagnostic methods, such as, for example, x-ray diagnostics, for example, to facilitate and/or improve medical diagnosis.

Furthermore, it is preferred that the human-related emergency data includes the physical operational data of the vehicle, i. e. its speed and/or acceleration immediately before and/or during the emergency. Thus, one can, for example, use this in addition to image data for pattern recognition.

Preferably, the method according to the invention is carried out with an algorithm for implementing machine learning, in which the algorithm is further preferably suitable for recognizing in advance, using a provided set of human-related emergency data, a set of patterns from image data that is stored as previously stored templates.

Machine learning, in this case, in general, refers to the generation of knowledge (in this case, the extraction of certain patterns from human-related emergency data, in particular image data, and possibly in connection with medical diagnostic data) from experience (t .e. earlier accidents). Basically, the algorithm is designed for learning examples, i.e. previous accidents, extracting the corresponding patterns, and then evaluating unknown human-related accident data, in particular image data.

In order to learn the patterns, which are then stored as previously stored patterns, the algorithm is provided, preferably in advance, with a set of human-related emergency data (i.e., a set of human-related emergency data), in particular image data, from previous accidents. situations, especially in relation to relevant medical diagnostic data. After this learning step, the algorithm is able to at least partially implement the method described above, in particular to recognize a pattern from the provided human related crash data, and correlate it to, for example, a specific injury and/or injury category. For example, various injuries can be grouped into injury categories.

With such an algorithm for implementing machine learning, it is possible, for example, to carry out the method described above at least partially automatically. In this case, in particular, a large amount of data (previously provided human-related emergency data related to previous emergencies) can be presented, on the basis of which previously provided templates are determined. In addition, for example, in the event of an emergency situation, the appropriate template is determined quickly and easily.

Preferably, previously saved templates are stored in the emergency database. In this case, on the one hand, it is possible, for example, to provide access to previously saved templates. On the other hand, such a database allows, for example, updating and/or expansion, for example by storing additional and/or newly registered templates and/or by adding additional medical diagnostic data. In particular, the above mentioned machine learning algorithm can be designed to store a newly recognized pattern in a database.

The device according to the invention is used for computer analysis of human related emergency data of one or more vehicle users, wherein the human related emergency data includes at least video image data of the vehicle user during the emergency. The apparatus includes a recognition unit for pattern recognition using image data, and a comparison unit for comparing the recognized pattern with a number of previously stored patterns. Preferably, the device further includes an output unit for outputting medical diagnostic data associated with the recognized pattern. In this way, for example, it is possible to achieve the effects described above with respect to the computer-implemented method also for the device.

The emergency assistance system according to the invention includes an apparatus as described above for computer-implemented analysis of human-related data about an emergency, and a high-speed camera for collecting image data of video footage of vehicle users during an emergency, and a data memory, preferably a ring buffer for storing image data recorded by a high-speed camera. Preferably, the emergency assistance system also includes an operational data logging functional block for recording the physical operational data of the vehicle. Therefore, it is possible, for example, to provide information about a possible injury to a vehicle user more quickly.

Further features and objects of the invention follow from the description of illustrative examples using the accompanying figures of the drawing.

Fig. 1 is a schematic view of a vehicle with a human-related emergency data acquisition system that is connected to a device for performing computer-assisted analysis of human-related emergency data,

Fig. 2 is a schematic view of an apparatus for performing computer-assisted analysis of human-related emergency data, in an illustrative example of the present invention, and

Fig. 3 schematically shows a method for analyzing human-related emergency data, in accordance with the present invention.

Fig. 1 shows a vehicle 1 with a vehicle user 3 and a schematically shown integrated in the vehicle 3 system 2 for capturing human-related emergency data connected, via a data link, to a device 20 according to the invention for carrying out a computer-implemented analysis of human-related emergency data. The vehicle 1 shown in the example of FIG. 1 is a motor vehicle, but it may, for example, be designed as a general purpose vehicle.

The vehicle-integrated system 2 for capturing human-related emergency data includes at least one camera configured as a high-speed camera 4 for capturing image data, a data memory, not shown in more detail, configured as a ring buffer, and a transmission unit, not shown in more detail, for transmitting the data stored in the ring buffer to the device 20. Optionally, the system 2 also includes an operational data logging function (not shown) for recording the physical operational data of the vehicle 1.

The high speed camera 4 of the system 2 is configured and positioned in the vehicle 1 in such a way that it is suitable for capturing the area of the interior of the vehicle in which the user 3 of the vehicle is located, i.e. to record a video sequence in the form of images of the user of the vehicle 3. In this case, the high-speed camera 4 should not capture the entire body of the user of the vehicle 3, but it is enough to capture the upper body, in particular the head 5, the neck and the upper part of the body and/or the shoulder area. Preferably, the camera also captures the distal regions of the body.

The high speed camera 4 exhibits a high recording speed, such as at least 700 fps, preferably at least 1000 fps. The images of the footage are recorded by the high-speed camera 4 as digital image data.

The high speed camera 4 is connected via a data connection, such as a data cable, to the ring buffer. The ring buffer is suitable for storing image data 20 digitally recorded by the high-speed camera 4. Alternatively, the ring buffer may also be the internal memory of the high speed camera 4.

A ring buffer is a digital memory with a predetermined storage size or capacity in which data, in this case including image data captured by a high-speed camera, can be permanently stored. If the maximum size of the ring buffer is reached and the ring buffer is full, then the corresponding oldest stored element is overwritten so that the data is stored in a so-called loop. Thus, the graphical representation of memory is the shape of a ring. The ring buffer can, for example, be implemented by means of suitable software, with which the storage and reading of data in the digital memory is controlled accordingly. Preferably, the size of the ring buffer or its capacity is sufficient to store images recorded by the high speed camera over a time interval of a few seconds, such as 10 s or 20 s, and any physical vehicle operating data that will be stored before it is overwritten again.

The optional performance data logging unit is preferably also connected via a data connection, eg data cable, to the ring buffer to store the physical performance data of the vehicle 1 recorded by the data logging functional unit in the ring buffer. For example, the data logging functional block may be configured as a speed sensor and/or a position sensor, such as a GPS and/or an accelerometer to record the instantaneous speed or position or acceleration of the vehicle 1 as operational data.

In addition, the ring buffer is connected via a data connection, for example, a data cable, to the transmission unit of the system 2. The transmission unit is suitable for transmitting image data stored in the ring buffer and additionally operating data in the form of emergency data 10 using the method wireless transmission to device 20, preferably in encrypted form. To this end, the transmission unit has a data interface, not shown.

The system 2 may include other components not described in more detail, such as, for example, a control unit for controlling individual components of the system 2, a sensor for detecting an emergency, a lighting device for illuminating the interior of the vehicle, an analysis unit for analyzing data and/ or other storage device.

When the system 2 is operating, the high speed camera 4 continuously records the vehicle user 3 while the vehicle 1 is moving, i.e. it records the image data of the vehicle user 3, and transfers it to the ring buffer. If necessary, at the same time, the operational data recording function, preferably continuously, records the operational data of the vehicle 1 and also transmits to the ring buffer. Until an emergency occurs, the data stored in the ring buffer is respectively overwritten with new data again depending on the determined recording time, which depends, inter alia, on the recording capacity of the ring buffer.

In an emergency, this continuous data storage is interrupted so that the data present in the ring buffer is no longer overwritten. Thus, after the end of data recording, the ring buffer contains image data and additionally physical operational data for the period of time in which the emergency occurred. Then, they are transmitted by the transmission unit as human-related emergency data 10 to the human-related emergency data analysis device 20 . Alternatively, human-related emergency data may also be transferred to another external location, such as a rescue rescue control center and/or a medical facility and/or a non-vehicle data store, such as an external server or cloud. and provided by this external location for use in the device 20. Thus, the transmission of human-related emergency data 10 occurs over a wireless network, such as radio, especially mobile communications or over the Internet, and may, for example, occur in conjunction with an emergency call. electronically (known as eCall). Preferably, the human-related emergency data 10 is transmitted in encrypted form, for example, using OpenPGP, wherein the human-related emergency data 10 is encrypted by the transmission unit using a public key generated by the device 20 or at an external location and again encrypted with an appropriate shared (private) device key 20 or even in an external location.

In addition, the device 20 for performing computer-implemented analysis of human-related emergency data may be implemented in a mobile device (not shown in the figures) that guides the first assistant and, for example, can be connected to the transmission unit by a data connection, for example, data cable. The mobile device together with the vehicle-integrated system 2 is an example of an emergency assistance system. It is used for timely recognition of possible injuries to vehicle users.

The device 20 shown schematically in FIG. 2, for performing computer-implemented analysis of human-related emergency data, includes at least one recognition unit 21, a comparison unit 22, and an output unit 23. The operation of the apparatus 20 for performing computer-implemented analysis of human-related emergency data is described next with reference to FIG. 3.

In the first step S1 (see FIG. 3) of the computer-implemented method according to the invention for analyzing human-related emergency data, the recognition unit 21 recognizes the pattern M(x) based on the human-related image data 10 . If necessary, the recognition unit for pattern recognition M(x) can use, in addition to the image data, the physical performance data of the vehicle 1.

The template M usually describes one or more sequences of movement of the user 3 of the vehicle in an emergency. For example, the pattern may include an acceleration or acceleration time sequence of at least one body part of the vehicle user 3 . Alternatively or additionally, the pattern may include a time sequence of relative and/or absolute deviation of at least one body part of the vehicle user 3 . Such a body part can be, for example, the head 5 and/or the neck and/or the chest and/or the torso and/or a limb, for example the arm of the user 3 of the vehicle. In particular, the relative movement and/or acceleration of the various body parts relative to each other can be considered, such as, for example, the relative displacement of the head 5 with respect to the shoulder or upper body region of the vehicle user 3 . Optionally, the template may also include physical operational data, such as the acceleration and/or speed of the vehicle just before and/or during the emergency.

The comparison unit 22 then compares, in a second step S2, the pattern M(x) recognized by the recognition unit with a number of previously stored patterns from M(1) to M(n). Previously saved templates may, for example, be stored in a database, the database being the internal memory of the device 20, or the comparator 22 accessing an external device database (also referred to as an emergency database). The result of the comparison may be, in this case, that the recognized pattern M(x) matches one of the previously stored patterns M(1) to M(n), or that the recognized pattern M(x) does not match any of the previously stored patterns.

For example, a database may consist of a series n of patterns from M(1) to M(n), where n is a natural number greater than or equal to 1 (n ≥ 1). Preferably, if the database consists of several templates from M(1) to M(n), i.e. has more than one template, so n > 1. As templates from M(1) to M(n), different templates are preferably meant.

It is preferable to store in the database additional medical diagnostic data from D(1) to D(p) (or even sets of medical diagnostic data), where p is also a natural number greater than or equal to 1 (p ≥ 1). Preferably, the number p corresponds to medical diagnostic data from D(1) to D(p) of the number n of templates previously stored in the database (i.e. p=n), where each of the n templates from M(1) to M (n) exactly matches one set of medical diagnostic data from D(1) to D(p).

The set of medical diagnostic data D(i) (1 ≤ i ≤ p) includes at least one medical diagnosis and/or part of information about medical long-term consequences. Medical diagnostic data, i.e. medical diagnosis and/or information on medical long-term consequences in this case refers to information obtained from a previous accident, in particular a specific other user of the vehicle, for which the template M(i) was recognized and matched with a medical diagnostic data set D(i).

At the same time, medical diagnostic data includes information about the type and/or severity of injuries to vehicle users. In this case, not only injuries immediately after the accident are established, but also the consequences that occur later, i.e. days, weeks or even months after the accident (long-term consequences).

Alternatively or additionally, the medical diagnostic data set D(i) may be associated with the severity of injuries. To do this, injuries can be classified, for example, as minor, moderate, serious, very serious, or life-threatening and the corresponding indicator value (for example, "minor" or "moderate" or "severe" or "very serious" or "life-threatening or on a scale of 1 (slight) to 5 (life-threatening)) can be stored in medical diagnostic data D(i).

If at step S2 the comparison of the recognized template M(x) with the previously stored templates from M(1) to M(n) leads to the detection of a match between the template M(x) and one of the previously stored templates, i.e. M(x) = M(j) with 1 ≤ j ≤ n, the output unit 23 (see FIG. 2) outputs in step S3 (see FIG. 3) diagnostic data D(j) corresponding to this pattern and/or the template M(j) itself.

To store the previously stored template and the corresponding medical diagnostic data in the database, for example, one can proceed as follows: respectively, pairs of human-related emergency data and medical diagnostic data associated with them are provided. For example, human-related emergency data and medical diagnostic data may be determined from previous emergencies. Using a suitable algorithm, an appropriate pattern is recognized from the image data, possibly in association with physical vehicle performance data and/or medical diagnostic data. Preferably, the algorithm is based on a machine learning method, i. e. it is suitable for recognizing relevant patterns from data on its own and only using provided data.

Thus, each recognized pattern is stored as a previously stored pattern with the corresponding medical diagnostic data in the database or other data memory.

For the case that in step S2 comparing the recognized pattern M(x) with the previously stored patterns M(1) to M(n) it is found that the recognized pattern M(x) does not match any of the previously stored patterns, the output block is preferably executed with the possibility of displaying the relevant part of the information, for example, in the form of text and/or a visual or sound signal. Alternatively or additionally, the output unit may also output the recognized pattern M(x) itself.

Optionally, a recognized pattern M(x) that does not match any of the previously stored patterns M(1) - M(n) is then also given, i.e. stored in a database and, if appropriate diagnostic data is available, which, for example, includes initial and/or subsequent diagnosis and/or information on long-term consequences, they can subsequently be assigned to the newly stored template M(x) and stored in the database in association with him.

Claims (22)

1. A computer-implemented method for analyzing human-related data (10) about an emergency situation of one or more users (3) of a vehicle, while human-related data (10) about the emergency includes at least image data of the video sequence of the user (3) of the vehicle at the time of the emergency, wherein the method contains one first step (S1) of pattern recognition (M(x)) based on image data, and one second step (S2) comparing the recognized pattern (M(x)) with a number of previously stored patterns (M(1)...M(n)), wherein the recognized pattern includes the sequence of movement of the user (3) of the vehicle in an emergency, characterized in that the recognized and/or previously stored pattern includes the temporal sequence of acceleration processes of the head (5) and/or neck and/or chest and/or torso and/or limbs of the vehicle user (3) and/or the temporal sequence deviations of the head (5) and/or neck and/or torso and/or chest and/or limbs of the user (3) of the vehicle, wherein the previously stored templates (M(1)...M(n)) are respectively correlated with the medical diagnostic data (D(1)...D(p)), the method further comprising a third step (S3) of outputting the medical diagnostic data (D( j)), which are correlated with the recognized pattern (M(j)). 2. The method of claim. 1, in which the medical diagnostic data (D(1)...D(p)) includes at least one medical diagnosis and/or information about medical long-term consequences. 3. The method according to claim. 1 or 2, in which the medical diagnostic data (D(1)...D(p)) contains an indicator that determines the degree of possible severity of injury to the user (3) of the vehicle. 4. The method according to any one of paragraphs. 1-3, wherein the human-related emergency data (10) further includes the physical operational data of the vehicle just before and/or during the emergency. 5. The method according to any one of paragraphs. 1-4, which is carried out using an algorithm for implementing machine learning. 6. The method according to claim 5, wherein the algorithm is configured to recognize a number of patterns from the image data previously recorded as previously stored patterns (M(1) …M(n)). 7. The method according to any one of paragraphs. 1-6, in which the previously stored patterns (M(1)...M(n)) are recorded in the emergency database. 8. A device for performing a computer-implemented analysis of human-related data (10) about an emergency of one or more users (3) of the vehicle, in which the human-related data (10) includes at least image data of the video sequence of the user (3) of the vehicle in case of an emergency, and the device (20) includes: a recognition unit (21) for pattern recognition (M(x)) based on image data, block (22) comparison for comparing the recognized template (M(x)) containing the sequence of movement of the user (3) of the vehicle in an emergency, with a number of previously stored templates (M(1) ... M(n)), different in that the recognized and/or previously stored pattern includes the temporal sequence of acceleration processes of the head (5) and/or neck and/or chest and/or torso and/or limbs of the vehicle user (3) and/or the temporal sequence deviations of the head (5) and/or neck and/or torso and/or chest and/or limbs of the user (3) of the vehicle, while the previously saved templates (M(1)...M(n)) are respectively correlated with medical diagnostic data (D(1)...D(p)), and the device contains an output unit (23) for outputting medical diagnostic data (D( j)), which are correlated with the recognized pattern (M(j)). 9. Emergency assistance system, including the device according to claim 8 and a high-speed camera for capturing image data of the video sequence of the user (3) of the vehicle during an emergency and a data storage device, preferably a ring buffer, for storing image data recorded by the high-speed camera . 10. The system of claim. 9, further comprising an operational data logging functional block for recording the physical operational data of the vehicle (1).

Images