JP7022674B2 - Collision injury prediction model creation method, collision injury prediction method, collision injury prediction system and advanced accident automatic notification system - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act On September 12, 2018, through IRCOBI's "http://www.ircobi.org/wordpress/downloads/irc18/pdf-files/46.pdf address website" Announcement Presented as a poster at "2018 International IRCOBI Conference Athes Grece" sponsored by IRCOBI from September 12, 2018 to September 14, 2018 "2018 International IRCOBI Conference Athen" sponsored by IRCOBI on September 13, 2018. Presented at the presentation at "Greece"

The present invention relates to a collision injury prediction model creation method for predicting a collision injury level of an injured person at the time of a car accident, a collision injury prediction method, a collision injury prediction system, and an advanced accident automatic notification system.

Conventionally, a collision injury prediction system is known as a system using image data (see, for example, Patent Document 1). The conventional system is an image sensor that detects the position of a person's head and the relative speed of the head in the vehicle left-right direction and the vehicle vertical direction, and a distance sensor that detects the relative speed and the relative distance between the person and the vehicle in the vehicle front-rear direction. The primary collision prediction unit that determines whether or not there is a high possibility of a primary collision (collision between a person and a vehicle) based on the detection results of the image sensor and the distance sensor, and the primary collision prediction unit perform a primary collision. Based on the detection results of the image sensor and the distance sensor, the primary collision determination unit that determines the position of the human head at the time of the primary collision and the relative speed with respect to the vehicle, and the primary collision determination unit A secondary collision estimation unit that estimates the situation of a secondary collision (collision between a person and a road surface after a vehicle collision) based on the judgment result of the collision determination unit, and a head injury based on the estimation results of the secondary collision estimation unit. It is equipped with an injury level prediction unit that predicts the level.

Japanese Unexamined Patent Publication No. 2014-141127

In the conventional system, the primary collision is determined based on the position of the human head and the relative speed with respect to the vehicle, the secondary collision situation is estimated based on the determination result, and the head is estimated based on the estimation result of the secondary collision. Predicts the level of injury in the area. In this way, the input image information is simply treated as a behavior function of the human head, and the collision injury level is predicted by a method of determining and estimating a primary collision or a secondary collision. However, since only the head position and the relative speed of the vehicle are targeted, it is necessary to consider the difference in the collision position between the human head and the vehicle and the influence of the human defense posture (head protection by the arm, etc.) on the injury. Can not. For this reason, especially for pedestrians and cyclists whose behavior during a collision is complicated, the collision injury level is often over-predicted or under-predicted, and the prediction accuracy of the collision injury level is low, resulting in injury. There was a problem that the part was limited to only the head.

The present invention has been made by paying attention to the above problems, and an object of the present invention is to improve the accuracy of predicting the collision injury level of an injured person in the event of a car collision.

In order to achieve the above object, the present invention is a method for creating a collision injury prediction model in which a collision injury prediction model used for predicting a collision injury level of an injured person in the event of a car accident is created by a deep learning program of a computer. , Has the following computer-based procedures.
Create a database of collision accident images and injury information of injured persons observed at the time of a car accident that occurred in the past.
A large number of collision accident image data in the database are converted into time-series collision accident image data acquired at the timing when the difference in the behavior of the injured person appears from the time of the accident , and the collision accident image data and the injury information data are associated with each other. Create a training data set.
The deep learning program has an artificial intelligence model that uses a multi-layered structure algorithm modeled on a human brain neural circuit selected from many artificial intelligence models based on image recognition accuracy as a deep learning model, and the selected artificial intelligence. Deep learning is performed on the training data set using the model .
As a collision injury prediction model, a trained artificial intelligence model with deep learning is created.

In this way, by performing deep learning on the learning data set of the collision accident image and the injury information, an artificial intelligence model used for predicting the collision injury level is created. As a result, it is possible to improve the accuracy of predicting the collision injury level of the injured person at the time of a car collision accident.

FIG. 3 is an overall system diagram showing an advanced accident automatic notification system to which the collision injury prediction model creation method, the collision injury prediction method, and the collision injury prediction system of the first embodiment are applied. It is a system block diagram which shows the detailed structure of the advanced accident automatic report system of Example 1. FIG. It is a schematic diagram which shows the advanced accident automatic report system of a conventional example. It is an operation flow diagram which shows the procedure of the creation operation of the artificial intelligence model of Example 1. FIG. It is a block diagram which shows the deep learning model (VGG16 model) which has in the deep learning program of Example 1. FIG. It is an error rate comparison diagram which shows the error rate comparison of the class classification by the recognition level of the Alexnet model (AlexNet) which is a deep learning model, the VGG16 model (VGG16), and the person (Human). It is a flowchart which shows the prediction procedure of the degree of injury occurrence using the collision injury prediction system of Example 1. FIG. It is a figure which shows the characteristic example when it was judged as HIC1000 or less at the time of applying Grad-CAM in the prediction accuracy verification of the occurrence degree of head injury of a pedestrian by a collision accident simulation analysis. It is a figure which shows the characteristic example when it was judged that the HIC1000 was exceeded when Grad-CAM was applied in the prediction accuracy verification of the occurrence degree of a pedestrian's head injury by a collision accident simulation analysis. It is a figure which shows the time series image from the side of a vehicle about the behavior of a pedestrian model at the time of a vehicle collision in the prediction accuracy verification of the occurrence degree of a pedestrian's head injury by a collision accident simulation analysis. It is a characteristic diagram which shows the relationship of the head injury prediction accuracy of a pedestrian with respect to the elapsed time from the time of a collision of a pedestrian model in the prediction accuracy verification of the occurrence degree of a pedestrian's head injury by a collision accident simulation analysis. It is a comparison result diagram which shows the pedestrian injury prediction accuracy comparison result by the conventional method and the present proposal method.

Hereinafter, embodiments for implementing a collision injury prediction model creation method, a collision injury prediction method, a collision injury prediction system, and an advanced accident automatic notification system according to the present invention will be described with reference to Example 1 shown in the drawings.

The first embodiment is an advanced accident automatic notification system to which all of the collision injury prediction model creation method, the collision injury prediction method, and the collision injury prediction system of the present invention are applied. Hereinafter, the configuration of the first embodiment will be described separately for "overall configuration of the advanced accident automatic notification system" and "detailed configuration of the advanced accident automatic notification system".

[Overall system configuration]
FIG. 1 shows an advanced accident automatic notification system to which the collision injury prediction model creation method, the collision injury prediction method, and the collision injury prediction system of the first embodiment are applied. Hereinafter, the overall configuration of the advanced accident automatic notification system will be described with reference to FIG. Here, the advanced accident automatic notification system is called "AACN (abbreviation of Advanced Automatic Collision Notification)", and hereinafter, it is referred to as "advanced accident automatic notification system AACN".

Advanced Automatic Collision Notification System AACN is a system that requests the dispatch of emergency medical transportation to the accident site according to the collision injury level of the injured person when a car accident occurs. Here, the "injured person" means a person who can be injured by the occurrence of a collision accident such as a car occupant, a bicycle occupant, a pedestrian, or the like. "Emergency medical transportation means" means an ambulance that transports an injured person while providing first aid, a doctor car for emergency medical treatment with a doctor, a doctor helicopter, or the like, which is dispatched for critical care.

As shown in FIG. 1, the advanced accident automatic notification system AACN includes an automobile A having an in-vehicle camera 1, an in-vehicle controller 2, and an in-vehicle communication device 3, an information management center B having an external server 4 and an external communication device 5, and an emergency. It is equipped with an emergency medical information center C having a system controller 6 and an emergency communication device 7.

Here, the "vehicle A" refers to a vehicle that is incorporated in the advanced accident automatic notification system AACN and has a function of automatically transmitting accident information acquired by the own vehicle when a vehicle accident occurs. "Information management center B" is, for example, an infrastructure that manages traffic information in a probe information system that receives a huge amount of vehicle data transmitted from a group of probe vehicles and records the received vehicle data by classifying them into information usage types. A facility, etc. "Emergency medical information center C" means, for example, an infrastructure facility that manages medical information handled in a network when constructing a medical network in a predetermined area.

The vehicle-mounted controller 2 transmits the image data of the collision accident image captured by the vehicle-mounted camera 1 when a vehicle accident occurs to the outside via the vehicle-mounted communication device 3. Here, the "vehicle-mounted camera 1" is mounted on the vehicle A, such as a camera for a drive recorder and a front camera used for driving support control, and is a collision accident image of the vehicle front or the vehicle interior image when a vehicle accident occurs. An imaging means capable of acquiring data (including moving image data and still image data).

The external server 4 uses the image data received from the vehicle A and the artificial intelligence model created by performing deep learning to predict the degree of collision injury of the injured person and the degree of occurrence of collision injury. Obtain the judgment level of collision injury based on the prediction. Then, the determined collision injury determination level is transmitted to the emergency medical information center C via the external communication device 5. Here, the details of the "artificial intelligence model created by the implementation of deep learning" will be described later. The external server 4 is connected to a display device 8 that visually displays the determination level result after making a determination to divide the degree of collision injury into a plurality of stages.

Upon receiving the collision injury determination level information from the information management center B, the emergency system controller 6 immediately displays / voices the "collision injury determination level", "vehicle accident occurrence location", etc. by the accident information notification device 9. do. Upon receiving this information, the operator in charge of the Emergency Medical Information Center C takes appropriate emergency / lifesaving measures according to the "collision injury determination level", "vehicle accident occurrence location", and the like. For example, if the collision injury determination level is low, a procedure for requesting the dispatch of an ambulance capable of emergency processing according to the level is performed. In addition, if the collision injury judgment level is high and it is possible to access the place where the car accident occurred in a short time by car, a procedure for requesting the dispatch of a doctor car will be carried out. Furthermore, if the collision injury determination level is high and the location where the car accident occurred cannot be accessed by car, or if it takes a long time to access by car, a procedure for requesting the dispatch of a doctor helicopter is performed.

[Detailed configuration of advanced accident automatic notification system]
FIG. 2 shows the detailed configuration of the advanced accident automatic notification system AACN of the first embodiment. Hereinafter, the detailed configuration of the advanced accident automatic notification system AACN will be described with reference to FIG.

As shown in FIG. 2, the advanced accident automatic notification system AACN includes an in-vehicle camera 1, an in-vehicle controller 2, an in-vehicle communication device 3, an external server 4, an external communication device 5, an emergency system controller 6, and emergency communication. It includes a device 7, a display device 8, and an accident information notification device 9. Here, the collision injury prediction system is a system excluding the emergency medical information center C from the advanced accident automatic notification system AACN. That is, a collision injury prediction system is configured by including an automobile A having an in-vehicle camera 1, an in-vehicle controller 2, and an in-vehicle communication device 3, and an information management center B having an external server 4 and an external communication device 5 (FIG. 1). reference).

As shown in FIG. 2, the in-vehicle controller 2 has an image acquisition unit 2a and an image data transmission unit 2b.

The image acquisition unit 2a acquires a collision accident image captured by the vehicle-mounted camera 1 when a vehicle accident occurs. Here, in the determination of the occurrence of a car accident, the occurrence of a car accident is determined by using the deployment of the airbag as a trigger. The determination of the occurrence of a car accident includes, for example, analysis of an image captured by the in-vehicle camera 1, sudden deceleration G generation exceeding the threshold value when the vehicle is stopped, generation of sudden deceleration G characteristics peculiar to the occurrence of an accident, and impact on the front bumper. Judgment may be made by one or a combination of detection and the like.

The image data transmission unit 2b transmits the acquired image data of the collision accident to the external information management center B via the in-vehicle communication device 3. Here, the image data to be transmitted to the information management center B is still image data or moving image data acquired from the time of the accident.

As shown in FIG. 2, the external server 4 has an input unit 4a, an injury occurrence degree prediction unit 4b, a display output unit 4c (output unit), and a collision injury determination result transmission unit 4d, and has a collision injury. A prediction program is configured.

The input unit 4a receives the image data transmitted from the vehicle-mounted communication device 3 via the external communication device 5. The input unit 4a performs image processing on the received image data. Here, the image processing is, for example, image resizing, gray scale change, brightness change, and the like.

The injury occurrence degree prediction unit 4b uses an artificial intelligence model created by performing deep learning as a collision injury prediction model, and predicts the occurrence degree of collision injury of the injured person from the image data after image processing.

The display output unit 4c outputs a display command for displaying the determination level of the collision injury obtained from the predicted degree of occurrence of the collision injury by the display device 8.

The collision injury determination result transmission unit 4d transmits the collision injury determination level obtained based on the prediction of the occurrence degree of the collision injury to the emergency medical information center C via the external communication device 5.

Here, on the external server 4, a deep learning program that executes a collision injury prediction model creation method for creating an artificial intelligence model (collision injury prediction model) used for predicting the collision injury level of the injured person when a car accident occurs. 40 is built-in.

As shown in FIG. 2, the deep learning program 40 has a database creation unit 4e, a learning data set creation unit 4f, a deep learning implementation unit 4g, and an artificial intelligence model creation unit 4h.

The database creation unit 4e creates a database based on the collision accident image and the injury information of the injured person observed at the time of the automobile accident that occurred in the past. Note that 4e1 is a collision accident image database and 4e2 is an injury information database.

The learning data set creation unit 4f converts the database from the database creation unit 4e into a format suitable for carrying out machine learning, and creates a large number of learning data sets in which collision accident image data and injury information data at the time of a car accident are associated with each other. create.

The deep learning implementation unit 4g selects an artificial intelligence model using a multi-layer structure algorithm modeled on a human brain neural circuit, and performs deep learning on the learning data set from the learning data set creation unit 4f. Here, as the artificial intelligence model, an artificial intelligence model having a convolutional neural network (CNN) structure, which is a kind of neural network, is selected.

"Deep Learning" is one of the machine learning methods that represent artificial intelligence AI (abbreviation of Artificial Intelligence), and is a multi-layered neural network (deep neural network) modeled on human brain neural circuits. ) Is used, and the artificial intelligence AI thinks and decides the setting and combination of the feature amount by itself. In other words, it is a method that is an extension of conventional machine learning, and it is necessary to specify features (features) that should be noted in machine learning that finds regularity and relationships from a large amount of data and makes judgments and predictions. On the other hand, in the case of deep learning, the difference is that learning is performed automatically without specifying features (features).

The artificial intelligence model creation unit 4h creates a trained artificial intelligence model in which deep learning is performed as a collision injury prediction model for predicting a collision injury.

The deep learning program 40 may be set in the information management center B on a dedicated server different from the external server 4, or may be set in an independent server in a facility different from the information management center B. When an artificial intelligence model is created by a dedicated server or an independent server, the created artificial intelligence model is input to or transmitted to the external server 4.

In addition, at the initial creation stage of the artificial intelligence model, an artificial intelligence model (initial model) is created using a database of collision accident images and injury information of the injured person observed at the time of a car accident that occurred in the past before the initial creation. do. Subsequent artificial intelligence models will carry out deep learning at any time when a predetermined amount of collision accident image and injury information database is accumulated, or when a predetermined amount of collision accident image and injury information database is accumulated. Update.

As shown in FIG. 2, the emergency system controller 6 appeals visually and audibly to the collision injury determination level received from the information management center B via the emergency communication device 7 by the accident information notification device 9. It has a determination level notification unit 6a.

Next, "Background technology and issues for collision injury prediction" and "Measures to solve the issues" will be explained. Then, the actions of Example 1 are "artificial intelligence model creation action", "collision injury prediction action", "prediction accuracy verification action of pedestrian head injury occurrence degree", and "automatic notification action by advanced accident automatic notification system". Will be explained separately.

[Background technology and issues for collision injury prediction]
The purpose is to transport car occupants and pedestrians (lifesaving / emergency patients) to medical institutions accurately and promptly by ambulance in the event of a car accident, and to dispatch doctor helicopters and doctor cars for emergency medical care to the accident site at an early stage. The advanced accident automatic notification system AACN is known.

As shown in Fig. 3, the Advanced Automatic Collision Notification System AACN uses EDR data (EDR is an abbreviation for "Event Data Recorder") that is automatically transmitted when a car accident occurs, and the degree of injury to car occupants and pedestrians (death). / Serious injury / minor injury / uninjured, etc.) is a method (hereinafter referred to as "conventional method").

However, in the conventional method, logistic regression analysis uses only the limited data recorded in the EDR data such as whether or not the automobile occupant is wearing a seatbelt, the collision speed, and the collision direction, and simply uses the input EDR data as an explanatory function. Collision injury prediction is carried out by.

Therefore, it cannot be said that the accuracy of collision injury prediction (correct answer rate) is high. In addition, while the car occupant at the time of the accident is in a sitting posture, the pedestrian may take various walking postures and avoidance actions, and the collision position with the car is not uniform. In comparison, pedestrian accident types are diverse. Therefore, for example, in predicting the degree of injury to a pedestrian, it is more difficult to predict than a car occupant, and there are cases where the prediction accuracy is 46.8% (see FIG. 12).

As a result, when the conventional method is used, (1) if the degree of injury is overdetermined, the number of unnecessary dispatches of the doctor helicopter or doctor car for emergency medical care is increased, or (2) the lifesaving that should be saved originally. If it is underestimated that the emergency patient does not need emergency treatment, the original purpose of lifesaving cannot be achieved.

That is, when the conventional method is used, a small number of doctor helicopters and doctor cars for emergency medical care are unnecessarily dispatched to the accident site due to an erroneous judgment of the degree of injury, and the appropriate lifesaving / emergency system is paralyzed. The problem is that there is a risk of this, and improvement is required.

[Measures to solve problems]
As a measure to solve the above problems, the present inventors use image data at the time of a collision accident and a trained artificial intelligence model that has been subjected to deep learning, which is a further development of conventional machine learning, for collision injury prediction. I focused on the point of use. According to this point of interest, we proposed a collision injury prediction model creation method, a collision injury prediction method, a collision injury prediction system, and an advanced accident automatic notification system AACN.

The collision injury prediction model creation method creates a database of collision accident images and injury information of the injured person observed at the time of a car accident that occurred in the past, converts the database into a format suitable for conducting machine learning, and collides accidents. Create a large number of training data sets that relate image data and injury information data. Then, using the selected artificial intelligence model, deep learning is performed on the training data set, and a trained artificial intelligence model on which deep learning is performed is created as a collision injury prediction model.

The collision injury prediction method, the collision injury prediction system, and the advanced accident automatic notification system all use "image data" at the time of a car accident and an "artificial intelligence model" created by the collision injury prediction model creation method to generate a car accident. Sometimes it is a method or system used to predict the level of collision injury of an injured person.

That is, in the "artificial intelligence model", artificial intelligence AI optimizes a large number of input learning data sets and performs deep learning to extract and predict the characteristics of the learning data set related to collision injury. Created with. Then, in the collision injury prediction, "image data" at the time of a car accident and a trained "artificial intelligence model" in which deep learning excellent in image recognition is performed are used. As a result, in the event of a car collision, the accuracy of predicting the collision injury level of the injured person can be improved as compared with the conventional method.

This greatly contributes to a significant reduction in the number of unnecessary dispatches of doctor helicopters and doctor cars for emergency medical care due to misjudgment of the collision injury level of the injured person, and to the reduction of missed patients who originally need lifesaving actions. Ultimately, however, it will lead to a reduction in the number of fatalities due to car accidents and the number of patients with sequelae.

[Artificial intelligence model creation action]
FIG. 4 is a flowchart showing the flow of the artificial intelligence model creation process executed by the deep learning program 40 of the first embodiment. Hereinafter, the artificial intelligence model creating action will be described with reference to FIGS. 4 to 6.

First, a database (collision accident image database, injury information database) based on collision accident images and injury information of injured persons observed at the time of a car accident that occurred in the past is created (S1). Next, the database is converted into a format suitable for performing machine learning, and a large number of learning data sets in which the collision accident image data and the injury information data are associated are created (S2). After that, image preprocessing (for example, changing the image glue size and gray scale, changing the brightness, etc.) is performed on the image data in the training data set (S3).

The following processes S4 to S6 are carried out in the deep learning program. First, the training data set subjected to image preprocessing is input to the deep learning program (S4). Next, deep learning is performed on the training data set using the selected artificial intelligence model (S5). Then, as a collision injury prediction model, a trained artificial intelligence model in which deep learning is performed is created, and the created artificial intelligence model is saved (S6).

In other words, in deep learning, by repeatedly machine learning using images of a large number of car occupants and pedestrians and injury information, the characteristics of the behavior of car occupants and pedestrians related to a specific injury are learned. be able to. Then, after deep learning, an "artificial intelligence model" that has been learned about the relationship between the image of a car occupant or a pedestrian at the time of a car accident and the injury is created.

Here, the deep learning program has, for example, the deep learning model (VGG16 model) shown in the block diagram of FIG. This deep learning model has a deeper layer of convolutional neural network for learning and higher image recognition accuracy than the Alexnet model, which is one of the artificial intelligence models with a CNN structure. Note that FIG. 6 shows a comparison of the error rates of the classifications shown in the image recognition contest ILSVRC (ImageNet Large Scale Visual Recognition Challenge). As is clear from the comparison of the error rates of the classifications in FIG. 6, the deep learning model (VGG16) shown in FIG. 5 has half the error rate as compared with the AlexNet model (AlexNet), and the human (Human). It can be seen that the recognition level of) is greatly approached.

[Collision injury prediction effect]
FIG. 7 is a flowchart showing a procedure for predicting the degree of injury occurrence using the collision injury prediction system of the first embodiment. Hereinafter, the collision injury prediction effect will be described with reference to FIG. 7.

First, when a car accident occurs (S11), images of car occupants, pedestrians, and the like are acquired when the car accident occurs (S12). Here, as an image acquisition method, for example, an image recorded on a drive recorder can be mentioned.

Next, image processing is performed on an image of a car occupant, a pedestrian, or the like at the time of a car accident (S13). Here, examples of the image processing include image resizing, gray scale, and luminance change. Next, the image-processed images of the automobile occupants, pedestrians, and the like at the time of the automobile accident are input to the collision injury prediction program of the external server 4 (S14). The following processes S15 to S17 are executed by the collision injury prediction program.

When the image is input to the collision injury prediction program, the collision injury prediction is performed using the artificial intelligence model created by the deep learning program 40 (S15). Here, the artificial intelligence model is a model created in advance using a deep learning method, and the injured person is injured from the image (behavior at the time of collision, etc.) at the time of a collision accident such as a car occupant or a pedestrian. Features related to degree can be extracted and predicted.

Next, the degree of injury occurrence (severity determination) of the injured subject is predicted from the prediction result by the artificial intelligence model (S16). Finally, the prediction result of the degree of injury of the injured person is displayed (S17). In the display of the prediction result, the severity determination obtained from the degree of injury occurrence determined by the prediction unit is displayed, and the basis for prediction is shown.

In this way, in the collision injury prediction action, instead of the conventional method of injury prediction using EDR data, the injury of the injured person is injured from the image (behavior at the time of collision, etc.) when a collision accident occurs such as a car occupant or a pedestrian. Extract and predict the characteristics related to the degree of occurrence. Therefore, it is possible to predict the degree of injury of the injured subject (severity determination) with higher accuracy than the conventional method.

[Prediction accuracy verification effect of the degree of head injury of pedestrians]
FIG. 8 shows an example of features when it is determined that the HIC is 1000 or less when Grad-CAM is applied. FIG. 9 shows an example of features when it is determined that the HIC1000 is exceeded when Grad-CAM is applied. FIG. 10 shows a pedestrian model behavior at the time of a vehicle collision as a time-series image from the side surface of the vehicle. FIG. 11 shows the relationship characteristic of the pedestrian's head injury prediction accuracy with respect to the elapsed time from the time of collision of the pedestrian model. FIG. 12 is a comparison result diagram of pedestrian injury prediction accuracy between the conventional method and the proposed method. Hereinafter, the predictive accuracy verification action of the degree of head injury of a pedestrian will be described with reference to FIGS. 8 to 12.

Actually, using the collision injury prediction program of Example 1, the degree of head injury occurrence of an adult male at the time of a primary collision with a simple vehicle simulating a commercial vehicle by collision accident simulation analysis is learned, and limited conditions are applied. Although it was inside, the prediction accuracy of the degree of head injury of pedestrians was verified.

The prediction accuracy verification of the occurrence of head injury of pedestrians was carried out using a data set (verification data set) different from the learning data set used at the time of deep learning. First, the image data of the verification data set is input to the trained artificial intelligence model to predict the degree of head injury. By comparing the prediction result with the injury information in the verification data, the correct answer rate of the prediction accuracy was verified. In addition, in the prediction validity study of the degree of head injury of pedestrians, Grad-CAM (Gradient-weighted Class Activation Mapping) was applied to the trained artificial intelligence model created by deep learning, and the trained artificial intelligence model was applied. Identified the featured part that was focused on during image recognition. Here, "Grad-CAM" has a function to identify and draw an image part that has a large influence on the classification, and the feature part that the deep-learned artificial intelligence model is paying attention to at the time of image recognition. This is a method of expressing it as a heat map. The level of injury applied to the head of the pedestrian model was determined using the value of the head injury standard HIC (Head Injury Criterion) at the time of a collision with a vehicle.

FIG. 8 shows an example of features when it is determined that the HIC is 1000 or less when Grad-CAM is applied. In determining a pedestrian model collision image with a HIC of 1000 or less, the trained artificial intelligence model focuses on the torso of the pedestrian model, and if the torso can be confirmed on the image 90 msec after the collision, the HIC will be 1000 or less. It seems to be predicting.

FIG. 9 shows an example of the characteristics when it is determined that the HIC1000 is exceeded when Grad-CAM is applied. It can be seen that the trained artificial intelligence model focuses on the boundary between the pedestrian model and the vehicle part and the foot of the pedestrian model in the determination of exceeding HIC1000. Therefore, it is considered that the trained artificial intelligence model recognizes the characteristics of the pedestrian model's flip-up by the vehicle at 90 msec after the collision from the image.

These features have been identified in most cases of other Grad-CAM-applied validation data, and these features show that the trained artificial intelligence model predicts the injury level of the pedestrian model. rice field.

FIG. 10 shows the behavior of the pedestrian model at the time of a vehicle collision as a time-series image from the side surface of the vehicle. The collision phenomenon between the pedestrian model and the vehicle model begins with the collision between the leg and the bumper, and the leg bends generally along the shape of the bumper. Then, 50 msec after the collision, the legs are separated from the bumper, and the waist and the hood mainly come into contact with each other. At the same time, in this case, the contact between the elbow on the vehicle side and the hood begins. After that, in the pedestrian model, the movement of the upper body is suppressed by the cushioning of the elbow and the hood, and the head and the vehicle collide at 98 msec.

FIG. 10 is similar to the characteristics of HIC1000 or higher, but the analysis result was HIC1000 or lower. It is considered that the reason for this is that the elbow on the vehicle-to-vehicle side of the pedestrian model suppressed the movement of the upper body due to the collision with the vehicle, so that the impact on the head was alleviated. Even when it is difficult for a person to judge an injury, the artificial intelligence model enables accurate prediction, and the effectiveness of the injury level prediction method using the artificial intelligence model can be confirmed. In addition, from the results of Grad-CAM, it is considered that the trained artificial intelligence model focused on the elbow of the pedestrian model at the time of collision and appropriately predicted the HIC value, which was consistent with the human findings.

When AlexNet, which is one of the artificial intelligence models, is used, the timing at which the injury level can be effectively predicted from the collision behavior of the pedestrian model is as shown in FIG. 11, the elapsed time from the collision is 50 msec to 60 msec. When it becomes, it rises to about 90% at a stretch. Then, it was found that a high prediction accuracy in the 90% range can be secured from 60 msec to 130 msec (the maximum value is 90.4% at 90 msec).

Furthermore, as a result of using VGG16, which has higher image recognition accuracy than AlexNet, as an artificial intelligence model, the correct answer rate of the prediction accuracy of the occurrence of head injury of pedestrians is the conventional method using EDR data, as shown in FIG. It increased from 46.8% in the above method to 99.37% in the proposed method. That is, it was confirmed that a program having extremely high prediction accuracy could be constructed by using the image data based on the elapsed time from the time of collision shown in FIG. 11 and the deep-learned artificial intelligence model.

In addition to the above verification targets, the collision injury prediction program adds information such as a wide variety of vehicle types and accident types, the height / weight of the subject, and the injury site to the learning data set, and deeply learns. It is suggested that it is possible to predict the degree of injury in any part of the human body under various accident conditions.

Therefore, in the future, by repeating deep learning using collision images of car occupants and pedestrians in the real world and injury information, the collision injury prediction program will be further evolved and the accuracy of injury prediction in the real world will be further improved. Be expected.

[Automatic notification action by advanced accident automatic notification system]
In the automatic notification by the advanced accident automatic notification system AACN, in the flowchart of FIG. 7, when the injury occurrence degree prediction (severity determination) of the injured person is performed in S16, the injury prediction information is transmitted to the emergency system controller 6. To. The emergency system controller 6 that has received the injury prediction information from the information management center B is immediately notified by the accident information notification device 9 of the "collision injury determination level", the "vehicle accident occurrence location", and the like by display / voice. Then, the operator in charge of the emergency medical information center C who received this information takes appropriate emergency and lifesaving measures according to the "collision injury judgment level" and "vehicle accident occurrence location" which are finely divided into levels. Will be done.

In this way, by applying the collision injury prediction method and the collision injury prediction system of the present invention to the advanced accident automatic notification system, unnecessary dispatch of a doctor helicopter or a doctor car for emergency medical care can be significantly reduced, and It is possible to reduce the number of missed patients who need critical care, and it is possible to provide appropriate lifesaving and emergency care. In addition, as a result, it leads to reduction of casualties due to automobile accidents and reduction of damage to residual injuries.

Regarding the spread of the advanced accident automatic notification system AACN, in Japan, an emergency automatic notification system (D-call Net) equipped with an algorithm for predicting the degree of injury has been installed, and full-scale operation has started in 2018. In Russia and Europe, it is obligatory to install an automatic accident notification system that reports only the location of a car accident to all new cars sold. In other countries such as Malaysia and South Korea, the installation of an automatic accident notification system is being considered.

Regarding drive recorders, the government has established a system for autonomous driving. In the examination of data storage during driving of autonomous vehicles in the network, the government said, "By 2020, a data recording device (event data recorder (EDR)). , We will consider making it mandatory to install drive recorders, etc. ”, and further expansion is expected in the future from the current drive recorder penetration rate of about 15%.

Therefore, it is expected that the number of automobiles with the function of the advanced accident automatic notification system AACN will increase year by year. On the other hand, since the collision injury prediction model creation method, the collision injury prediction method, and the collision injury prediction system of the present invention are the core technologies for predicting the degree of injury, they are adopted as the injury prediction algorithm of the advanced accident automatic notification system AACN. Be expected.

As described above, the collision injury prediction model creation method, the collision injury prediction method, the collision injury prediction system, and the advanced accident automatic notification system AACN of the first embodiment have the effects listed below.

(1) A collision injury prediction model creation method for creating a collision injury prediction model used to predict the collision injury level of an injured person in the event of a car accident.
Created a database of collision accident images and injury information of injured persons observed at the time of a car accident that occurred in the past.
Convert the database to a format suitable for performing machine learning, create a large number of training data sets that relate collision accident image data and injury information data,
Select an artificial intelligence model that uses a multi-layer algorithm modeled on the human brain neural circuit, perform deep learning on the training data set, and perform deep learning.
As a collision injury prediction model, a trained artificial intelligence model with deep learning is created.
Therefore, it is possible to create an artificial intelligence model that improves the accuracy of predicting the collision injury level of the injured person in the event of a car collision.

(2) As an artificial intelligence model, an artificial intelligence model having a convolutional neural network structure, which is a kind of neural network, is selected.
Therefore, since the neural network layer is deep and the image recognition accuracy is high, it is possible to create an artificial intelligence model that improves the prediction accuracy of the collision injury level of the injured person in the event of a car collision accident.

(3) A collision injury prediction method executed by an external server 4 (or an advanced in-vehicle device) by communicating with an in-vehicle camera A equipped with an in-vehicle camera 1.
When a car accident occurs, the image data of the collision accident image captured by the in-vehicle camera 1 at the time of the accident is acquired.
When image data is input, the collision injury level of the injured person is predicted by using the artificial intelligence model created by performing deep learning as the collision injury prediction model.
The judgment level of the collision injury obtained from the predicted collision injury level is output.
Therefore, it is possible to provide a collision injury prediction method that improves the prediction accuracy of the collision injury level of the injured person in the event of a vehicle collision accident.

(4) Of the image data of the collision accident image captured by the in-vehicle camera 1, the image data used for predicting the collision injury level of the injured person is the timing (for example, 60 msec) when the difference in the behavior of the injured person appears from the time of the accident. It shall be still image data or moving image data acquired at the above timing).
Therefore, it is possible to achieve high prediction accuracy of the collision injury level of the injured person by the prediction based on the image data of the collision accident image acquired from the vehicle-mounted camera 1 when the accident occurs.

(5) A collision injury prediction system including an automobile A having an in-vehicle camera 1, an in-vehicle controller 2, and an in-vehicle communication device 3, and an information management center B having an external server 4 and an external communication device 5.
The in-vehicle controller 2 has an image acquisition unit 2a that acquires a collision accident image captured by the in-vehicle camera 1 when an automobile accident occurs, and an image data transmission unit 2b that transmits the acquired image data of the collision accident to the outside. ,
The external server 4 uses the input unit 4a that receives image data via the external communication device 5 and performs image processing, and the artificial intelligence model created by performing deep learning as a collision injury prediction model, and receives the received image data. It has an injury occurrence degree prediction unit 4b that predicts the occurrence degree of collision injury of the injured person, and an output unit (display output unit 4c) that outputs a collision injury determination level obtained from the predicted collision injury level.
Therefore, it is possible to provide a collision injury prediction system that improves the prediction accuracy of the collision injury level of the injured person in the event of a vehicle collision accident.

(6) Vehicle A having an in-vehicle camera 1, in-vehicle controller 2, and in-vehicle communication device 3, information management center B having an external server 4 and an external communication device 5, and emergency medical care having an emergency system controller 6 and an emergency communication device 7. In the advanced accident automatic notification system AACN, which is equipped with Information Center C and requests the dispatch of emergency medical transportation to the accident site according to the collision injury level of the injured person.
The vehicle-mounted controller 2 has an image data transmission unit 2c that transmits image data of a collision accident image captured by the vehicle-mounted camera 1 to the outside when a vehicle accident occurs.
The external server 4 uses the image data received from the vehicle A and the artificial intelligence model created by performing deep learning, and the injury occurrence degree prediction unit 4b for predicting the occurrence degree of the collision injury of the injured person and the collision injury. The collision injury determination result transmission unit 4d, which transmits the collision injury determination level obtained based on the occurrence degree prediction of the above to the emergency medical information center C, is provided.
The emergency system controller 6 has a collision injury determination level notification unit 6a for notifying the collision injury determination level received from the information management center B.
For this reason, we have achieved the accurate and prompt transportation of injured persons (lifesaving / emergency patients) by ambulance in the event of a car accident, and the early dispatch of doctor helicopters and doctor cars for emergency medical care to the accident site. can do. As a result, unnecessary dispatch of doctor helicopters and doctor cars for emergency medical care can be significantly reduced, and the number of missed patients in need of critical care can be reduced, enabling appropriate lifesaving / emergency care.

The collision injury prediction model creation method, the collision injury prediction method, the collision injury prediction system, and the advanced accident automatic notification system of the present invention have been described above based on the first embodiment. However, the specific configuration is not limited to the first embodiment, and design changes and additions are permitted as long as the gist of the invention according to each claim is not deviated from the claims.

In Example 1, as an example of selecting an artificial intelligence model, an example of selecting an artificial intelligence model having a convolutional neural network structure, which is a kind of neural network, is shown. However, as the artificial intelligence model, of course, an example of selecting an artificial intelligence model having a neural network structure that is not convolution may be used. In this case as well, it is possible to improve the collision injury prediction accuracy and reduce the collision injury prediction error as compared with the conventional method. In addition, moving images can be used as learning data used for deep learning.

In the first embodiment, among the image data of the collision accident image captured by the vehicle-mounted camera 1, the image data used for predicting the collision injury level of the injured person is still image data acquired at the timing when 60 msec or more has elapsed from the time of the accident. Alternatively, an example of moving image data is shown. However, if the image data of the collision accident image captured by the in-vehicle camera is still image data or video data acquired at the timing when the difference in the behavior of the injured person appears from the time of the accident, it is from the time of the accident. The timing may be shorter or longer than 60 msec. For example, multiple still image data at predetermined time intervals from the time of the accident, or moving image data from the time of the accident to the specified time, and after the external server inputs the image data, the behavior of the featured part changes. Image data or moving image data may be extracted, and the extracted image data may be input to the damage occurrence degree prediction unit.

In the first embodiment, as a collision injury prediction method, an example is shown in which the external server 4 is executed by communication with the vehicle A equipped with the vehicle-mounted camera 1. However, if the advanced in-vehicle device can perform the same function as the external server, the advanced in-vehicle device may be used instead of the external server. In this case, it is effective to predict the collision injury of the occupant in the vehicle interior based on the image data from the vehicle-mounted camera arranged toward the vehicle interior.

In Example 1, an example in which the collision injury prediction model creation method, the collision injury prediction method, and the collision injury prediction system of the present invention are applied to the advanced accident automatic notification system AACN is shown. However, even when only the method for creating a collision injury prediction model of the present invention is carried out, it is included in the present invention. Further, even when each of the collision injury prediction method and the collision injury prediction system is implemented, it is included in the present invention. Further, even when each of the collision injury prediction method and the collision injury prediction system is applied to other than the advanced accident automatic notification system, it is included in the present invention.

AACN Advanced Automatic Collision Notification System A Automobile 1 In-vehicle camera 2 In-vehicle controller 2a Image acquisition unit 2b Image data transmission unit 3 In-vehicle communication device B Information management center 4 External server 4a Input unit 4b Injury occurrence degree prediction unit 4c Display output unit (output unit) )
4d Collision injury judgment result transmission unit 40 Deep learning program 4e Database creation unit 4f Learning data set creation unit 4g Deep learning implementation unit 4h Artificial intelligence model creation unit 5 External communication device C Emergency medical information center 6 Emergency system controller 6a Collision injury judgment level Notification unit 7 Emergency communication device

Claims (5)

It is a collision injury prediction model creation method that creates a collision injury prediction model used to predict the collision injury level of an injured person in the event of a car accident by a computer deep learning program .
The computer
Created a database of collision accident images and injury information of injured persons observed at the time of a car accident that occurred in the past.
The collision accident image in the database is converted into time-series collision accident image data acquired at the timing when a difference in the behavior of the injured person appears from the time of the accident , and the collision accident image data and the injury information data are associated with each other. Created a large number of training datasets
The deep learning program has, as a deep learning model, an artificial intelligence model using a multi-layered structure algorithm modeled on a human brain neural circuit selected from many artificial intelligence models based on image recognition accuracy, and was selected. Using an artificial intelligence model, deep learning was performed on the training data set, and
A method for creating a collision injury prediction model, which comprises creating a trained artificial intelligence model in which deep learning is performed as the collision injury prediction model. In the collision injury prediction model creation method according to claim 1,
A method for creating a collision injury prediction model, which comprises selecting an artificial intelligence model having a convolutional neural network structure, which is a kind of neural network, as the artificial intelligence model. It is a collision injury prediction method executed by a collision injury prediction program of an external server or a computer of an advanced in-vehicle device by communicating with a car equipped with an in-vehicle camera.
The computer
When a car accident occurs, the image data of the collision accident image captured by the in-vehicle camera at the time of the accident is acquired.
The image data of the collision accident image is taken as still image data or moving image data acquired at the timing when the difference in the behavior of the injured person appears from the time of the accident.
When the still image data or the moving image data is input to the collision injury prediction program, the injured person can use the trained artificial intelligence model created by the method according to claim 1 as the collision injury prediction model. Predict collision injury levels and
A collision injury prediction method characterized by outputting a collision injury judgment level obtained from a predicted collision injury level. A collision injury prediction system including an in-vehicle camera, an in-vehicle controller, an automobile having an in-vehicle communication device, and an information management center having an external server and an external communication device.
The in-vehicle controller is
It has an image acquisition unit that acquires a collision accident image captured by the vehicle-mounted camera when a car accident occurs, and an image data transmission unit that transmits the acquired image data of the collision accident image to the outside.
The external server is
Received using the input unit that receives the image data via the external communication device and performs image processing, and the trained artificial intelligence model created by the method described in claim 1 as a collision injury prediction model. It is characterized by having an injury occurrence degree prediction unit that predicts the occurrence degree of collision injury of the injured person from the image data, and an output unit that outputs a collision injury determination level obtained from the predicted collision injury level. Collision injury prediction system. It is equipped with an in-vehicle camera, an in-vehicle controller, an in-vehicle communication device, an information management center with an external server and an external communication device, and an emergency medical information center with an emergency system controller and an emergency communication device. In the advanced accident automatic notification system that requests the dispatch of emergency medical transportation to the accident site according to the injury level,
The in-vehicle controller is
It has an image data transmission unit that transmits the image data of the collision accident image captured by the in-vehicle camera to the outside when a car accident occurs.
The external server is
An injury occurrence degree prediction unit that predicts the occurrence degree of collision injury of an injured person by using the image data received from the automobile and the trained artificial intelligence model created by the method according to claim 1 . It has a collision injury determination result transmission unit that transmits the collision injury determination level obtained based on the prediction of the occurrence degree of collision injury to the emergency medical information center.
The emergency system controller is
An advanced accident automatic notification system characterized by having a collision injury determination level notification unit for notifying the collision injury determination level received from the information management center.

Images