CN109858553B - Method, device and storage medium for updating driving state monitoring model - Google Patents
Method, device and storage medium for updating driving state monitoring model Download PDFInfo
- Publication number
- CN109858553B CN109858553B CN201910103594.1A CN201910103594A CN109858553B CN 109858553 B CN109858553 B CN 109858553B CN 201910103594 A CN201910103594 A CN 201910103594A CN 109858553 B CN109858553 B CN 109858553B
- Authority
- CN
- China
- Prior art keywords
- monitoring model
- neural network
- convolutional neural
- network monitoring
- probability value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 254
- 238000000034 method Methods 0.000 title claims abstract description 76
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 174
- 239000013598 vector Substances 0.000 claims abstract description 51
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims description 13
- 238000007781 pre-processing Methods 0.000 claims description 11
- 230000006870 function Effects 0.000 claims description 7
- 230000000750 progressive effect Effects 0.000 claims description 4
- 230000008921 facial expression Effects 0.000 claims description 3
- 230000002159 abnormal effect Effects 0.000 abstract description 33
- 238000004891 communication Methods 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 6
- 230000006399 behavior Effects 0.000 description 5
- 241001282135 Poromitra oscitans Species 0.000 description 4
- 241001122767 Theaceae Species 0.000 description 4
- 206010048232 Yawning Diseases 0.000 description 4
- 230000001133 acceleration Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 4
- 230000001149 cognitive effect Effects 0.000 description 4
- 230000002996 emotional effect Effects 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 238000010606 normalization Methods 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 206010039203 Road traffic accident Diseases 0.000 description 2
- 238000013145 classification model Methods 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000005484 gravity Effects 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 208000032140 Sleepiness Diseases 0.000 description 1
- 206010041349 Somnolence Diseases 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000004397 blinking Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000035622 drinking Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000037321 sleepiness Effects 0.000 description 1
Abstract
The invention discloses a driving state monitoring model updating method, which comprises the following steps: reading new sample data of the driving state acquired by the driving state acquisition device and a first category of the driving state to which the new sample data belongs; acquiring key data in newly added sample data, and processing the key data to form a normalized feature vector; taking the normalized feature vector and the first category as input of an initial tree convolutional neural network monitoring model, and obtaining an output probability value of newly added sample data at a node of the initial tree convolutional neural network monitoring model; and adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model. The invention also discloses an updating device and a storage medium. The method and the device can dynamically update the abnormal driving state monitoring model through the incremental data, ensure the low redundancy of the model in the dynamic model updating process, and improve the recognition precision of the abnormal driving state and the robustness of the model.
Description
Technical Field
The present invention relates to the field of communications technologies, and in particular, to a method and an apparatus for updating a driving state monitoring model, and a storage medium.
Background
In recent years, traffic accidents caused by fatigue driving, distraction, emotional driving, sudden diseases and the like are increasing. In order to basically reduce the occurrence of traffic accident conditions, a driving state monitoring model is established, so that the driving behavior of a driver can be monitored, and an alarm is given to abnormal driving behavior, and the method has important significance in improving the driving capability of the driver, reducing the driving load of the driver and coordinating the relationship between the driver and the vehicle and the traffic environment.
However, the traditional driving state monitoring model only collects various sensor information to judge whether a driver is in a fatigue driving state, and identifies that the abnormal driving state is single, and factors influencing the driving state often have various aspects, such as excessive cognitive load, external interference, vehicle driving state and the like. The monitoring of the single abnormal driving state of the fatigue driving can not meet the requirements of high precision and strong robustness, and the monitoring model is used for training a classification model by using the existing calibration data and identifying and classifying the driving state by using the classification model. Such models, when modeling the driving state of the user, often model the existing driving state of the user, to reflect the current driving mode thereof. Over time, the driving state of the user is likely to change, and when a new abnormal driving behavior occurs to the user, the monitoring model cannot correctly identify and classify the new behavior. When a new abnormal driving state occurs, only a retraining mode can be adopted, in the processing mode, the required training time grows exponentially along with the increase of the number of samples, the model redundancy is high, and the requirement of incremental learning cannot be met.
Disclosure of Invention
The invention mainly aims to provide a driving state monitoring model updating method, a driving state monitoring model updating device and a storage medium, and aims to dynamically update an abnormal driving state monitoring model through incremental data, ensure low redundancy of the model in the dynamic model updating process and improve the abnormal driving state identification precision and the model robustness.
In order to achieve the above object, the present invention provides a method for updating a monitoring model of a driving state, the method comprising the steps of:
reading new sample data of the driving state acquired by the driving state acquisition device and a first category of the driving state to which the new sample data belongs;
acquiring key data in newly added sample data, and processing the key data to form a normalized feature vector;
taking the normalized feature vector and the first category as input of an initial tree convolutional neural network monitoring model, and obtaining an output probability value of newly added sample data at a node of the initial tree convolutional neural network monitoring model;
and adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model.
Preferably, before the step of obtaining the key data in the newly added sample data and processing the key data to form the normalized feature vector, the method further includes:
and preprocessing the newly added sample data.
Preferably, the step of using the normalized feature vector and the first class as input of the initial tree convolutional neural network monitoring model to obtain an output probability value of the newly added sample data at a node of the initial tree convolutional neural network monitoring model includes:
inputting the normalized feature vector and the first category into a first-stage branch node of an initial tree convolutional neural network monitoring model, and outputting a sample probability value of newly added sample data at the first-stage branch node;
calculating the output probability value of the newly added sample data at the first-stage branch node of the initial tree convolutional neural network monitoring model according to the sample probability value of the newly added sample data at the first-stage branch node;
the step of adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model comprises the following steps:
and adding the first class into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node, and updating the initial tree convolutional neural network monitoring model.
Preferably, the step of calculating the output probability value of the newly added sample data at the first stage branch node of the initial tree convolutional neural network monitoring model according to the sample probability value of the newly added sample data at the first stage branch node comprises the following steps:
taking an average value of sample probability values of a plurality of newly-added sample data of the same first-stage branch node as a first node probability value of the first-stage branch node;
and performing softmax function processing on the first node probability value to obtain an output probability value of each first-stage branch node of the initial tree convolutional neural network monitoring model of the newly added sample data.
Preferably, the step of adding the first class to the node of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node, and updating the initial tree convolutional neural network monitoring model includes:
when the output probability values of the first-stage branch nodes are smaller than a preset threshold value, the first class is used as a new first-stage branch node to be added into an initial tree convolutional neural network monitoring model;
when the output probability value of each first-stage branch node is larger than a preset threshold value, generating a new first-stage branch node in an initial convolutional neural network monitoring model, and adding the first class into the generated child nodes of the new first-stage branch node;
And updating the initial number convolutional neural network monitoring model.
Preferably, the step of adding the first class to the node of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node, and updating the initial tree convolutional neural network monitoring model includes:
when the output probability value of only one first-stage branch node is larger than a preset threshold value in the output probability values of the first-stage branch nodes, calculating the output probability value of a second-stage branch node in the first-stage branch nodes;
and adding the first class into the second-level branch nodes or progressive sub-nodes of the second-level branch nodes in the initial tree convolutional neural network monitoring model according to the output probability value of the second-level branch nodes, and updating the initial tree convolutional neural network monitoring model.
Preferably, the step of inputting the normalized feature vector and the first category into a first-stage branch node of the initial tree convolutional neural network monitoring model and outputting a sample probability value of newly added sample data at the first-stage branch node includes:
inputting the normalized feature vector and the first category into each first-stage branch node of an initial tree convolutional neural network monitoring model;
Acquiring weight values of all first-stage branch nodes of the initial tree convolutional neural network model;
and outputting the sample probability value of the newly added sample data at each first-stage branch node according to the normalized feature vector and the weight value of each first-stage branch node.
Preferably, the step of adding the first class to the node of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model includes:
adding the first category into nodes of an initial tree convolutional neural network monitoring model according to the output probability value;
and training the gradient descent of the nodes, updating the node weight of the initial convolutional neural network monitoring model, and finishing updating the initial tree convolutional neural network monitoring model.
In addition, to achieve the above object, the present invention also provides an updating apparatus including: the driving state monitoring model updating method comprises the steps of a memory, a processor and a driving state monitoring model updating program which is stored in the memory and can be run on the processor, wherein the driving state monitoring model updating program is executed by the processor to realize the driving state monitoring model updating method.
In addition, in order to achieve the above object, the present invention also provides a storage medium having stored thereon a driving state monitoring model update program which, when executed by a processor, implements the steps of the driving state monitoring model update method as described above.
The method reads the newly added sample data of the driving state acquired by the driving state acquisition device and the first category of the driving state to which the newly added sample data belongs; acquiring key data in newly added sample data, and processing the key data to form a normalized feature vector; taking the normalized feature vector and the first category as input of an initial tree convolutional neural network monitoring model, and obtaining an output probability value of newly added sample data at a node of the initial tree convolutional neural network monitoring model; and adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model.
By the method, the new sample data can be processed, the normalized feature vector formed after the processing and the first category of the driving state of the new sample data are input into the initial tree convolutional neural network monitoring model, the output probability value of the node is obtained, the category of the driving state is added into the node of the initial monitoring model according to the output probability value, the initial tree convolutional neural network monitoring model is updated, the abnormal driving state monitoring model is dynamically updated through incremental data, the low redundancy of the model can be ensured in the dynamic updating process of the model, the recognition precision of the abnormal driving state and the robustness of the model are improved, the problem of the increase of the batch learning training time index is effectively solved, and the training time of the monitoring model is greatly shortened
Drawings
FIG. 1 is a schematic diagram of a terminal structure of a hardware operating environment according to an embodiment of the present invention;
FIG. 2 is a flowchart of a first embodiment of a driving state monitoring model updating method according to the present invention;
FIG. 3 is a flowchart of a second embodiment of a driving state monitoring model updating method according to the present invention;
FIG. 4 is a flowchart of a third embodiment of a driving state monitoring model updating method according to the present invention;
FIG. 5 is a flowchart of a fourth embodiment of a driving state monitoring model updating method according to the present invention;
FIG. 6 is a flowchart of a fifth embodiment of a driving state monitoring model updating method according to the present invention;
FIG. 7 is a flowchart of a sixth embodiment of a driving state monitoring model updating method according to the present invention;
FIG. 8 is a flowchart of a seventh embodiment of a driving state monitoring model updating method according to the present invention;
fig. 9 is a flowchart of an eighth embodiment of a driving state monitoring model updating method according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The main solutions of the embodiments of the present invention are: reading new sample data of the driving state acquired by the driving state acquisition device and a first category of the driving state to which the new sample data belongs; acquiring key data in newly added sample data, and processing the key data to form a normalized feature vector; taking the normalized feature vector and the first category as input of an initial tree convolutional neural network monitoring model, and obtaining an output probability value of newly added sample data at a node of the initial tree convolutional neural network monitoring model; and adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model.
The existing driving state monitoring model has the problems that the single abnormal state is identified, the identification model is fixed, and the requirement of incremental learning updating cannot be met.
The invention realizes the dynamic update of the abnormal driving state monitoring model through the incremental data, can ensure the low redundancy of the model in the dynamic update process of the model, and improves the recognition precision of the abnormal driving state and the robustness of the model.
As shown in fig. 1, fig. 1 is a schematic diagram of a terminal structure of a hardware running environment according to an embodiment of the present invention.
The terminal of the embodiment of the invention can be a PC, or can be a mobile terminal device with a display function, such as a smart phone, a tablet personal computer, an MP4 (Moving Picture Experts Group Audio Layer IV, dynamic image expert compression standard audio layer 3) player, a portable computer and the like.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, a communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a Display, an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may further include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface). The memory 1005 may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also optionally be a storage device separate from the processor 1001 described above.
Preferably, the terminal may further include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among other sensors, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display screen according to the brightness of ambient light, and a proximity sensor that may turn off the display screen and/or the backlight when the mobile terminal moves to the ear. As one of the motion sensors, the gravity acceleration sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and the direction when the mobile terminal is stationary, and the mobile terminal can be used for recognizing the gesture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; of course, the mobile terminal may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, which are not described herein.
It will be appreciated by those skilled in the art that the terminal structure shown in fig. 1 is not limiting of the terminal and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a monitoring model update program of a driving state may be included in a memory 1005 as one type of computer storage medium.
In the terminal shown in fig. 1, the network interface 1004 is mainly used for connecting to a background server and performing data communication with the background server; the user interface 1003 is mainly used for connecting a client (user side) and performing data communication with the client; and the processor 1001 may be configured to call a monitoring model update program of the driving state stored in the memory 1005, and perform the following operations:
reading new sample data of the driving state acquired by the driving state acquisition device and a first category of the driving state to which the new sample data belongs;
acquiring key data in newly added sample data, and processing the key data to form a normalized feature vector;
taking the normalized feature vector and the first category as input of an initial tree convolutional neural network monitoring model, and obtaining an output probability value of newly added sample data at a node of the initial tree convolutional neural network monitoring model;
And adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model.
Further, the processor 1001 may call a monitoring model update program of the driving state stored in the memory 1005, and further perform the following operations: before the step of obtaining the key data in the newly added sample data and processing the key data to form the normalized feature vector, the method further includes:
and preprocessing the newly added sample data.
Further, the processor 1001 may call a monitoring model update program of the driving state stored in the memory 1005, and further perform the following operations: the step of using the normalized feature vector and the first category as the input of the initial tree convolutional neural network monitoring model to obtain the output probability value of the newly added sample data at the node of the initial tree convolutional neural network monitoring model comprises the following steps:
inputting the normalized feature vector and the first category into a first-stage branch node of an initial tree convolutional neural network monitoring model, and outputting a sample probability value of newly added sample data at the first-stage branch node;
Calculating the output probability value of the newly added sample data at the first-stage branch node of the initial tree convolutional neural network monitoring model according to the sample probability value of the newly added sample data at the first-stage branch node;
the step of adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model comprises the following steps:
and adding the first class into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node, and updating the initial tree convolutional neural network monitoring model.
Further, the processor 1001 may call a monitoring model update program of the driving state stored in the memory 1005, and further perform the following operations: the step of calculating the output probability value of the newly added sample data at the first-stage branch node of the initial tree convolutional neural network monitoring model according to the sample probability value of the newly added sample data at the first-stage branch node comprises the following steps:
taking an average value of sample probability values of a plurality of newly-added sample data of the same first-stage branch node as a first node probability value of the first-stage branch node;
And performing softmax function processing on the first node probability value to obtain an output probability value of each first-stage branch node of the initial tree convolutional neural network monitoring model of the newly added sample data.
Further, the processor 1001 may call a monitoring model update program of the driving state stored in the memory 1005, and further perform the following operations: the step of adding the first category into the node of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node, and updating the initial tree convolutional neural network monitoring model comprises the following steps:
when the output probability values of the first-stage branch nodes are smaller than a preset threshold value, the first class is used as a new first-stage branch node to be added into an initial tree convolutional neural network monitoring model;
when the output probability value of each first-stage branch node is larger than a preset threshold value, generating a new first-stage branch node in an initial convolutional neural network monitoring model, and adding the first class into the generated child nodes of the new first-stage branch node;
and updating the initial number convolutional neural network monitoring model.
Further, the processor 1001 may call a monitoring model update program of the driving state stored in the memory 1005, and further perform the following operations: the step of adding the first category into the node of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node, and updating the initial tree convolutional neural network monitoring model comprises the following steps:
when the output probability value of only one first-stage branch node is larger than a preset threshold value in the output probability values of the first-stage branch nodes, calculating the output probability value of a second-stage branch node in the first-stage branch nodes;
and adding the first class into the second-level branch nodes or progressive sub-nodes of the second-level branch nodes in the initial tree convolutional neural network monitoring model according to the output probability value of the second-level branch nodes, and updating the initial tree convolutional neural network monitoring model.
Further, the processor 1001 may call a monitoring model update program of the driving state stored in the memory 1005, and further perform the following operations: inputting the normalized feature vector and the first category into a first-stage branch node of an initial tree convolutional neural network monitoring model, and outputting a sample probability value of newly added sample data at the first-stage branch node, wherein the step of outputting the sample probability value comprises the following steps:
Inputting the normalized feature vector and the first category into each first-stage branch node of an initial tree convolutional neural network monitoring model;
acquiring weight values of all first-stage branch nodes of the initial tree convolutional neural network model;
and outputting the sample probability value of the newly added sample data at each first-stage branch node according to the normalized feature vector and the weight value of each first-stage branch node.
Further, the processor 1001 may call a monitoring model update program of the driving state stored in the memory 1005, and further perform the following operations: the step of adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model comprises the following steps:
adding the first category into nodes of an initial tree convolutional neural network monitoring model according to the output probability value;
and training the gradient descent of the nodes, updating the node weight of the initial convolutional neural network monitoring model, and finishing updating the initial tree convolutional neural network monitoring model.
Based on the above hardware structure, the method embodiment of the present invention is presented.
Referring to fig. 2, fig. 2 is a flowchart of a first embodiment of a driving state monitoring model updating method according to the present invention, where the driving state monitoring model updating method includes:
Step S10, reading newly added sample data of the driving state acquired by the driving state acquisition device and a first category of the driving state to which the newly added sample data belongs;
the invention is applied to a driving state monitoring system, which comprises a construction module of an initial tree convolutional neural network monitoring model and a monitoring model dynamic updating module, when the system obtains newly-increased sample data of a first category which does not belong to a predefined driving state category, the monitoring model dynamic updating module reads the newly-increased sample data of the driving state acquired by a driving state acquisition device and the category of the driving state to which the newly-increased sample data belongs.
Step S20, obtaining key data in newly added sample data, and processing the key data to form normalized feature vectors;
in an embodiment, key data in the newly added sample data is obtained, for example: the driving state collector comprises a camera, wherein the camera captures newly-added video sample data, key data are data related to the type of the driving state such as the face, the hand and the like of a driver in a video image, the key data of the newly-added sample data can be subjected to feature coding, pooling, normalization and the like to form normalized feature vectors, if the newly-added sample data are the newly-added video sample data, the newly-added video sample data can be subjected to feature coding, pooling, normalization and the like to form normalized feature vectors describing the video, namely normalized feature vectors describing the newly-added video sample data are formed, blink newly-added sample data are read on the assumption that blinks are added into an initial monitoring model, the key data of the newly-added sample data are eye and facial expression image data (namely RGB data) of the driver, and the eye and the facial expression image data are processed to form normalized feature vectors describing the video.
Step S30, taking the normalized feature vector and the first category as input of an initial tree convolutional neural network monitoring model, and obtaining an output probability value of newly added sample data at a node of the initial tree convolutional neural network monitoring model;
the initial Tree convolutional neural network monitoring model is a Tree-CNN monitoring model, and aiming at abnormal driving state monitoring, the state of a driver can have four basic dimensions, namely cognitive load, physical load, external interference and actual running condition of a vehicle, wherein the cognitive load is the condition of judging the load such as the spirit of the driver through physiological data such as heart rate, skin electricity and myoelectricity of the driver, and the physical load is the condition of judging the physical load such as sleepiness and fatigue of the driver through collecting the gazing direction of glasses, yawning, eye blinking and the like through a camera. External interference, such as shooting by a camera whether a driver is playing a mobile phone or not, or drinking tea while driving, and interference of passengers. The actual running condition of the vehicle CAN be judged by the vehicle running information obtained by the vehicle-mounted CAN bus.
The construction module of the initial Tree convolutional neural network monitoring model (Tree-CNN monitoring model) firstly divides abnormal data into a plurality of dimensions, namely 4 dimensions, then sequentially divides each dimension, namely continuously branches and leaves like a Tree, the category obtained by the final leaf node is the category to be finally identified, the Tree-CNN monitoring model comprises a root node, a branch node and a leaf node, the leaf node is the final node in the Tree-CNN monitoring model, abnormal driving can be divided into 4 dimensions in the abnormal driving Tree-CNN monitoring model, and the 4 dimensions are continuously divided, for example: the external interference dimension of 4 dimensions is continuously divided into 3 categories of mobile phone taking, tea taking and passenger interference, wherein the mobile phone taking, tea taking and passenger interference are final nodes, the mobile phone taking category, the tea taking category and the passenger interference category are leaf nodes, if the passenger interference category can be continuously divided into two categories of a passenger driving driver and a passenger rushing steering wheel, the passenger driving driver node and the passenger rushing steering wheel node are final nodes, the passenger driving driver node is leaf node, the passenger interference node is root node of the passenger driving driver node, and the passenger driving driver node is branch node of passenger interference.
The initial monitoring model construction method of the driving state monitoring system comprises the following steps: for example, when the driving state collector is a vehicle-mounted camera,
1) Reading abnormal driving state data acquired by the vehicle-mounted camera and categories of the abnormal driving state data;
2) The method comprises the following steps of carrying out key pretreatment on a driver video, wherein the key pretreatment comprises an illumination correction method, a communication component quick marking, a direction projection and the like;
3) The method comprises the steps of carrying out a series of operations such as feature coding, pooling and normalization on key information such as the face and the hand of a driver to finally form a normalized feature vector describing a video;
4) And combining the extracted normalized feature vector with the abnormal category to construct a Tree-CNN monitoring model.
And when the driving state collector further comprises an acceleration gyroscope sensor, reading calibrated new types of acceleration gyroscope sensor data, and performing 2) and 3) processing, and constructing a Tree-CNN monitoring model based on the convolutional neural network by integrating the data acquired by the vehicle-mounted camera, namely constructing and forming an initial Tree convolutional neural network monitoring model.
The Tree-CNN monitoring model references a layer classifier, the Tree convolutional neural network is composed of nodes, each node has its own ID, father (Parent) and child (child), net (convolutional neural network for processing images), LT ("Labels Transform", label corresponding to each node, for root node and branch node, it can be a division of the final classification class, for leaf node, it is the final classification class.) where the top is the root node of the Tree. For a sample, the sample is firstly sent to a root node network to be classified to obtain super-classes, then the sample is sent to a corresponding node to be further classified according to the identified super-classes, so as to obtain a more specific class, and recursion is sequentially carried out until the class which is finally wanted by the user is classified. For fine-grained classification of abnormal states, there is, for example, the following stack of labels: fatigue driving, distraction, sudden diseases and emotional driving. If emotional driving is to be identified, we may go through the process of finding abnormal states at the cognitive level in the heap and then finding emotional driving.
Taking the normalized feature vector and the first category of the driving state to which the newly added sample data belong as the input of the initial tree convolutional neural network monitoring model, and obtaining the output probability value of the newly added sample data at the node of the initial tree convolutional neural network monitoring model, for example: the method comprises the steps of obtaining a matrix (244 x 3) of feature vectors by preprocessing new sample data of a plurality of images acquired in advance, inputting the matrix into a first-stage branch node of a network of an initial Tree-CNN monitoring model, multiplying each layer by weights obtained during initial model training, and finally obtaining a sample probability value O (K, M, I) of the first-stage branch node, wherein O (K, M, I) represents a sample probability value of a kth node when the ith new sample data belongs to the mth class, wherein K epsilon [1, K ], M epsilon [1, M ], I epsilon [1, I ], the number of first-stage branch nodes in the K initial number convolutional neural network monitoring model, M represents the class number of driving states in the new sample data, and I represents the sample number of each new class.
Taking an average value of sample probability values of a plurality of newly added sample data of the same first-stage branch node as a first node probability value of the first-stage branch node, and adopting the following calculation formula:
Wherein, K represents the number of first-stage branch nodes in the initial number convolutional neural network monitoring model, M represents the number of categories of the driving state of the newly added sample data, and I represents the number of samples of each newly added category; o (k, m, i) represents a sample probability value of a kth node when the ith newly added sample data belongs to the mth class, oavg (k, m) represents the newly added sample data of the mth class, and a first node probability value of a kth first-stage branch node;
and carrying out softmax function processing on the probability value of the first node to obtain the output probability value of each first-stage branch node of the new sample data in the initial tree convolutional neural network monitoring model, wherein the calculation formula is as follows:
wherein L (k, m) represents newly added sample data of the mth class, and an output probability value at the kth first-stage branch node.
If the output probability value of the first-stage branch node is larger than the preset threshold value in the initial Tree-CNN monitoring model, continuously calculating the output probability value of the branch node under the first-stage branch node which is larger than the preset threshold value according to the calculation method, namely the output probability value of the second-stage branch node, and if only one of the output probability values of the second-stage branch nodes is larger than the preset threshold value, continuously calculating the output probability value of the third-stage branch node until the first-stage branch node is added into the Tree-CNN monitoring model to become a leaf node.
And step S40, adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model.
Adding the first category into the node of the initial tree convolutional neural network monitoring model according to the output probability value, and understandably adding the first category of the driving state of the m-th newly added sample data into the node of the initial tree convolutional neural network monitoring model,
according to the formula (3), the output probability values of the first-stage branch nodes are arranged according to probability distribution conditions, and a set S is obtained.
S＝[l m1 ，l m2 ，l m3 ，…，l mM ]，max k (l m1 (k))≥max k (l m2 (k))≥…≥max k (l mM (k)) (3)
If a plurality of probability values in the output set S are larger than a preset threshold value, generating a new first-stage branch node in the initial convolutional neural network monitoring model, and adding the first class into the generated child nodes of the new first-stage branch node. For example, if the newly added category is abnormal (brake failure, tire problem, etc.), the abnormal vehicle is set as a new first-stage branch node, and the brake failure and the tire problem are added into child nodes of the abnormal vehicle to form leaf nodes together with the same level of fatigue driving.
And if all the output probability values in the output set S are smaller than a preset threshold value, adding the first class serving as a new first-stage branch node into an initial tree convolutional neural network monitoring model, wherein the first-stage branch node is a leaf node.
If the maximum output probability value l in the output set S m1 When the output probability value of the first-stage branch node is larger than the preset threshold, as shown in the formula (4), namely, when the output probability value of only one first-stage branch node is larger than the preset threshold, the relation between the first category and the first-stage branch node is known to be larger, the output probability value of the branch node under the first-stage branch node which is larger than the preset threshold, namely, the output probability value of the second-stage branch node is continuously calculated, and when only the output probability value of one second-stage branch node in the output probability values of the second-stage branch node is larger than the preset threshold, the output probability value of the third-stage branch node is continuously calculated by analogy until the first category is added into the Tree-CNN monitoring model to become a leaf node.
By the method, the new category is learned, namely the incremental learning is dynamically updated. For example, if the new class does not belong to any existing major class, the new class is singly arranged in parallel with the major class of fatigue driving to form leaf nodes, and if the similar situation occurs later, the result is directly output, and the division is not continued.
When the first level branch node M class completes the operation, the first level branch node M class moves to the second level branch node level of the tree. The second level branch node calculation method is the same as the mode of adding the first class to the model, and when a new class is to be added to the model. When the next level is entered, only those sample images of the newly added class assigned to the particular child node are displayed. In general, for one level, all M classes of sample images are passed through different child nodes. For example, two new classes are added to one child node. If the child node is a leaf node, it will be changed to a branch node, which now has 3 child nodes. If the child node is a branch node, the process of calculating the likelihood matrix is repeated and it is determined how to add the two new classes to its output. The decision on how to grow the tree is semi-supervised in that the algorithm itself decides how to grow the tree given constraints. The values of the parameters may be limited according to requirements, such as the maximum child node of the node, the maximum depth of the tree, etc. After assigning new classes in the tree, the modified/new nodes are trained based on supervised gradient descent. In this way, the entire network does not have to be modified, and only the affected part of the network needs to be retrained/trimmed. When the original new abnormal driving state is changed, only a retraining mode can be adopted, the abnormal driving state monitoring model is dynamically updated through incremental data, the low redundancy of the model can be ensured in the dynamic model updating process, and the abnormal driving state identification precision and the model robustness are improved.
Further, referring to fig. 3, fig. 3 is a flowchart illustrating a second embodiment of a driving state monitoring model updating method according to the present invention. Based on the embodiment shown in fig. 2, step S20 may further include:
s50, preprocessing the newly added sample data;
preprocessing the newly added sample data, for example: when the sample data is newly added to the driver video collected by the vehicle-mounted camera, key preprocessing is carried out on the driver video, wherein the key preprocessing comprises an illumination correction method, a communication component quick marking, direction projection and the like; and acquiring the key data of the newly added sample data after preprocessing, and processing the key data to form a normalized feature vector. And increasing the accuracy of the newly added sample data.
Further, referring to fig. 4, fig. 4 is a flowchart of a third embodiment of a driving state monitoring model updating method according to the present invention. Based on the above embodiment, step S30 may further include:
step S31, inputting the normalized feature vector and the first category into a first-stage branch node of an initial tree convolutional neural network monitoring model, and outputting a sample probability value of newly added sample data at the first-stage branch node;
taking the normalized feature vector and the first category of the driving state to which the newly added sample data belong as the input of the initial tree convolutional neural network monitoring model, and obtaining the output probability value of the newly added sample data at the node of the initial tree convolutional neural network monitoring model, for example: the method comprises the steps of obtaining a matrix (244 x 3) of feature vectors by preprocessing new sample data of a plurality of images acquired in advance, inputting the matrix into a first-stage branch node of a network of an initial Tree-CNN monitoring model, multiplying each layer by weights obtained during initial model training, and finally obtaining a sample probability value O (K, M, I) of the first-stage branch node, wherein O (K, M, I) represents a sample probability value of a kth node when the ith new sample data belongs to the mth class, wherein K epsilon [1, K ], M epsilon [1, M ], I epsilon [1, I ], the number of first-stage branch nodes in the K initial number convolutional neural network monitoring model, M represents the class number of driving states in the new sample data, and I represents the sample number of each new class.
Step S32, calculating the output probability value of the newly added sample data at the first-stage branch node of the initial tree convolutional neural network monitoring model according to the sample probability value of the newly added sample data at the first-stage branch node;
further, referring to fig. 5, fig. 5 is a flowchart of a fourth embodiment of a driving state monitoring model updating method according to the present invention. Based on the above embodiment, step S32 may further include:
step S321, taking the average value of the sample probability values of a plurality of newly-added sample data of the same first-stage branch node as the first-stage branch node probability value;
the first node probability value is calculated as follows:
wherein, K represents the number of first-stage branch nodes in the initial number convolutional neural network monitoring model, M represents the number of categories of the driving state of the newly added sample data, and I represents the number of samples of each newly added category; o (k, m, i) represents a sample probability value of a kth node when the ith newly added sample data belongs to the mth class, oavg (k, m) represents the newly added sample data of the mth class, and a first node probability value of a kth first-stage branch node;
and step S322, performing softmax function processing on the first node probability value to obtain the output probability value of each first-stage branch node of the initial tree convolutional neural network monitoring model of the newly added sample data.
The output probability value of the first-stage branch node is calculated as follows:
wherein L (k, m) represents newly added sample data of the mth class, and an output probability value at the kth first-stage branch node.
And calculating the sample probability value of the class of the driving state of the newly added sample data at the first-stage branch node according to the step S321 and the step S322, monitoring the first-stage branch node probability value of the model at the initial tree convolutional neural network, and processing the first-stage node probability value to obtain the output probability value of the first-stage branch node.
Step S40 may further include:
and S41, adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node, and updating the initial tree convolutional neural network monitoring model.
And adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node and the magnitude of a preset threshold value, and updating the initial tree convolutional neural network monitoring model.
Further, referring to fig. 6, fig. 6 is a flowchart of a fifth embodiment of a driving state monitoring model updating method according to the present invention. Based on the above embodiment, step S41 may further include:
Step S411, when the output probability values of the first-stage branch nodes are smaller than a preset threshold value, the first class is used as a new first-stage branch node to be added into an initial tree convolutional neural network monitoring model;
when the output probability values of the first-stage branch nodes are smaller than a preset threshold value, the first-stage branch nodes are added into an initial tree convolutional neural network monitoring model as new first-stage branch nodes, and the first-stage branch nodes are leaf nodes.
Step S412, when the output probability value of each first-stage branch node is larger than a preset threshold value, generating a new first-stage branch node in the initial convolutional neural network monitoring model, and adding the first class into the generated child nodes of the new first-stage branch node;
and when the output probability values of a plurality of first-stage branch nodes are larger than a preset threshold value, generating new first-stage branch nodes in an initial convolutional neural network monitoring model, and adding the first types into the generated child nodes of the new first-stage branch nodes. For example, if the newly added category is abnormal (brake failure, tire problem, etc.), the abnormal vehicle is set as a new first-stage branch node, and the brake failure and the tire problem are added into child nodes of the abnormal vehicle to form leaf nodes together with the same level of fatigue driving.
And step S413, updating the initial number convolutional neural network monitoring model.
When the first category is added into the nodes of the initial tree convolutional neural network monitoring model, the modified/new nodes are trained based on the supervised gradient descent, the node weight of the initial convolutional neural network monitoring model is updated, and the updating of the initial tree convolutional neural network monitoring model is completed.
Further, referring to fig. 7, fig. 7 is a flowchart of a sixth embodiment of a driving state monitoring model updating method according to the present invention. Based on the above embodiment, step S41 may further include:
step S414, when the output probability value of only one first-stage branch node is larger than the preset threshold value in the output probability values of the first-stage branch nodes, calculating the output probability value of a second-stage branch node in the first-stage branch nodes;
and when the output probability value of only one first-stage branch node is larger than a preset threshold value, the relation between the first category and the first-stage branch node is larger, and the output probability value of the branch node under the first-stage branch node which is larger than the preset threshold value, namely the output probability value of the second-stage branch node, is continuously calculated.
And step 415, adding the first class into the second-level branch nodes or progressive sub-nodes of the second-level branch nodes in the initial tree convolutional neural network monitoring model according to the output probability value of the second-level branch nodes, and updating the initial tree convolutional neural network monitoring model.
And if the output probability values of the second-level branch nodes are larger than a preset threshold value, adding the first class into the generated child nodes of the new second-level branch nodes.
And when the output probability values of the second-level branch nodes are smaller than a preset threshold value, the first-level branch nodes are used as new second-level branch nodes and added into the initial tree convolutional neural network monitoring model, and the second-level branch nodes are leaf nodes.
And if the output probability value of only one second-level branch node in the output probability values of the second-level branch nodes is larger than a preset threshold value, and so on, continuously calculating the output probability value of the third-level branch node until the first class is added into the Tree-CNN monitoring model to become a leaf node.
Adding the category of the driving state to the node of the initial tree convolutional neural network monitoring model according to the output probability value of the node, performing supervised gradient descent-based training on the modified/new node, updating the node weight of the initial convolutional neural network monitoring model, completing updating the initial tree convolutional neural network monitoring model, and realizing learning of the new category, namely incremental learning dynamic updating.
Further, referring to fig. 8, fig. 8 is a flowchart of a seventh embodiment of a driving state monitoring model updating method according to the present invention. Based on the above embodiment, step S31 may further include:
step S311, inputting the normalized feature vector and the first category into each first-stage branch node of the initial tree convolutional neural network monitoring model;
taking the normalized feature vector and the first category of the driving state to which the newly added sample data belong as the input of the initial tree convolutional neural network monitoring model, and obtaining the output probability value of the newly added sample data at the node of the initial tree convolutional neural network monitoring model, for example: two abnormal driving behaviors of yawning and eye closing are needed to be added, a matrix (244 x 3 for example) of a feature vector is obtained by preprocessing a plurality of image newly added sample data acquired in advance, and then the feature vector is input into a first-stage branch node of a network of an initial Tree-CNN monitoring model.
Step S312, obtaining weight values of all first-stage branch nodes of the initial tree convolutional neural network model;
the weight values of all the level branch nodes of the initial number convolution neural network model are formed during training, and the weight values of all the first level branch nodes are obtained.
Step S313, outputting the sample probability value of the newly added sample data at each first stage branch node according to the normalized feature vector and the weight value of each first stage branch node.
And multiplying the normalized feature vector by a weight value to output a sample probability value O (K, M, I) of a first-stage branch node of the newly added sample data in the monitoring model, wherein O (K, M, I) represents the sample probability value of a kth node when the ith newly added sample data belongs to the mth class, K is [1, K ], M is [1, M ], I is [1, I ], the number of the first-stage branch nodes in the K initial number convolution neural network monitoring model is equal to the number of the first-stage branch nodes in the monitoring model, M represents the class number of the driving state to which the newly added sample data belongs, and I represents the sample number of each newly added class.
Further, referring to fig. 9, fig. 9 is a flowchart of an eighth embodiment of a driving state monitoring model updating method according to the present invention. Based on the above embodiment, step S40 may further include:
step S42, adding the first category into nodes of an initial tree convolutional neural network monitoring model according to the output probability value;
adding the class of the driving state to the nodes in the initial tree convolutional neural network model according to the output probability value of the nodes of the monitoring model, for example: if a yawning new class is added to the fatigue driving, the fatigue driving branch node is changed into a fatigue driving branch node like an initial low head, and the yawning and the low head are leaf nodes of a monitoring model.
And step S43, performing gradient descent training on the nodes, updating the node weight of the initial convolutional neural network monitoring model, and finishing updating the initial tree convolutional neural network monitoring model.
After adding new category, after distributing new category in tree, training modified/new node based on monitoring gradient drop, updating weight of initial model, and finishing updating initial tree convolutional neural network monitoring model. The entire network does not have to be modified and only the affected part of the network needs to be retrained/trimmed.
The device for updating the monitoring model of the driving state comprises: the driving state monitoring model updating method comprises the steps of a memory, a processor and a driving state monitoring model updating program which is stored in the memory and can be run on the processor, wherein the driving state monitoring model updating program is executed by the processor to realize the driving state monitoring model updating method.
The method implemented when the driving state monitoring model updating program running on the processor is executed may refer to various embodiments of the driving state monitoring model updating method of the present invention, which are not described herein.
The invention also provides a storage medium.
The storage medium of the present invention stores thereon a driving state monitoring model updating program which, when executed by a processor, implements the steps of the driving state monitoring model updating method described above.
The method implemented when the driving state monitoring model updating program running on the processor is executed may refer to various embodiments of the driving state monitoring model updating method of the present invention, which are not described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (10)
1. A method for updating a monitoring model of a driving state, the method comprising the steps of:
Reading new sample data corresponding to the driving state acquired by the driving state acquisition device and a first category of the driving state to which the new sample data belongs, wherein the new sample data represents data which does not belong to a predefined driving state category;
obtaining key data in newly-added sample data, and processing the key data to form normalized feature vectors, wherein the key data in the newly-added sample data are eye expression image data and facial expression image data of a driver;
taking the normalized feature vector and the first category as input of an initial tree convolutional neural network monitoring model, and obtaining an output probability value of newly added sample data at a node of the initial tree convolutional neural network monitoring model;
and adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model.
2. The method for updating a monitoring model of a driving state according to claim 1, wherein before the step of obtaining key data in the newly added sample data and processing the key data to form a normalized feature vector, the method further comprises:
And preprocessing the newly added sample data.
3. The method for updating a driving state monitoring model according to claim 2, wherein the step of using the normalized feature vector and the first class as an input of the initial tree convolutional neural network monitoring model to obtain an output probability value of the newly added sample data at a node of the initial tree convolutional neural network monitoring model comprises:
inputting the normalized feature vector and the first category into a first-stage branch node of an initial tree convolutional neural network monitoring model, and outputting a sample probability value of newly added sample data at the first-stage branch node;
calculating the output probability value of the newly added sample data at the first-stage branch node of the initial tree convolutional neural network monitoring model according to the sample probability value of the newly added sample data at the first-stage branch node;
the step of adding the first category into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value, and updating the initial tree convolutional neural network monitoring model comprises the following steps:
and adding the first class into the nodes of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node, and updating the initial tree convolutional neural network monitoring model.
4. The driving state monitoring model updating method according to claim 3, wherein the step of calculating the output probability value of the newly added sample data at the first level branch node of the initial tree convolutional neural network monitoring model according to the sample probability value of the newly added sample data at the first level branch node comprises:
taking an average value of sample probability values of a plurality of newly-added sample data of the same first-stage branch node as a first node probability value of the first-stage branch node;
and performing softmax function processing on the first node probability value to obtain an output probability value of each first-stage branch node of the initial tree convolutional neural network monitoring model of the newly added sample data.
5. The method for updating a driving state monitoring model according to claim 3, wherein the step of adding the first class to the nodes of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node comprises:
when the output probability values of the first-stage branch nodes are smaller than a preset threshold value, the first class is used as a new first-stage branch node to be added into an initial tree convolutional neural network monitoring model;
When the output probability value of each first-stage branch node is larger than a preset threshold value, generating a new first-stage branch node in an initial convolutional neural network monitoring model, and adding the first class into the generated child nodes of the new first-stage branch node;
and updating the initial number convolutional neural network monitoring model.
6. The method for updating a driving state monitoring model according to claim 5, wherein the step of adding the first class to the nodes of the initial tree convolutional neural network monitoring model according to the output probability value of the first-stage branch node comprises:
when the output probability value of only one first-stage branch node is larger than a preset threshold value in the output probability values of the first-stage branch nodes, calculating the output probability value of a second-stage branch node in the first-stage branch nodes;
and adding the first class into the second-level branch nodes or progressive sub-nodes of the second-level branch nodes in the initial tree convolutional neural network monitoring model according to the output probability value of the second-level branch nodes, and updating the initial tree convolutional neural network monitoring model.
7. The method for updating a monitoring model of a driving state according to claim 6, wherein,
inputting the normalized feature vector and the first category into a first-stage branch node of an initial tree convolutional neural network monitoring model, and outputting a sample probability value of newly added sample data at the first-stage branch node, wherein the step of outputting the sample probability value comprises the following steps:
inputting the normalized feature vector and the first category into each first-stage branch node of an initial tree convolutional neural network monitoring model;
acquiring weight values of all first-stage branch nodes of the initial tree convolutional neural network model;
and outputting the sample probability value of the newly added sample data at each first-stage branch node according to the normalized feature vector and the weight value of each first-stage branch node.
8. The driving state monitoring model updating method according to claim 7, wherein the step of adding the first class to a node of an initial tree convolutional neural network monitoring model according to the output probability value comprises:
adding the first category into nodes of an initial tree convolutional neural network monitoring model according to the output probability value;
And training the gradient descent of the nodes, updating the node weight of the initial convolutional neural network monitoring model, and finishing updating the initial tree convolutional neural network monitoring model.
9. An updating apparatus, characterized in that the updating apparatus comprises: memory, a processor and a driving state monitoring model update program stored on the memory and executable on the processor, which when executed by the processor, implements the steps of the driving state monitoring model update method according to any one of claims 1 to 8.
10. A storage medium, wherein a driving state monitoring model update program is stored on the storage medium, which when executed by a processor, implements the steps of the driving state monitoring model update method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910103594.1A CN109858553B (en) | 2019-01-31 | 2019-01-31 | Method, device and storage medium for updating driving state monitoring model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910103594.1A CN109858553B (en) | 2019-01-31 | 2019-01-31 | Method, device and storage medium for updating driving state monitoring model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109858553A CN109858553A (en) | 2019-06-07 |
| CN109858553B true CN109858553B (en) | 2023-12-12 |
Family
ID=66897446
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910103594.1A Active CN109858553B (en) | 2019-01-31 | 2019-01-31 | Method, device and storage medium for updating driving state monitoring model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109858553B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110693508A (en) * | 2019-09-02 | 2020-01-17 | 中国航天员科研训练中心 | Multi-channel cooperative psychophysiological active sensing method and service robot |
| CN112580795A (en) * | 2019-09-29 | 2021-03-30 | 华为技术有限公司 | Neural network acquisition method and related equipment |
| CN113837211A (en) * | 2020-06-23 | 2021-12-24 | 华为技术有限公司 | Driving decision method and device |
| CN113837834B (en) * | 2021-09-22 | 2023-09-01 | 东北大学 | Shopping basket recommendation method based on deep learning |
| CN114355184B (en) * | 2022-01-05 | 2023-09-26 | 国网江苏省电力有限公司宿迁供电分公司 | Online learning-based high-voltage circuit breaker state monitoring and early warning method |
| CN114565964A (en) * | 2022-03-03 | 2022-05-31 | 网易（杭州）网络有限公司 | Emotion recognition model generation method, recognition method, device, medium and equipment |
Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007122362A (en) * | 2005-10-27 | 2007-05-17 | Toyota Motor Corp | State estimation method using neural network and state estimation apparatus using neural network |
| CN101038623A (en) * | 2006-03-13 | 2007-09-19 | 欧姆龙株式会社 | Feature point detecting device, feature point detecting method, and feature point detecting program |
| JP2007257043A (en) * | 2006-03-20 | 2007-10-04 | Nissan Motor Co Ltd | Occupant state estimating device and occupant state estimating method |
| CN102750827A (en) * | 2012-06-26 | 2012-10-24 | 浙江大学 | System for sampling and identifying data of driver response behaviors under group guidance information |
| CN104036299A (en) * | 2014-06-10 | 2014-09-10 | 电子科技大学 | Human eye contour tracking method based on local texture AAM |
| CN104504400A (en) * | 2015-01-08 | 2015-04-08 | 中国科学院重庆绿色智能技术研究院 | Detection method of driver's abnormal behavior modeled based on online behavior |
| CN106909879A (en) * | 2017-01-11 | 2017-06-30 | 开易（北京）科技有限公司 | A kind of method for detecting fatigue driving and system |
| CN106934368A (en) * | 2017-03-13 | 2017-07-07 | 长安大学 | A kind of driving fatigue detecting system and recognition methods based on the dynamic achievement data of eye |
| CN107316457A (en) * | 2016-04-27 | 2017-11-03 | 上海炬宏信息技术有限公司 | Judge whether road traffic condition meets the method for automatic driving |
| CN107437147A (en) * | 2017-08-02 | 2017-12-05 | 辽宁友邦网络科技有限公司 | Reduce the vehicle travel risk dynamic assessment method and its system of freight logistics scene |
| CN107907886A (en) * | 2017-11-07 | 2018-04-13 | 广东欧珀移动通信有限公司 | Travel conditions recognition methods, device, storage medium and terminal device |
| CN108248608A (en) * | 2017-12-28 | 2018-07-06 | 北京百度网讯科技有限公司 | For assessing the method and apparatus of the comfort level of control loop |
| CN108363968A (en) * | 2018-01-31 | 2018-08-03 | 上海瀚所信息技术有限公司 | A kind of tired driver driving monitoring system and method based on key point extraction |
| CN108423006A (en) * | 2018-02-02 | 2018-08-21 | 辽宁友邦网络科技有限公司 | A kind of auxiliary driving warning method and system |
| CN108520268A (en) * | 2018-03-09 | 2018-09-11 | 浙江工业大学 | The black box antagonism attack defense method evolved based on samples selection and model |
| CN108596087A (en) * | 2018-04-23 | 2018-09-28 | 合肥湛达智能科技有限公司 | A kind of driving fatigue degree detecting regression model based on dual network result |
| CN108688675A (en) * | 2017-03-29 | 2018-10-23 | 马自达汽车株式会社 | Vehicle drive support system |
| CN108806667A (en) * | 2018-05-29 | 2018-11-13 | 重庆大学 | The method for synchronously recognizing of voice and mood based on neural network |
| CN109284661A (en) * | 2017-07-21 | 2019-01-29 | 松下电器（美国）知识产权公司 | Learning data generating means, learning data generation method and program |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4683123B2 (en) * | 2008-12-24 | 2011-05-11 | 株式会社デンソー | Action estimation device, program |
| US9493130B2 (en) * | 2011-04-22 | 2016-11-15 | Angel A. Penilla | Methods and systems for communicating content to connected vehicle users based detected tone/mood in voice input |
| WO2013105869A1 (en) * | 2012-01-13 | 2013-07-18 | Pulse Function F6 Limited | Telematics system with 3d inertial sensors |
| US10210761B2 (en) * | 2013-09-30 | 2019-02-19 | Sackett Solutions & Innovations, LLC | Driving assistance systems and methods |
| US20180025289A1 (en) * | 2016-07-20 | 2018-01-25 | Qualcomm Incorporated | Performance Provisioning Using Machine Learning Based Automated Workload Classification |
| US20180053102A1 (en) * | 2016-08-16 | 2018-02-22 | Toyota Jidosha Kabushiki Kaisha | Individualized Adaptation of Driver Action Prediction Models |
| US11573573B2 (en) * | 2017-06-06 | 2023-02-07 | Plusai, Inc. | Method and system for distributed learning and adaptation in autonomous driving vehicles |
-
2019
- 2019-01-31 CN CN201910103594.1A patent/CN109858553B/en active Active
Patent Citations (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2007122362A (en) * | 2005-10-27 | 2007-05-17 | Toyota Motor Corp | State estimation method using neural network and state estimation apparatus using neural network |
| CN101038623A (en) * | 2006-03-13 | 2007-09-19 | 欧姆龙株式会社 | Feature point detecting device, feature point detecting method, and feature point detecting program |
| JP2007257043A (en) * | 2006-03-20 | 2007-10-04 | Nissan Motor Co Ltd | Occupant state estimating device and occupant state estimating method |
| CN102750827A (en) * | 2012-06-26 | 2012-10-24 | 浙江大学 | System for sampling and identifying data of driver response behaviors under group guidance information |
| CN104036299A (en) * | 2014-06-10 | 2014-09-10 | 电子科技大学 | Human eye contour tracking method based on local texture AAM |
| CN104504400A (en) * | 2015-01-08 | 2015-04-08 | 中国科学院重庆绿色智能技术研究院 | Detection method of driver's abnormal behavior modeled based on online behavior |
| CN107316457A (en) * | 2016-04-27 | 2017-11-03 | 上海炬宏信息技术有限公司 | Judge whether road traffic condition meets the method for automatic driving |
| CN106909879A (en) * | 2017-01-11 | 2017-06-30 | 开易（北京）科技有限公司 | A kind of method for detecting fatigue driving and system |
| CN106934368A (en) * | 2017-03-13 | 2017-07-07 | 长安大学 | A kind of driving fatigue detecting system and recognition methods based on the dynamic achievement data of eye |
| CN108688675A (en) * | 2017-03-29 | 2018-10-23 | 马自达汽车株式会社 | Vehicle drive support system |
| CN109284661A (en) * | 2017-07-21 | 2019-01-29 | 松下电器（美国）知识产权公司 | Learning data generating means, learning data generation method and program |
| CN107437147A (en) * | 2017-08-02 | 2017-12-05 | 辽宁友邦网络科技有限公司 | Reduce the vehicle travel risk dynamic assessment method and its system of freight logistics scene |
| CN107907886A (en) * | 2017-11-07 | 2018-04-13 | 广东欧珀移动通信有限公司 | Travel conditions recognition methods, device, storage medium and terminal device |
| CN108248608A (en) * | 2017-12-28 | 2018-07-06 | 北京百度网讯科技有限公司 | For assessing the method and apparatus of the comfort level of control loop |
| CN108363968A (en) * | 2018-01-31 | 2018-08-03 | 上海瀚所信息技术有限公司 | A kind of tired driver driving monitoring system and method based on key point extraction |
| CN108423006A (en) * | 2018-02-02 | 2018-08-21 | 辽宁友邦网络科技有限公司 | A kind of auxiliary driving warning method and system |
| CN108520268A (en) * | 2018-03-09 | 2018-09-11 | 浙江工业大学 | The black box antagonism attack defense method evolved based on samples selection and model |
| CN108596087A (en) * | 2018-04-23 | 2018-09-28 | 合肥湛达智能科技有限公司 | A kind of driving fatigue degree detecting regression model based on dual network result |
| CN108806667A (en) * | 2018-05-29 | 2018-11-13 | 重庆大学 | The method for synchronously recognizing of voice and mood based on neural network |
Non-Patent Citations (3)
| Title |
|---|
| 一种考虑驾驶员姿态及光照因素的疲劳驾驶稳健检测算法;张旭东;严宏斌;张忠伟;周亚同;;电子技术(第12期);第13、26-30页 * |
| 基于深度时空域卷积神经网络的表情识别模型;杨格兰;邓晓军;刘琮;;中南大学学报(自然科学版)(第07期);第2311-2319页 * |
| 驾驶员眼部状态识别方法的研究;郭克友, 储江伟, 王荣本;系统工程与电子技术(第10期);第1186-1188、1228页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109858553A (en) | 2019-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109858553B (en) | Method, device and storage medium for updating driving state monitoring model | |
| CN106682602B (en) | Driver behavior identification method and terminal | |
| CN111310562B (en) | Vehicle driving risk management and control method based on artificial intelligence and related equipment thereof | |
| CN111008640B (en) | Image recognition model training and image recognition method, device, terminal and medium | |
| KR20190094133A (en) | An artificial intelligence apparatus for recognizing object and method for the same | |
| CN113015984A (en) | Error correction in convolutional neural networks | |
| CN103745235A (en) | Human face identification method, device and terminal device | |
| CN111797854B (en) | Scene model building method and device, storage medium and electronic equipment | |
| CN109949438B (en) | Abnormal driving monitoring model establishing method and device and storage medium | |
| CN111091044B (en) | Network appointment-oriented in-vehicle dangerous scene identification method | |
| CN110516113B (en) | Video classification method, video classification model training method and device | |
| CN110264093B (en) | Credit model establishing method, device, equipment and readable storage medium | |
| CN103106394A (en) | Human body action recognition method in video surveillance | |
| CN110135497A (en) | Method, the method and device of Facial action unit intensity estimation of model training | |
| CN111694954B (en) | Image classification method and device and electronic equipment | |
| CN111340213B (en) | Neural network training method, electronic device, and storage medium | |
| KR20200080418A (en) | Terminla and operating method thereof | |
| CN113435335B (en) | Microscopic expression recognition method and device, electronic equipment and storage medium | |
| CN107423663A (en) | A kind of image processing method and terminal | |
| CN115954019A (en) | Environmental noise identification method and system integrating self-attention and convolution operation | |
| CN111814523A (en) | Human body activity recognition method and device | |
| CN111797849A (en) | User activity identification method and device, storage medium and electronic equipment | |
| CN111796663B (en) | Scene recognition model updating method and device, storage medium and electronic equipment | |
| CN114140822A (en) | Pedestrian re-identification method and device | |
| CN210983433U (en) | Driver smoking detection system based on computer vision technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 518000 706, building 3b, hongrongyuan Shangjun phase II, Longping community, Dalang street, Longhua District, Shenzhen City, Guangdong Province Applicant after: Jintu computing technology (Shenzhen) Co.,Ltd. Address before: 518000 area a, 18 / F, building 1, creative park, broadcasting group, No.1 Qingqing Road, Qinghu community, Longhua sub district office, Longhua District, Shenzhen City, Guangdong Province Applicant before: SHENZHEN SEMISKY TECHNOLOGY Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |