WO2022142614A1

WO2022142614A1 - Dangerous driving early warning method and apparatus, computer device and storage medium

Info

Publication number: WO2022142614A1
Application number: PCT/CN2021/125225
Authority: WO
Inventors: 熊玮
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2020-12-28
Filing date: 2021-10-21
Publication date: 2022-07-07
Also published as: CN112820072A

Abstract

The present application relates to the technical field of micro-expression recognition, and discloses a dangerous driving early warning method and apparatus, a computer device, and a storage medium. The method comprises: associating and recording face images acquired in real time during a vehicle driving process as a face image sequence according to an acquisition order; detecting whether there are changes in micro-expressions in a face image in the face image sequence, and when it is detected that there are changes in micro-expressions in a face image, acquiring target expression features of the face image after the micro-expressions have changed; inputting the target expression features into a preset expression encoding system to determine a target expression category; if the target expression category belongs to a preset dangerous expression category, acquiring dialogue information of a driver by means of a multi-round dialogue device; according to voiceprint features in the dialogue information and a preset fatigue measurement table, determining whether the driver is driving while fatigued; and when it is determined that the driver is driving while fatigued, triggering a dangerous driving voice prompt according to the voiceprint features and the target expression category. The present application improves the accuracy of dangerous driving early warning.

Description

Dangerous driving warning method, device, computer equipment and storage medium

This application claims the priority of the Chinese patent application with the application number 202011584251.0 and the invention titled "Dangerous Driving Warning Method, Device, Computer Equipment and Storage Medium", which was filed with the China Patent Office on December 28, 2020, the entire contents of which are by reference Incorporated in this application.

technical field

The present application relates to the technical field of micro-expression recognition, and in particular, to a dangerous driving early warning method, device, computer equipment and storage medium.

Background technique

At present, with the improvement of people's living standards, the traffic flow on the road increases year by year, and the number of traffic accidents also increases. Among them, dangerous driving behavior is one of the main causes of traffic accidents, so early warning of dangerous driving behavior is very important.

The inventor realized that the existing dangerous driving warning system detects the driver's driving behavior through hardware devices installed on the car, and issues a warning to the driver when a driving violation occurs, for example, by detecting the speed of the car to make a judgment. However, this method has the problem that the accuracy of dangerous driving warning is low, and the sudden alarm is more likely to cause panic of the driver, resulting in an increased possibility of an accident.

Application content

Embodiments of the present application provide a dangerous driving warning method, device, computer equipment and storage medium to solve the problem of low accuracy of dangerous driving warning.

A dangerous driving warning method, comprising:

Acquire the face image of the driver in real time during the driving of the vehicle, and associate and record the acquired face image as a sequence of face images according to the acquisition sequence;

Detecting whether the facial images in the sequence of facial images have micro-expression changes, and when detecting that the facial images have micro-expression changes, acquiring the target expression features of the face images after the micro-expression changes; the target expression features Refers to the facial expression feature with the largest difference from the first facial image among all the second facial images; the first facial image refers to the first facial image of the first micro-expression type before the micro-expression changes; the second facial image Refers to the face image in the back-end sequence segment that is continuous with the first face image in the sequence of facial images, and all the second face images in the back-end sequence segment are of the second micro-expression type;

Inputting the target facial expression feature into a preset facial expression coding system, and determining the target facial expression category corresponding to the target facial expression feature;

If the target expression category belongs to the preset dangerous expression category, conduct a dialogue with the driver through a multi-round dialogue device, and obtain the dialogue information of the driver;

extracting the voiceprint features of the driver in the dialogue information, and determining whether the driver has fatigued driving according to the voiceprint features and a preset fatigue scale;

When it is determined that the driver is fatigued, a dangerous driving voice prompt is triggered according to the voiceprint feature and the sample expression.

A dangerous driving warning device, comprising:

A face image sequence recording module, configured to acquire the driver's face image in real time during the driving process of the vehicle, and record the acquired face image as a face image sequence according to the acquisition sequence;

The expression feature acquisition module is used to detect whether the facial image in the sequence of facial images has a micro-expression change, and when detecting the micro-expression change of the facial image, obtain the target expression of the facial image after the micro-expression change feature;

an expression category determination module, configured to input the target expression feature into a preset expression encoding system, and determine the target expression category corresponding to the target expression characteristic;

a dialogue information acquisition module, configured to conduct dialogue with the driver through a multi-round dialogue device if the target expression category belongs to the preset dangerous expression category, and acquire dialogue information of the driver;

A voiceprint feature matching module, configured to extract the voiceprint feature of the driver in the dialogue information, and determine whether the driver has fatigued driving according to the voiceprint feature and a preset fatigue scale;

The voice prompt module is used to trigger a dangerous driving voice prompt according to the voiceprint feature and the target expression category when it is determined that the driver is fatigued driving.

A computer device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, the processor implementing the following steps when executing the computer-readable instructions:

Detecting whether the facial images in the sequence of facial images have micro-expression changes, and when detecting that the facial images have micro-expression changes, acquiring the target expression features of the facial images after the micro-expression changes;

When it is determined that the driver is driving fatigued, a dangerous driving voice prompt is triggered according to the voiceprint feature and the target expression category.

One or more readable storage media storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps:

The above-mentioned dangerous driving warning method, device, computer equipment and storage medium, the method obtains the face image of the driver in real time during the driving process of the vehicle, and records the obtained face image as a face image sequence according to the acquisition sequence; Whether the facial images in the sequence of facial images have micro-expression changes, and when detecting that the facial images have micro-expression changes, obtain the target expression features of the face images after the micro-expression changes; the target expression features are Refers to the facial expression feature with the largest difference from the first facial image among all the second facial images; the first facial image refers to the first facial image of the first micro-expression type before the micro-expression changes; the second facial image is Refers to the face images in the back-end sequence segment that is continuous with the first face image in the sequence of facial images, and all the second face images in the back-end sequence segment are of the second micro-expression type; The target facial expression feature is input into the preset facial expression coding system, and the target facial expression category corresponding to the target facial expression feature is determined; if the target facial expression category belongs to the preset dangerous facial expression category, the multi-round dialogue device is activated to communicate with the driver. dialogue with the driver, and obtain the dialogue information of the driver; extract the voiceprint characteristics of the driver in the dialogue information, and determine whether the driver has fatigue driving according to the voiceprint characteristics and the preset fatigue scale; When it is determined that the driver is fatigued, a dangerous driving voice prompt is triggered according to the voiceprint feature and the target expression category.

The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below, and other features and advantages of the application will become apparent from the description, drawings, and claims.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application. , for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative labor.

1 is a schematic diagram of an application environment of a dangerous driving warning method in an embodiment of the present application;

2 is a flowchart of a dangerous driving warning method in an embodiment of the present application;

FIG. 3 is a flowchart of step S10 in the dangerous driving warning method in an embodiment of the present application;

4 is a flowchart of step S20 in the dangerous driving warning method in an embodiment of the present application;

5 is another flowchart of step S20 in the dangerous driving warning method in an embodiment of the present application;

6 is a flowchart of step S30 in the dangerous driving warning method in an embodiment of the present application;

7 is a schematic block diagram of a dangerous driving warning device in an embodiment of the present application;

8 is a schematic block diagram of a face image sequence recording module in a dangerous driving warning device according to an embodiment of the present application;

9 is a schematic block diagram of an expression feature acquisition module in a dangerous driving warning device according to an embodiment of the present application;

10 is another principle block diagram of the facial expression feature acquisition module in the dangerous driving warning device according to an embodiment of the present application;

11 is a schematic block diagram of an expression category determination module in a dangerous driving warning device according to an embodiment of the present application;

FIG. 12 is a schematic diagram of a computer device in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The dangerous driving early warning method provided by the embodiment of the present application can be applied in the application environment shown in FIG. 1 . Specifically, the dangerous driving early warning method is applied in a dangerous driving early warning system. The dangerous driving early warning system includes a client and a server as shown in FIG. 1 , and the client and the server communicate through the network, which is used to compare the accuracy of the dangerous driving early warning. low problem. Among them, the client, also known as the client, refers to the program corresponding to the server and providing local services for the client. Clients can be installed on, but not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented as an independent server or a server cluster composed of multiple servers.

In one embodiment, as shown in FIG. 2, a dangerous driving warning method is provided, and the method is applied to the server in FIG. 1 as an example for description, including the following steps:

S10: Acquire a face image of the driver in real time during the driving of the vehicle, and record the acquired face image as a sequence of face images in an associated order according to the acquisition sequence;

Understandably, a face image sequence refers to a collection of face images acquired within a period of time, and the sorting of the face images is associated with the acquisition time sequence, thereby forming a multi-frame face image sequence arranged in the acquisition time sequence. .

In one embodiment, as shown in FIG. 3 , step S10 includes:

S101: During the driving process of the vehicle, use a preset shooting device to shoot an image within a preset range;

It can be understood that, during the driving of the vehicle, the driver's face image can be obtained by photographing a photographing device installed in the vehicle. Exemplarily, the photographing device can be a camera, a mobile phone, or other device with a photographing and storage function. The preset range can be adjusted according to the driver's seat of different vehicles, the preset range is used to limit the driver's seat range, that is, the driver's face image is detected within the preset range, indicating that the driver is driving. During the process, there were no actions such as bending over, turning head, etc.

S102: When the preset photographing device captures the face image of the driver, record the obtained face image as a sequence of face images in an associated sequence according to the acquisition sequence;

It is understandable that when a face image including the driver is captured by the preset shooting device, it indicates that the driver is driving normally. At this time, there is no need to trigger a dangerous driving prompt, and then the obtained face image can be associated and recorded as a human in the order of acquisition. face image sequence.

S103: Trigger a dangerous driving prompt when the preset photographing device does not photograph a face image of the driver within a preset range, and stop the dangerous driving prompt when a face image including the driver is re-shot.

Understandably, when the preset photographing device does not photograph a face image including the driver, it indicates that the driver may not be driving normally at present. For example, for example, the driver is bent over to pick up something, and the photograph is taken within the preset range at this time. If the driver's face image is not available, or the driver's face image cannot be captured when the driver is looking down and playing with the mobile phone, the dangerous driving prompt will be triggered immediately, or when the vehicle has an automatic driving mode, it will automatically switch to Automatic driving mode, and stop the dangerous driving prompt when the driver's face image is re-captured. At this time, the previously captured face image of the driver can be deleted, so that the driver will not experience other conditions such as fatigue driving temporarily; the previously captured face image of the driver can also be retained to match the subsequent captured face image. Images are compared.

S20: Detecting whether the facial images in the sequence of facial images have micro-expression changes, and when detecting that the facial images have micro-expression changes, obtain the target expression features of the facial images after the micro-expression changes;

Wherein, the target facial expression feature refers to the facial expression feature with the greatest difference from the first facial image among all the second face images; it is understandable that the process of micro-expression changes is determined by frame-by-frame image, while In order to more accurately determine the expression category of the micro-expression, it is necessary to obtain the expression feature with the greatest difference from the first face image. Exemplarily, it is assumed that the first face image is a calm expression, which may be caused by the driver during driving. Fatigue driving causes the corresponding face image to change to a fatigued expression, and when the difference in expression is the largest, the expression features of the second face image can include drooping eyebrows (the eyebrows may be flush in a calm expression), tight eyes. closed (the degree of closure of the eyes can be determined by the distance between the upper eyelid and the lower eyelid, the distance between the upper eyelid and the lower eyelid in the eyes is larger when the expression is calm), and the drooping eyebrows and the closed eyes are the micro-expression changes The target expression features of the face image afterward. The first face image refers to the first face image of the first micro-expression type before the micro-expression changes; the second face image refers to the back-end sequence segment that is continuous with the first face image in the face image sequence face images in the back-end sequence segment, all second face images in the back-end sequence segment are of the second micro-expression type;

Understandably, in order to judge dangerous driving behaviors through face images, it is necessary to judge whether there is a micro-expression change between two adjacent frames of face images, and the micro-expression changes between two adjacent frames of face images. , indicating that the driver's mood or state changes at this time, and then the target expression features of the face image after the micro-expression change can be obtained.

Understandably, the back-end sequence segment refers to a sequence in the face image sequence, in which the micro-expression type has not changed temporarily, that is, all the second face images in the back-end sequence segment. Both are the second micro-expression type.

In one embodiment, as shown in FIG. 4 , in step S20, that is, the detecting whether the facial images in the sequence of facial images have micro-expression changes, including:

S201: Record the first frame of face image in the face image sequence as an initial face image, and perform pixel labeling on the initial face image to obtain an initial feature label corresponding to the initial face image;

S202: Record the next frame of face image corresponding to the initial face image in the face image sequence as a comparison face image, and perform pixel labeling on the comparison face image, and obtain a comparison with the comparison face image. The contrast feature annotation corresponding to the face image;

Understandably, for a face image, the face images corresponding to each micro-expression are different, such as the position of the eyebrows (such as the eyebrows are flush or the eyebrows are raised), etc. After the face image is associated and recorded as a face image sequence according to the acquisition sequence, the first frame of face image in the face image sequence is recorded as the initial face image, and the initial face image is marked with pixels, that is, The position information of each part (such as eyebrows, eyes, mouth, etc.) in the face image is annotated to determine the first feature annotation corresponding to the initial face image.

Similarly, after the initial face image is marked with pixels, the next frame of face image corresponding to the initial face image in the face image sequence is recorded as a contrasting face image, and the contrasting face image is recorded. The face image is labeled with pixels, and the contrast feature label corresponding to the contrast face image is obtained.

S203: Perform pixel feature comparison between the initial feature label and the comparison feature label, and determine a label difference value between the initial feature label and the comparison feature label;

It can be understood that, after performing pixel labeling on the initial face image, an initial feature label corresponding to the initial face image is obtained, and pixel labeling is performed on the comparison face image, and the comparison face image is obtained with the comparison face image. After the corresponding contrast feature is marked, compare the pixel features between the initial feature label and the contrast feature label, such as the position between the eyebrows, the degree of eye opening, etc., for example, compare the position of the eyebrows in the initial feature label with the contrast. Compare the position of the eyebrows in the feature annotation to determine the difference between the eyebrow positions. For example, the degree of eye opening in the initial feature annotation (such as recording the distance between the upper eyelid and the lower eyelid) is compared with the eye opening in the feature annotation. The difference between the degrees of eye opening is determined, and then the label difference value between the initial feature label and the comparison feature label is determined according to the feature difference value of each part information above.

S204: Compare the marked difference value with a preset difference threshold;

S205: When the marked difference value is greater than or equal to a preset difference threshold, prompt the facial images in the sequence of facial images to undergo micro-expression changes, and record the initial facial image as the first facial image image, and record the comparison face image and the face image after the comparison face image as the second face image.

The preset difference threshold can be determined according to actual needs. For example, when the driver is an older person, considering that his response is not so fast, the preset difference threshold can be set to a smaller value, such as 20%, 30% %Wait.

It can be understood that after comparing the marked difference value with the preset difference threshold, when the marked difference value is greater than or equal to the preset difference threshold, the micro-expressions in the comparison face image are characterized and compared with the micro-expressions in the initial face image. A major change occurs, and at this time, it is necessary to pay attention to situations where dangerous driving may occur. It is understandable that the driver's micro-expression is relatively calm and in a state of concentration during the initial driving. In the case of excessive driving time, the driver's micro-expression may change. Therefore, in this embodiment, the initial When the marked difference value between the face image and the comparison face image is greater than or equal to the preset difference threshold, the driver may have a change in the micro-expression, and the micro-expression may be one of the fatigue micro-expression types.

After comparing the marked difference value with the preset difference threshold, if the marked difference value is smaller than the preset difference threshold, it means that there is no large difference between the microexpressions in the comparison face image and the microexpressions in the initial face image Then, you can continue to compare other face images in the face image sequence, such as comparing the face image in the next frame of the comparison face image with the comparison face image.

Further, as shown in FIG. 5 , in step S20, the target expression feature of the face image after the micro-expression change is obtained, including:

S206: Perform pixel labeling on the first face image to obtain a first feature label corresponding to the first face image;

Specifically, after detecting whether the facial images in the sequence of facial images have micro-expression changes, and after detecting that the facial images have micro-expression changes, the first micro-expression type of the first micro-expression before the micro-expression changes Pixel labeling is performed on a face image, that is, the position information of each part (such as eyebrows, eyes, mouth, etc.) in the first face image is obtained, and a first feature label corresponding to the first face image is obtained.

S207: carry out pixel labeling to all the second human face images, and obtain the second feature labeling corresponding to each of the second human face images;

Specifically, after detecting whether a micro-expression change occurs in the face image in the sequence of face images, and after detecting the micro-expression change in the face image, pixel labeling is performed on the second face image of the second micro-expression type , that is, the position information of each part (such as eyebrows, eyes, mouth, etc.) in the second face image, understandably, in order to better distinguish the difference between the first face image and the second face image, Therefore, for the marked parts included in the first feature annotation of the first face image, the second feature annotation of the second face image also has corresponding marked parts, so that the second feature annotation corresponding to each second face image is obtained.

S208: Compare the first feature label with each of the second feature labels, and determine a label difference value between the first feature label and each of the second feature labels;

S209: Record the second feature label corresponding to the largest label difference value as the target expression feature.

Specifically, performing pixel labeling on the first face image to obtain a first feature label corresponding to the first face image, and performing pixel labeling on all the second face images to obtain corresponding After the second feature annotation corresponding to the second face image is described, the first feature annotation is compared with each second feature annotation to determine the annotation difference value between the first feature annotation and each second feature annotation, and the maximum value of the annotation difference is determined. The second feature label corresponding to the label difference value is recorded as the target expression feature. Understandably, in this embodiment, the second feature label corresponding to the largest label difference value is recorded as the target expression feature because the second feature label before this may not be able to more accurately judge the current state of the driver, and then After the second feature label corresponding to the largest label difference value is recorded as the target expression feature, the accuracy of the dangerous driving warning can be improved.

S30: Input the target facial expression feature into a preset facial expression coding system, and determine the target facial expression category corresponding to the target facial expression feature;

Among them, the encoding system of specific expressions under various micro-expressions is stored in the preset expression encoding system.

Specifically, after detecting whether a micro-expression change occurs in the facial image in the sequence of facial images, and when detecting a micro-expression change in the facial image, after acquiring the target expression feature of the facial image after the micro-expression change, The target expression features are input into the preset expression encoding system, so as to determine the target expression category corresponding to the target expression characteristics in the preset expression encoding system.

In one embodiment, before step S30, it further includes:

S01: obtain a plurality of muscle movement units obtained after the preset face image is divided into regions, and one described muscle movement unit is associated with an expression code;

Understandably, the preset face image may be an expressionless face image. For example, in the preset face image, the eyebrows, the mouth or the eyes are in a flush state, that is, the eyebrows are not raised and the eyes are not raised. closed etc. Further, the area division of the preset face image refers to dividing according to the parts that may have obvious changes in the face image to obtain a plurality of muscle movement units. Exemplarily, the muscle movement units may be mouth, eye , forehead muscles, etc. A muscle motor unit consists of one muscle or multiple muscles in the human face. The expression code is used to characterize the classification of the muscle movement units. For example, the mouth muscle movement unit is associated with an expression code as A; the eye muscle movement unit is associated with an expression code as B, and so on.

S02: Obtain a preset expression image set; the preset expression image set includes at least one micro-expression sample image; a micro-expression sample image is associated with an expression label;

Among them, in order to improve the accuracy of the data in the preset expression coding system, the micro-expression sample images in the preset expression image set are selected as many images as possible in the driving scene, so as to better reflect the corresponding micro-expressions in the driving scene. image features. The expression label indicates the meaning of the specific micro-expression in the micro-expression sample image. Exemplarily, the micro-expression in the micro-expression sample image is unhappy, and the corresponding expression label may be a sad expression label. Understandably, there is a micro-expression category in the A variety of different micro-expressions, that is, the same micro-expression category, may have different movement modes of the muscle motor units corresponding to the micro-expressions.

S03: After performing pixel labeling on the micro-expression sample image to obtain the sample image features corresponding to the micro-expression sample image, determine all expression movement units corresponding to the sample image features;

Among them, the expression movement unit refers to the existence of different muscle movement units between the micro-expression sample image and the preset face image. It is understandable that the micro-expression sample image is associated with an expression label, and the muscle movement unit between each micro-expression There are differences in the specific information (such as the position of the eyebrows, the radian of the mouth, etc.), so after the micro-expression sample image is pixel-labeled to obtain the sample image features corresponding to the micro-expression sample image, the sample image features are compared with the sample image. The preset image features corresponding to the preset face image (the preset image features can be obtained after pixel labeling of the preset face image) are compared, and the muscles corresponding to the different features between the sample image features and the preset image features are compared. Motor units were recorded as expression motor units.

S04: Categorize each of the expression movement units into the corresponding muscle movement units, and set an expression sub-code for each expression movement unit according to the expression code associated with its matching muscle movement unit, and assign the an emoticon code is associated with the emoticon code;

Specifically, after performing pixel labeling on the micro-expression sample image to obtain sample image features corresponding to the micro-expression sample image, all expression motion units corresponding to the sample image features are determined; Among the matched muscle movement units, for example, the expression movement unit is the raising of the eyebrows, and the expression movement unit is classified into the eyebrow muscle movement unit. In classifying each expression movement unit to the corresponding muscle movement unit, an expression code is set for each expression movement unit according to the expression code associated with the matching muscle movement unit, and the expression movement unit is assigned an expression code. The code is associated with the expression code; exemplarily, assuming that the expression code of the eyebrow muscle motor unit is A, the expression code of the raised eyebrow may be A1.

S05: record the expression label, the expression sub-code and the expression code association corresponding to the same micro-expression sample image as the code combination of the micro-expression sample image;

S06: Construct a preset expression encoding system according to the encoding combinations of the micro-expression sample images.

Specifically, classifying each expression movement unit into the matching muscle movement unit, and setting an expression sub-code for each expression movement unit according to the expression code associated with its matching muscle movement unit, and set the expression sub-code for each expression movement unit. After the expression code is associated with the expression code, the expression label corresponding to the same micro-expression sample image, the expression code and the expression code are associated and recorded as the code combination of the micro-expression sample image, exemplarily, can be The expression label, the expression sub-code and the expression code association are recorded as expression triples, and then a code combination of micro-expression sample images is formed, so as to construct a preset expression encoding system according to the code combination of each of the micro-expression sample images.

In one embodiment, as shown in FIG. 6 , in step S30, that is, inputting the target facial expression feature into the preset facial expression coding system to determine the target facial expression category corresponding to the target facial expression feature, including:

S301: Obtain each first motion unit corresponding to the first feature annotation, and each second motion unit corresponding to the target expression feature; the first feature annotation is obtained by performing pixel annotation on the first face image ;

S302: Record the second motion unit different from the first motion unit as the motion unit to be matched;

It can be understood that the first motion unit is related to the part marked on the first face image in the first feature annotation, and the second motion unit is related to the part marked on the second face image in the target expression feature.

Exemplarily, assuming that the first feature is labeled as the position of the eyebrow and the position of the mouth in the first face image, then the first feature label includes the eyebrow motion unit and the mouth motion unit; for the same reason, the target has been pointed out in the above description. The second feature label corresponding to the expression feature has the same label position as the first feature label, so the target expression feature also includes the eyebrow motion unit and the mouth motion unit. The eyebrow motion unit marked by the first feature may be the eyebrow flush, and the eyebrow motion unit of the target expression feature may be the eyebrow raised. Therefore, the eyebrow motion unit in the first motion unit is the eyebrow flush motion unit, and the second motion unit is the eyebrow flush motion unit. The middle eyebrow motor unit is the eyebrow raising motor unit.

Further, after acquiring each first motion unit corresponding to the first feature label and each second motion unit corresponding to the target facial expression feature, the second motion unit different from the first motion unit is The unit is recorded as the motion unit to be matched, that is, the eyebrow motion unit in the first motion unit is the eyebrow flush motion unit in the above description, and the eyebrow motion unit in the second motion unit is the eyebrow raising motion unit, then the first motion unit is the eyebrow raising motion unit. The motion unit different from the second motion unit is the eyebrow motion unit.

Further, it is also possible to record the muscle motion unit corresponding to the facial expression feature of the difference as the motion unit to be matched by determining the facial expression feature that is different between the first feature label and the target facial expression feature.

S303: determine the muscle movement unit matched with the to-be-matched movement unit, and obtain the expression code corresponding to the muscle movement unit matched with it from the preset expression encoding system;

Specifically, after recording the second motion unit different from the first motion unit as the motion unit to be matched, determine the muscle motion unit that matches the motion unit to be matched, and acquire the expression corresponding to the muscle motion unit coding. Exemplarily, assuming that the motion unit to be matched is an eyebrow motion unit, the expression code corresponding to the eyebrow motion unit is obtained from a preset expression encoding system.

S304: from the expression encoding, determine the expression sub-encoding corresponding to the to-be-matched motion unit;

Further, after determining the muscle movement unit matched with the to-be-matched movement unit, and obtaining the expression code corresponding to the muscle movement unit matched with it from the preset expression coding system, determine the expression sub-code corresponding to the to-be-matched movement unit. , exemplarily, assuming that the motion unit to be matched is an eyebrow raising motion unit in the eyebrow motion unit, then from the eyebrow expression encoding, the expression sub-code corresponding to the eyebrow raising is determined.

S305: Determine a target expression category corresponding to the target expression feature according to the determined expression code and the expression sub-code.

It can be understood that, after determining the expression code and the expression sub-coding corresponding to the motion unit to be matched, since the expression label, the expression sub-coding and the expression encoding corresponding to the same micro-expression sample image are stored in the preset expression encoding system, It is recorded as the code combination of the micro-expression sample images, and then the target expression category corresponding to the target expression feature is determined according to the expression code and the expression sub-code. Further, a micro-expression may be composed of a plurality of different expression sub-codes, and then the target expression category can be determined according to the expression code and the expression sub-code corresponding to each muscle motion unit to be matched.

Exemplarily, assuming that in the fatigue expression category, the information of the corresponding parts is drooping eyebrows, closed eyes, etc., then the corresponding expression encoding is the expression encoding corresponding to the eyebrow motion unit, and the expression encoding corresponding to the eye motion unit, The corresponding expression sub-coding includes the expression sub-coding corresponding to drooping eyebrows, and the expression sub-coding corresponding to closed eyes, and then according to the above-mentioned expression encoding and the expression sub-coding, it is determined that the target expression category corresponding to the target expression feature is the fatigue expression category. .

S40: If the target expression category belongs to the preset dangerous expression category, conduct a dialogue with the driver through a multi-round dialogue device, and obtain the dialogue information of the driver;

The preset dangerous expression category may be a fatigue expression category. The multi-round dialogue device can be set in the intelligent voice system on the vehicle, and the multi-round dialogue device can communicate with the driver through the TTS broadcast technology, so as to improve the driver's spirit.

Specifically, after inputting the target facial expression feature into the preset facial expression coding system, and after determining the target facial expression category corresponding to the target facial expression feature, it is determined whether the target facial expression category is a preset dangerous facial expression category, so as to determine whether the target facial expression category is a preset dangerous facial expression category. When it belongs to the preset dangerous expression category, start the multi-round dialogue device, and ask the driver's current status through the multi-round dialogue device, or broadcast some interesting messages to the driver, and then have a dialogue with the driver and obtain the driver's dialogue information.

S50: Extract the voiceprint feature of the driver in the dialogue information, and determine whether the driver is fatigued according to the voiceprint feature and a preset fatigue scale;

It can be understood that the preset fatigue scale is generated according to the characteristics of the voice in the dialogue after learning the driver's voice in various states in advance through the multi-round dialogue device. Tests, such as extracting the voiceprint features of the driver during normal driving, encoding the voiceprint features and labeling the normal driving voiceprint, or extracting the voiceprint features of the driver when they are initially fatigued, and encoding the voiceprint features Code and label the initial fatigue voiceprint, and then construct a preset fatigue scale according to the voiceprint features and corresponding labels in different driving periods.

Understandably, there are levels corresponding to each fatigue level in the preset fatigue scale, and sample voiceprint features corresponding to the levels, and after extracting the driver's voiceprint features in the dialogue information, you can use the voiceprint features and the corresponding voiceprint features. The sample voiceprint features are matched, such as level adjustment and alignment of the voiceprint features and the sample voiceprint features, and the frequency characteristics of the voiceprint features and the sample voiceprint features are simulated by IRS filtering, so as to compare the voiceprint features and the sample voiceprint features. After compensating the frequency characteristics of the voiceprint feature, the similarity between the voiceprint feature and the sample voiceprint feature is determined by the asymmetric processing algorithm, and then the sample voiceprint feature with the highest similarity is selected as the basis for judging the voiceprint feature, so as to automatically The fatigue level corresponding to the voiceprint feature of the sample with the highest similarity is determined in the preset fatigue measurement table, so as to determine the current fatigue level of the driver, so as to determine whether the driver has a fatigue driving phenomenon.

S60: Trigger a dangerous driving voice prompt according to the voiceprint feature and the target expression category when it is determined that the driver is fatigued.

Understandably, when it is determined that the driver is driving fatigued, the current driver's fatigue level (such as mild fatigue, severe fatigue, etc.) can be determined according to the voiceprint feature or the target expression category, for example, according to the voiceprint feature and preset fatigue. When the metric table determines that the driver is driving fatigued, the current fatigue level of the driver can be determined according to the fatigue level corresponding to the voiceprint feature, or when the target expression category is determined according to the target expression feature, due to the micro-expressions corresponding to different fatigue levels. The expression characteristics of the two are also different, and then the specific fatigue level expressions can be obtained when determining the target expression category (for example, different fatigue levels are defined according to the distance range between the upper eyelid and the lower eyelid), and then the driver is determined to be fatigued. When driving, a dangerous driving voice prompt can be triggered according to the voiceprint feature and the target expression category. It can conduct continuous voice chat with the driver, or broadcast voice prompts for dangerous driving such as light-hearted talk shows; if it is heavily fatigued, it can broadcast through a louder voice prompt and switch when the driver is driving an autonomous car to automatic driving.

Further, the fatigue reminder strategy can be adjusted according to the fatigue frequency of the driver every time (such as frequent fatigue, general fatigue), usually in a certain period of time (for example, 10:00 to 12:00 in the evening). For example, drivers who are often tired can Reminder in advance, automatically give voice reminders when the driver is not fatigued (the frequency of reminders for frequent fatigue is higher than that of general fatigue); or play reminder information or music at the time when the driver is prone to fatigue (10 o'clock in the evening), and then you can By constantly interacting with the driver, the driver can be prevented from falling into a deep sleep, reducing the accident rate.

In this application, the use of intelligent expression technology and voice analysis can more sensitively and accurately capture the driver's fatigue state. Once the situation is found, multiple rounds of dialogue technology are used to remind, which improves the accuracy of dangerous driving warning, and in When it is found that there is fatigue driving in the face image, it is not prompted immediately, which alleviates the suddenness caused by the immediate triggering of the warning prompt.

It should be understood that the size of the sequence numbers of the steps in the above embodiments does not mean the sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

In one embodiment, a dangerous driving early warning device is provided, and the dangerous driving early warning device is in one-to-one correspondence with the dangerous driving early warning method in the above-mentioned embodiment. As shown in FIG. 7 , the dangerous driving warning device includes a face image sequence recording module 10 , an expression feature acquisition module 20 , an expression category determination module 30 , a dialogue information acquisition module 40 , a voiceprint feature matching module 50 and a voice prompt module 60 . The detailed description of each functional module is as follows:

The face image sequence recording module 10 is used to acquire the driver's face image in real time during the driving process of the vehicle, and record the acquired face image as a face image sequence according to the acquisition sequence;

The facial expression feature acquisition module 20 is used to detect whether the facial image in the sequence of facial images has a micro-expression change, and when detecting that the facial image has a micro-expression change, obtain the target of the facial image after the micro-expression change facial features;

An expression category determination module 30, configured to input the target expression feature into a preset expression encoding system, and determine the target expression category corresponding to the target expression characteristic;

Dialogue information acquisition module 40, configured to conduct dialogue with the driver through a multi-round dialogue device and acquire dialogue information of the driver if the target expression category belongs to the preset dangerous expression category;

A voiceprint feature matching module 50, configured to extract the voiceprint feature of the driver in the dialogue information, and determine whether the driver has fatigued driving according to the voiceprint feature and a preset fatigue scale;

The voice prompt module 60 is configured to trigger a dangerous driving voice prompt according to the voiceprint feature and the target expression category when it is determined that the driver is fatigued driving.

Preferably, as shown in FIG. 8 , the face image sequence recording module 10 includes the following units:

The image capturing unit 101 is used for capturing images within a preset range through a preset capturing device during the driving of the vehicle;

The face image sequence recording unit 102 is configured to associate and record the obtained face images as a face image sequence according to the acquisition order when the preset photographing device captures the driver's face images;

The danger prompting unit 103 is used for triggering a dangerous driving prompt when the preset photographing device fails to photograph the driver's face image within a preset range, and stops when the driver's face image is re-photographed Dangerous driving tips.

Preferably, as shown in FIG. 9 , the expression feature acquisition module 20 includes:

The first pixel labeling unit 201 is configured to record the first frame of the face image in the sequence of face images as an initial face image, and perform pixel labeling on the initial face image to obtain the same value as the initial face image. The corresponding initial feature annotation;

The second pixel labeling unit 202 is configured to record the next frame of face image corresponding to the initial face image in the sequence of face images as a comparison face image, and perform pixel labeling on the comparison face image , obtain the contrast feature annotation corresponding to the contrast face image;

Annotation difference value determination unit 203, configured to perform pixel feature comparison between the initial feature label and the comparison feature label, and determine the label difference value between the initial feature label and the comparison feature label;

a difference comparison unit 204, configured to compare the marked difference value with a preset difference threshold;

The second face image recording unit 205 is configured to, when the marked difference value is greater than or equal to a preset difference threshold, prompt the face images in the face image sequence to have micro-expression changes, and record the initial face image The image is recorded as the first face image, and the comparison face image and the face images ranked after the comparison face image are associated and recorded as the second face image.

Preferably, as shown in FIG. 10 , the expression feature acquisition module 20 further includes:

A third pixel labeling unit 206, configured to perform pixel labeling on the first face image to obtain a first feature label corresponding to the first face image;

a fourth pixel labeling unit 207, configured to perform pixel labeling on all the second face images to obtain second feature labels corresponding to each of the second face images;

A feature label comparison unit 208, configured to compare the first feature label with each of the second feature labels, and determine a label difference value between the first feature label and each of the second feature labels;

The target facial expression feature determining unit 209 is configured to record the second feature label corresponding to the largest label difference value as the target facial expression feature.

Preferably, the dangerous driving warning device further includes:

The muscle movement unit acquisition module 01 is used to obtain a plurality of muscle movement units obtained after the preset face image is divided into regions, and one of the muscle movement units is associated with an expression code;

An expression image set obtaining module 02, configured to obtain a preset expression image set; the preset expression image set includes at least one micro-expression sample image; a micro-expression sample image is associated with an expression label;

Expression motion unit determination module 03, used to perform pixel labeling on the micro-expression sample image to obtain sample image features corresponding to the micro-expression sample image, and determine all the expression motion units corresponding to the sample image features;

Expression sub-coding setting module 04, for classifying each described expression movement unit into the described muscle movement unit matched with it, and set an expression for each expression movement unit according to the expression code associated with its matching muscle movement unit subcoding, and associate the expression subcoding with the expression encoding;

The coding combination recording module 05 is used to associate and record the expression label, the expression sub-coding and the expression coding associated with the same micro-expression sample image as the coding combination of the micro-expression sample image;

Expression coding system building module 06, for constructing a preset expression coding system according to the coding combination of each described micro-expression sample image.

Preferably, as shown in FIG. 11 , the expression category determination module 30 includes:

The motion unit acquisition unit 301 is configured to acquire each first motion unit corresponding to the first feature label, and each second motion unit corresponding to the target expression feature; the first feature label is obtained by identifying the first person The face image is obtained by pixel labeling;

A motion unit recording unit 302 to be matched, configured to record the second motion unit different from the first motion unit as the motion unit to be matched;

The expression code obtaining unit 303 is used to determine the muscle movement unit matched with the described movement unit to be matched, and obtain the expression code of the muscle movement unit matched with it from the preset expression code system;

An expression code obtaining unit 304, configured to determine the expression code corresponding to the motion unit to be matched from the expression encoding;

The target expression category determination unit 305 is configured to determine a target expression category corresponding to the target expression feature according to the determined expression code and the expression sub-code.

For the specific limitation of the dangerous driving warning device, reference may be made to the above limitation on the dangerous driving warning method, which will not be repeated here. Each module in the above-mentioned dangerous driving warning device can be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, a computer device is provided, and the computer device may be a server, and its internal structure diagram may be as shown in FIG. 12 . The computer device includes a processor, memory, a network interface, and a database connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium, an internal memory. The readable storage medium stores an operating system, computer readable instructions and a database. The internal memory provides an environment for the execution of the operating system and computer-readable instructions in the readable storage medium. The database of the computer device is used to store the data used in the dangerous driving warning method in the above embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer readable instructions, when executed by a processor, implement a dangerous driving warning method.

In one embodiment, there is provided a computer apparatus comprising a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, the processor executing the computer readable instructions Implement the following steps when instructing:

In one embodiment, one or more readable storage media are provided that store computer-readable instructions that, when executed by one or more processors, cause the one or more processors to execute Follow the steps below:

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a non-volatile computer. In a readable storage medium or a volatile computer-readable storage medium, the computer-readable instructions, when executed, may include the processes of the foregoing method embodiments. Wherein, any reference to memory, storage, database or other medium used in the various embodiments provided in this application may include non-volatile and/or volatile memory. Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous chain Road (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated to different functional units, Module completion, that is, dividing the internal structure of the device into different functional units or modules to complete all or part of the functions described above.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it can still be used for the above-mentioned implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be included in the within the scope of protection of this application.

Claims

A dangerous driving warning method, comprising:

Acquire the face image of the driver in real time during the driving of the vehicle, and associate and record the acquired face image as a sequence of face images according to the acquisition sequence;

Detecting whether the facial images in the sequence of facial images have micro-expression changes, and when detecting that the facial images have micro-expression changes, acquiring the target expression features of the facial images after the micro-expression changes;

Inputting the target facial expression feature into a preset facial expression coding system, and determining the target facial expression category corresponding to the target facial expression feature;

If the target expression category belongs to the preset dangerous expression category, conduct a dialogue with the driver through a multi-round dialogue device, and obtain the dialogue information of the driver;

extracting the voiceprint features of the driver in the dialogue information, and determining whether the driver has fatigued driving according to the voiceprint features and a preset fatigue scale;

When it is determined that the driver is driving fatigued, a dangerous driving voice prompt is triggered according to the voiceprint feature and the target expression category.
The method for early warning of dangerous driving according to claim 1, wherein, acquiring the face image of the driver in real time during the driving of the vehicle, and recording the acquired face image as a sequence of face images in an order of acquisition, including:

During the driving process of the vehicle, the image within the preset range is shot by the preset shooting device;

When the preset photographing device captures the face image of the driver, the obtained face image is associated and recorded as a sequence of face images according to the acquisition sequence;

When the preset photographing device fails to photograph the driver's face image within the preset range, the dangerous driving prompt is triggered, and when the driver's face image is re-shot, the dangerous driving prompt is stopped.
The dangerous driving warning method according to claim 1, wherein the detecting whether the facial images in the sequence of facial images have micro-expression changes, comprising:

recording the first frame of face image in the face image sequence as an initial face image, and performing pixel labeling on the initial face image to obtain an initial feature label corresponding to the initial face image;

Recording the next frame of face image corresponding to the initial face image in the face image sequence as a comparison face image, and performing pixel labeling on the comparison face image to obtain the comparison face image with the comparison face image Corresponding contrast feature annotation;

performing pixel feature comparison between the initial feature annotation and the comparison feature annotation, and determining the annotation difference value between the initial feature annotation and the comparison feature annotation;

comparing the marked difference value with a preset difference threshold;

When the marked difference value is greater than or equal to the preset difference threshold, prompt the facial images in the sequence of facial images to have micro-expression changes, record the initial facial image as the first facial image, and record all the facial images as the first facial image. The above-mentioned comparison face images and the face images ranked after the comparison face images are associated and recorded as the second face image.
The dangerous driving warning method according to claim 1, wherein the target facial expression feature refers to the facial expression feature with the largest difference from the first facial image in all the second facial images; the first facial image refers to the change of micro-expression The first face image of the first micro-expression type before; the second face image refers to the face image in the back-end sequence segment that is continuous with the first face image in the face image sequence, and the back-end All the second face images in the sequence segment are of the second micro-expression type; the acquisition of the target expression features of the face images after the micro-expression changes includes:

performing pixel labeling on the first face image to obtain a first feature label corresponding to the first face image;

Perform pixel labeling on all the second face images to obtain second feature labels corresponding to each of the second face images;

Comparing the first feature label with each of the second feature labels, and determining a label difference value between the first feature label and each of the second feature labels;

Record the second feature label corresponding to the largest label difference value as the target expression feature.
The dangerous driving warning method according to claim 1, wherein, before the inputting the target facial expression feature into the preset facial expression coding system, the method further comprises:

Obtain multiple muscle movement units obtained after the preset face image is divided into regions, and each of the muscle movement units is associated with an expression code;

Obtaining a preset expression image set; the preset expression image set includes at least one micro-expression sample image; a micro-expression sample image is associated with an expression label;

After pixel labeling is performed on the micro-expression sample image to obtain the sample image features corresponding to the micro-expression sample image, all expression motion units corresponding to the sample image features are determined;

Each described expression movement unit is classified into the described muscle movement unit that it is matched, and according to the expression code associated with its matching muscle movement unit, an expression code is set for each expression movement unit, and the expression code is set. encoding is associated with the expression encoding;

Correspondingly record the expression label, the expression sub-code and the expression code corresponding to the same micro-expression sample image as the coding combination of the micro-expression sample image;

A preset expression encoding system is constructed according to the encoding combinations of the micro-expression sample images.
The dangerous driving warning method according to claim 5, wherein the inputting the target facial expression feature into a preset facial expression coding system to determine the target facial expression category corresponding to the target facial expression feature comprises:

Obtain each first motion unit corresponding to the first feature label, and each second motion unit corresponding to the target expression feature; the first feature label is obtained by performing pixel labeling on the first face image;

recording the second motion unit different from the first motion unit as the motion unit to be matched;

Determine the muscle movement unit matched with the described movement unit to be matched, and obtain the expression code of the muscle movement unit matched with it from the preset expression coding system;

From the expression encoding, determine the expression sub encoding corresponding to the motion unit to be matched;

According to the determined expression code and the expression sub-code, a target expression category corresponding to the target expression feature is determined.
A dangerous driving warning device, comprising:

A face image sequence recording module, configured to acquire the driver's face image in real time during the driving process of the vehicle, and record the acquired face image as a face image sequence according to the acquisition sequence;

An expression feature acquisition module, configured to detect whether a micro-expression change occurs in the face image in the sequence of face images, and when detecting a micro-expression change in the face image, acquire the target expression of the face image after the micro-expression change feature;

an expression category determination module, configured to input the target expression feature into a preset expression encoding system, and determine the target expression category corresponding to the target expression characteristic;

a dialogue information acquisition module, configured to conduct dialogue with the driver through a multi-round dialogue device if the target expression category belongs to the preset dangerous expression category, and acquire dialogue information of the driver;

A voiceprint feature matching module, configured to extract the voiceprint feature of the driver in the dialogue information, and determine whether the driver has fatigued driving according to the voiceprint feature and a preset fatigue scale;

The voice prompt module is used to trigger a dangerous driving voice prompt according to the voiceprint feature and the target expression category when it is determined that the driver is fatigued driving.
The dangerous driving warning device according to claim 7, wherein the face image sequence recording module comprises:

an image capturing unit, used for capturing an image within a preset range through a preset shooting device during the driving process of the vehicle;

A face image sequence recording unit, configured to associate and record the obtained face images as a face image sequence according to the acquisition order when the preset photographing device captures the driver's face images;

A danger prompting unit, used for triggering a dangerous driving prompt when the preset photographing device fails to photograph the driver's face image within a preset range, and stops the danger when the driver's face image is re-photographed Driving Tips.
A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer-readable instructions:

Acquire the face image of the driver in real time during the driving of the vehicle, and record the acquired face image as a sequence of face images according to the acquisition sequence;

Detecting whether the facial images in the sequence of facial images have micro-expression changes, and when detecting that the facial images have micro-expression changes, acquiring the target expression features of the facial images after the micro-expression changes;

Inputting the target facial expression feature into a preset facial expression coding system, and determining the target facial expression category corresponding to the target facial expression feature;

If the target expression category belongs to the preset dangerous expression category, conduct a dialogue with the driver through a multi-round dialogue device, and obtain the dialogue information of the driver;

extracting the voiceprint features of the driver in the dialogue information, and determining whether the driver has fatigued driving according to the voiceprint features and a preset fatigue scale;

When it is determined that the driver is fatigued, a dangerous driving voice prompt is triggered according to the voiceprint feature and the target expression category.
The computer device according to claim 9, wherein, acquiring the face image of the driver in real time during the driving of the vehicle, and recording the acquired face image as a sequence of face images in an order of acquisition, including:

During the driving process of the vehicle, the image within the preset range is shot by the preset shooting device;

When the preset photographing device captures the face image of the driver, the obtained face image is associated and recorded as a sequence of face images according to the acquisition sequence;

When the preset photographing device fails to photograph the driver's face image within the preset range, the dangerous driving prompt is triggered, and when the driver's face image is re-shot, the dangerous driving prompt is stopped.
The computer device according to claim 9, wherein the detecting whether the facial images in the sequence of facial images have micro-expression changes, comprising:

recording the first frame of face image in the face image sequence as an initial face image, and performing pixel labeling on the initial face image to obtain an initial feature label corresponding to the initial face image;

Recording the next frame of face image corresponding to the initial face image in the face image sequence as a comparison face image, and performing pixel labeling on the comparison face image to obtain the comparison face image with the comparison face image Corresponding contrast feature annotation;

performing pixel feature comparison between the initial feature annotation and the comparison feature annotation, and determining the annotation difference value between the initial feature annotation and the comparison feature annotation;

comparing the marked difference value with a preset difference threshold;

When the marked difference value is greater than or equal to the preset difference threshold, prompt the facial images in the sequence of facial images to have micro-expression changes, record the initial facial image as the first facial image, and record all the facial images as the first facial image. The above-mentioned comparison face images and the face images ranked after the comparison face images are associated and recorded as the second face image.
The computer device according to claim 9, wherein the target facial expression feature refers to the facial expression feature with the greatest difference from the first facial image among all the second facial images; The first face image of the first micro-expression type; the second face image refers to the face image in the back-end sequence segment that is continuous with the first face image in the sequence of face images, and the back-end sequence segment All the second face images in are the second micro-expression type; the target expression features of the obtained face images after the micro-expression change include:

performing pixel labeling on the first face image to obtain a first feature label corresponding to the first face image;

Perform pixel annotation on all the second face images to obtain second feature annotations corresponding to each of the second face images;

Comparing the first feature label with each of the second feature labels, and determining a label difference value between the first feature label and each of the second feature labels;

Record the second feature label corresponding to the largest label difference value as the target expression feature.
The computer device according to claim 9, wherein before the input of the target facial expression feature into the preset facial expression coding system, the processor further implements the following steps when executing the computer-readable instruction:

Obtain multiple muscle movement units obtained after the preset face image is divided into regions, and each of the muscle movement units is associated with an expression code;

Obtaining a preset expression image set; the preset expression image set includes at least one micro-expression sample image; a micro-expression sample image is associated with an expression label;

After pixel labeling is performed on the micro-expression sample image to obtain the sample image features corresponding to the micro-expression sample image, all expression motion units corresponding to the sample image features are determined;

Each described expression movement unit is classified into the described muscle movement unit matched with it, and according to the expression code associated with its matching muscle movement unit, an expression sub-code is set for each expression movement unit, and the expression sub-code is set. encoding is associated with the expression encoding;

Correspondingly record the expression label, the expression sub-code and the expression code corresponding to the same micro-expression sample image as the coding combination of the micro-expression sample image;

A preset expression encoding system is constructed according to the encoding combinations of the micro-expression sample images.
The computer device according to claim 13, wherein the inputting the target facial expression feature into a preset facial expression coding system to determine the target facial expression category corresponding to the target facial expression feature comprises:

Obtain each first motion unit corresponding to the first feature label, and each second motion unit corresponding to the target expression feature; the first feature label is obtained by performing pixel labeling on the first face image;

recording the second motion unit different from the first motion unit as the motion unit to be matched;

Determine the muscle movement unit matched with the described movement unit to be matched, and obtain the expression code of the muscle movement unit matched with it from the preset expression coding system;

From the expression encoding, determine the expression sub encoding corresponding to the motion unit to be matched;

According to the determined expression code and the expression sub-code, a target expression category corresponding to the target expression feature is determined.
One or more readable storage media storing computer-readable instructions, wherein the computer-readable instructions, when executed by one or more processors, cause the one or more processors to perform the following steps:

Acquire the face image of the driver in real time during the driving of the vehicle, and associate and record the acquired face image as a sequence of face images according to the acquisition sequence;

Detecting whether the facial images in the sequence of facial images have micro-expression changes, and when detecting that the facial images have micro-expression changes, acquiring the target expression features of the facial images after the micro-expression changes;

Inputting the target facial expression feature into a preset facial expression coding system, and determining the target facial expression category corresponding to the target facial expression feature;

If the target expression category belongs to the preset dangerous expression category, conduct a dialogue with the driver through a multi-round dialogue device, and obtain the dialogue information of the driver;

extracting the voiceprint features of the driver in the dialogue information, and determining whether the driver has fatigued driving according to the voiceprint features and a preset fatigue scale;

When it is determined that the driver is driving fatigued, a dangerous driving voice prompt is triggered according to the voiceprint feature and the target expression category.
The readable storage medium according to claim 15 , wherein the acquiring the face image of the driver in real time during the driving of the vehicle, and recording the acquired face images as a sequence of face images in an order of acquisition, comprises:

During the driving process of the vehicle, the image within the preset range is shot by the preset shooting device;

When the preset photographing device captures the face image of the driver, the obtained face image is associated and recorded as a sequence of face images according to the acquisition sequence;

When the preset photographing device fails to photograph the driver's face image within the preset range, the dangerous driving prompt is triggered, and when the driver's face image is re-shot, the dangerous driving prompt is stopped.
The readable storage medium according to claim 15, wherein the detecting whether the facial images in the sequence of facial images have micro-expression changes, comprising:

recording the first frame of face image in the face image sequence as an initial face image, and performing pixel labeling on the initial face image to obtain an initial feature label corresponding to the initial face image;

Recording the next frame of face image corresponding to the initial face image in the face image sequence as a comparison face image, and performing pixel labeling on the comparison face image to obtain the comparison face image with the comparison face image Corresponding contrast feature annotation;

performing pixel feature comparison between the initial feature annotation and the comparison feature annotation, and determining the annotation difference value between the initial feature annotation and the comparison feature annotation;

comparing the marked difference value with a preset difference threshold;

When the marked difference value is greater than or equal to the preset difference threshold, prompt the facial images in the sequence of facial images to have micro-expression changes, record the initial facial image as the first facial image, and record all the facial images as the first facial image. The above-mentioned comparison face images and the face images ranked after the comparison face images are associated and recorded as the second face image.
The readable storage medium according to claim 15, wherein the target facial expression feature refers to the facial expression feature with the largest difference from the first facial image in all the second facial images; the first facial image refers to the change of micro-expression The first face image of the first micro-expression type before; the second face image refers to the face image in the back-end sequence segment that is continuous with the first face image in the face image sequence, and the back-end All the second face images in the sequence segment are of the second micro-expression type; the acquisition of the target expression features of the face images after the micro-expression changes includes:

performing pixel labeling on the first face image to obtain a first feature label corresponding to the first face image;

Perform pixel labeling on all the second face images to obtain second feature labels corresponding to each of the second face images;

Comparing the first feature label with each of the second feature labels, and determining a label difference value between the first feature label and each of the second feature labels;

Record the second feature label corresponding to the largest label difference value as the target expression feature.
16. The readable storage medium of claim 15, wherein, before the input of the target expression feature into the preset expression encoding system, the computer-readable instructions, when executed by one or more processors, cause the The one or more processors also perform the following steps:

Obtain multiple muscle movement units obtained after the preset face image is divided into regions, and each of the muscle movement units is associated with an expression code;

Obtaining a preset expression image set; the preset expression image set includes at least one micro-expression sample image; a micro-expression sample image is associated with an expression label;

After pixel labeling is performed on the micro-expression sample image to obtain the sample image features corresponding to the micro-expression sample image, all expression motion units corresponding to the sample image features are determined;

Each described expression movement unit is classified into the described muscle movement unit matched with it, and according to the expression code associated with its matching muscle movement unit, an expression sub-code is set for each expression movement unit, and the expression sub-code is set. encoding is associated with the expression encoding;

Correspondingly record the expression label, the expression sub-code and the expression code corresponding to the same micro-expression sample image as the coding combination of the micro-expression sample image;

A preset expression encoding system is constructed according to the encoding combination of each of the micro-expression sample images.
The readable storage medium according to claim 19, wherein the inputting the target facial expression feature into a preset facial expression coding system to determine the target facial expression category corresponding to the target facial expression feature comprises:

Obtain each first motion unit corresponding to the first feature label, and each second motion unit corresponding to the target expression feature; the first feature label is obtained by performing pixel labeling on the first face image;

recording the second motion unit different from the first motion unit as the motion unit to be matched;

Determine the muscle movement unit matched with the described movement unit to be matched, and obtain the expression code of the muscle movement unit matched with it from the preset expression coding system;

From the expression encoding, determine the expression sub encoding corresponding to the motion unit to be matched;

According to the determined expression code and the expression sub-code, a target expression category corresponding to the target expression feature is determined.