TWI739675B - Image recognition method and apparatus - Google Patents

Abstract

An image recognition and apparatus are provided. After an image sequence is obtained, a behavior recognition model is used to analyze the image sequence to obtain a behavior posture of a user, and a gaze recognition model is used to analyze the image sequence to obtain a sight line feature of the user. Afterwards, a first score corresponding to the behavior posture is calculated, and a second score corresponding to the sight line feature is calculated. A risk score is calculated based on the first score and the second score and a corresponding warning signal is issued based on the risk score.

Description

Image recognition method and device

The present invention relates to an image processing method and device, and more particularly to an image recognition method and device.

With the development of transportation, how to improve safety is the direction of all car manufacturers' continuous efforts and diligence. In the technical application of smart cars, safety considerations become the first element. In order to effectively improve driving safety and further prevent accidents, the application and technology of smart cars are getting more and more attention. Accordingly, how to effectively and instantly monitor driving behavior through the assistance of smart cars, thereby reducing driving accidents caused by human negligence, is the focus of current consideration.

The invention provides an image recognition method and device, which can improve the recognition accuracy.

The image recognition method of the present invention includes: obtaining an image sequence; analyzing the image sequence using a behavior recognition model to obtain the user's behavior posture; calculating the first score corresponding to the behavior posture; using the sight recognition model to analyze the image sequence to obtain the use Calculate the second score corresponding to the sight feature; calculate the risk score based on the first score and the second score; and issue a corresponding warning signal based on the risk score.

In an embodiment of the present invention, after obtaining the behavior posture of the user, it further includes: in the case of determining that the behavior posture is a distracting behavior, calculating the first score corresponding to the behavior posture includes: setting the first score to The main score corresponding to the distracted behavior; and in the process of continuously determining that the behavior posture is the distracted behavior, each time the first judgment time passes, the first score is added to the auxiliary score corresponding to the main score.

In an embodiment of the present invention, after obtaining the behavior posture of the user, it further includes: in the case of determining that the behavior posture is a fatigue behavior, setting the first score as the first preset score corresponding to the fatigue level of 1; In the case where the fatigue level is 1, after the first preset time has elapsed, and in the case where it is determined that the behavior posture is still a fatigue behavior, the first score is set as the second preset score corresponding to the fatigue level 2, wherein the second prediction Set the score to be greater than the first preset score; and in the case of a fatigue level of 2, after the second preset time has elapsed, and in the case that the behavior posture is still a fatigue behavior, set the first score to correspond to the fatigue level of 3. The third preset score, where the third preset score is greater than the second preset score.

In an embodiment of the present invention, after obtaining the user's line of sight feature, it further includes: judging whether the user is not facing the designated direction based on the line of sight feature; and calculating the second score when it is determined that the user is not facing the designated direction. The step of calculating the second score includes: adding a designated score to the second score every time the second judgment time has elapsed.

In an embodiment of the present invention, after obtaining the user's line of sight feature, it further includes: judging whether the line of sight feature meets the line of sight fatigue feature based on the line of sight feature; when determining that the line of sight feature meets the line of sight fatigue feature, setting the second score to correspond to the fatigue level The first preset score is 1; after the first preset time has elapsed when the fatigue level is 1, when it is determined that the line of sight feature still meets the line of sight fatigue feature, the second score is set to correspond to the second score of fatigue level 2. A preset score, where the second preset score is greater than the first preset score; and when the second preset time has elapsed when the fatigue level is 2, when it is determined that the line of sight feature still meets the line of sight fatigue feature, the second score is set to Corresponds to a third preset score with a fatigue level of 3, where the third preset score is greater than the second preset score.

In an embodiment of the present invention, the above-mentioned image recognition method further includes: resetting the first score to zero when the behavior posture is detected to return to a normal state; Two points are returned to zero.

In an embodiment of the present invention, after the corresponding warning signal is issued according to the risk score, it further includes: continuously issuing the warning signal until the detected behavior posture returns to the normal state and the detected line of sight direction returns to the specified direction.

In an embodiment of the present invention, the step of issuing the corresponding warning signal according to the risk score includes: judging based on the risk score that the current risk level is the first warning level, the second warning level, the third warning level, or the fourth warning level. Warning level; when the current danger level is determined to be the first warning level, a flashing light signal and short sound are issued; when the current danger level is determined to be the second warning level, a flashing light signal is issued and the first long sound is continuously emitted; when the current danger level is determined The level is the third warning level, which emits a flashing signal and the second long beeping sound continuously; and when the current danger level is determined to be the fourth warning level, a notification signal is sent to the remote control center.

In an embodiment of the present invention, the behavior recognition model is used to mark the depth characteristics of the user in the image sequence, and the gaze recognition model is used to find multiple characteristic points of the user in the image sequence.

The image recognition device of the present invention includes: an image capture device that captures an image sequence; a storage device including a behavior recognition model, a line of sight recognition model, and a hazard determination model; and a processor coupled to the image capture device storage device. The processor is configured to: obtain the image sequence from the image capture device; analyze the image sequence using the behavior recognition model to obtain the user's behavior posture; use the line of sight recognition model to analyze the image sequence to obtain the user's line of sight characteristics; use the risk determination The model calculates the first score corresponding to the behavior and posture, and calculates the second score corresponding to the line of sight feature, and then calculates the risk score based on the first score and the second score, and issues corresponding warning signals based on the risk score.

Based on the above, the present invention uses the behavior recognition model and the line of sight recognition model line to simultaneously recognize the behavior posture and the line of sight direction to obtain a more accurate recognition result, and issue corresponding warning signals based on the hazard score, thereby reminding the driver and the rear vehicle.

FIG. 1 is a block diagram of an image recognition device according to an embodiment of the invention. Please refer to FIG. 1, the image recognition device 100 includes a processor 110, a storage device 120 and an image capture device 130. The processor 110 is coupled to the storage device 120 and the image capturing device 130. In an embodiment, the image recognition device 100 may be installed in a vehicle and connected to the center console of the vehicle. In another embodiment, the image recognition device 100 may be integrated in the center console of the vehicle.

The processor 110 is, for example, a central processing unit (CPU), a physical processing unit (PPU), a programmable microprocessor (Microprocessor), an embedded control chip, and a digital signal processor (Digital Signal Processor). Processor, DSP), Application Specific Integrated Circuits (ASIC) or other similar devices.

The storage device 120 includes a behavior recognition model 121, a sight recognition model 123 and a danger determination model 125. The behavior recognition model 121, the sight recognition model 123, and the danger determination model 125 are respectively composed of one or more code fragments, which are executed by the processor 110 to achieve their corresponding functions.

The image capturing device 130 is, for example, a video camera, a camera, etc., using a charge coupled device (CCD) lens, a complementary metal oxide semiconductor transistors (CMOS) lens, and the like to obtain an image sequence. Here, the image sequence can be a dynamic image or a static image.

The behavior recognition model 121 mainly deals with character behavior, including distraction, fatigue, abnormal driving, and other physical characteristics and actions of the character. In the training phase of the behavior recognition model 121, model training can be performed on the characteristics of a variety of behavior postures. These behaviors can also be divided into two categories: distracted behaviors and fatigue behaviors. Distracted behaviors include eating and drinking while the vehicle is running, operating the center console in the vehicle while the vehicle is running, using smart mobile devices while the vehicle is running, and leaving the steering wheel while the vehicle is running. Fatigue behaviors include yawning while driving, dozing off while driving, etc.

In one embodiment, the DensePose model is used to build the behavior recognition model 121. The DensePose model uses three-dimensional features of characters to build models. Compared with traditional two-dimensional images, it also adds depth features. Therefore, the behavior recognition model 121 can be used to mark the depth features of each human body part of the user in the image sequence, prompting Neural networks can infer the remaining occluded character features and actions based on the existing human body parts in the two-dimensional image, thereby greatly improving the recognition types and accuracy of traditional behavior recognition. At the same time, this architecture can use a general image capturing device 130 to obtain a two-dimensional image, and then use a DensePose model to build a three-dimensional feature. In addition to improving the recognition accuracy, it also has the advantages of reducing cost and easy popularization. In the behavior recognition model 121 using the DensePose model, the recognition rate for unoccluded images can be as high as 90%, and the recognition rate for partially occluded images can also reach 70% to 80%. In addition, the behavior recognition model 121 using the DensePose model can recognize behaviors from a single frame, thereby speeding up the recognition time.

The gaze recognition model 123 mainly processes the gaze direction of the character's face, and further processes characteristic actions such as dynamic panel movement, distraction, and fatigue recognition. In one embodiment, the OpenPose model is used to build the sight recognition model 123. The OpenPose model uses Vector to identify the line of sight. For example, the OpenPose model is used to find the three-point feature of the face, and actively output the character's feature coordinate axis to facilitate subsequent model training. Afterwards, in at least two adjacent frames in time captured based on the sequence, the person’s line of sight direction is determined by recognizing the deformation ratio of the three-point coplanar triangle.

Fig. 2 is a schematic diagram of feature points identified based on the OpenPose model according to an embodiment of the present invention. In Figure 2, the feature points (N0～N17) on the user identified through the OpenPose model are drawn. The feature points N0 to N17 are key points that constitute the skeleton of the human body. In this embodiment, the feature points N0, N14, and N15 on the face of the user identified by the OpenPose model are used as the line of sight features. Based on the deformation ratio of the coplanar triangle formed by the feature points N0, N14, and N15 in consecutive multiple frames, and the positions of the feature points N0, N14, and N15 in different frames, the user's line of sight direction is determined.

In the gaze recognition model 123 using the OpenPose model, by marking the feature points N0, N14, and N15 of the person’s face (as gaze features), it can determine whether the user’s gaze direction is offset, and further determine whether the user is distracted, Whether to check the left and right vehicles, whether to check the rear mirror, whether to use the electronic device of the car, whether to drive fatigue, etc. In addition, the processor 110 can also switch the screen of the display panel of the main console or drive a specific operation of the main console according to the line of sight characteristics. For example, when it is detected through the sight recognition model 123 that the user is looking in the direction of the A-pillar in the car, the processor 110 switches the screen of the display panel to display the A-pillar exterior image. Alternatively, the processor 110 uses the line of sight feature to control specific operations such as the size of the music and the navigation switch.

Generally speaking, to perform eye movement recognition, an eye tracker must be installed or the image capturing device 130 must be faced to the user to shoot, otherwise it cannot be performed. The gaze recognition model 123 using the OpenPose model can directly determine the gaze direction through the direction of the face, which greatly increases the degree of freedom of the mechanism setting, and can also reduce the error caused by the deviation of the eyeball. In the line recognition model 123 adopting the OpenPose model, whether the image capturing device 130 is set to face the user or is not set to face the user, the recognition rate of the line of sight feature can be as high as 90%.

FIG. 3 is a flowchart of an image recognition method according to an embodiment of the invention. Referring to FIG. 1 and FIG. 3 at the same time, in step S305, an image sequence is obtained through the image capturing device 130. After the image sequence is obtained, the behavior recognition model 121 and the sight recognition model 123 can be simultaneously driven to perform step S310 and step S315 respectively. Of course, the execution sequence of the behavior recognition model 121 and the sight recognition model 123 is not limited here.

In step S310, the behavior recognition model 121 is used to analyze the image sequence to obtain the user's behavior posture. Next, in step S320, the risk determination model 125 is used to calculate the first score corresponding to the behavior posture. Specifically, the behavior recognition model 121 uses a neural network-like algorithm to identify whether there is a user in the image sequence, and when there is a user, it determines whether the behavior and posture correspond to a distracted behavior or a fatigue behavior. If the behavior posture conforms to the distracted behavior or the fatigue behavior, the first score is calculated by the risk determination model 125.

Specifically, when the behavior recognition model 121 determines that the behavior posture is a distracted behavior, the method of calculating the first score corresponding to the behavior posture through the danger determination model 125 is as follows: In the process of continuously determining that the behavior posture is a distracting behavior, the first score is added to the auxiliary score corresponding to the main score every time the first judgment time passes, until the behavior recognition model 121 detects that the behavior posture returns to normal state. In one embodiment, multiple distraction behaviors are set in the training of the behavior recognition model 121, and different main scores and auxiliary scores are set for different distraction behaviors, as shown in Table 1.

Table 1 Distraction Main score Auxiliary score Eating and drinking while driving 10 5 Operating the center console in the vehicle while the vehicle is running 30 10 Use smart mobile devices while driving 60 20 Hands leave the steering wheel while the vehicle is driving 60 20

In Table 1, the main score and auxiliary score are set based on the degree of risk of distracted behavior. For example, in the case where it is determined that the behavior posture is that the hand leaves the steering wheel while the vehicle is running, the first score is set to 60 minutes, and the first score is added to 20 points every 1 second in the action duration. That is, when it is determined that the hand leaves the steering wheel for 5 seconds while the vehicle is running, the first score corresponding to: 60 (main score) + 5 (duration)×20 (auxiliary score). And, when it is detected that the behavior posture returns to a normal state, the first score is reset to zero.

The behavior recognition model 121 can also add up the first score when multiple distracting behaviors are detected. For example, referring to Table 1, if it is detected that the driver is eating and drinking in the vehicle for 5 seconds (10+5×5), and it is detected that the driver is operating the center console in the vehicle for 1 second (30+10×1) while the vehicle is moving. , At this time, the first score is 35+40 points.

On the other hand, in the case where the behavior identification model 121 determines that the behavior posture is a fatigue behavior, the method of calculating the first score corresponding to the behavior posture through the risk determination model 125 is as follows: In the case of a fatigue behavior, the first score is set as Corresponds to the preset score A1 with a fatigue level of 1. After the preset time B1 has elapsed when the fatigue level is 1, and the behavior posture is still a fatigue behavior, the first score is set to correspond to the fatigue level 2. The default score A2 (A2>A1); in the case of fatigue level 2 after the preset time B2, in the case of determining that the behavior posture is still fatigue behavior, the first score is set to correspond to the fatigue level 3 The default score is A3 (A3>A2). When it is detected that the behavior posture returns to a normal state, the first score is returned to zero.

Specifically, during the training of the behavior recognition model 121, the characteristic patterns of fatigue behaviors are further set, including yawning while the vehicle is running, and dozing off while the vehicle is running. In addition, multiple fatigue levels are further set, and different preset scores are set for different fatigue levels. For example, fatigue levels 1 to 3 are set from a light fatigue level to a heavy fatigue level, and the higher the fatigue level, the higher the corresponding preset score.

Assuming that the preset score B1 corresponding to fatigue level 1 is 60 points, the preset score B2 corresponding to fatigue level 2 is 80 points, the preset score B3 corresponding to fatigue level 3 is 100 points, and the preset time B1 is 5 seconds. The preset time B2 is 3 seconds. When the initial determination behavior is fatigue behavior such as yawning while the vehicle is running, dozing off while the vehicle is running, it is regarded as a fatigue level 1, so the first score is set to 60 points. In the case where the fatigue level is 1 and the normal state is not restored after 5 seconds, it is considered that the fatigue level has increased to the fatigue level 2, and the first score is set to 80 points at this time. In the case where the fatigue level is 2 and the normal state is not restored after 3 seconds, the first score is set to 100 points. Here, the setting of 3 levels of fatigue level is only an example, and it is not limited to this.

In addition, in step S315, the visual line recognition model 123 is used to analyze the image sequence to obtain the visual line characteristics of the user. Next, in step S325, the risk determination model 125 is used to calculate the second score corresponding to the line of sight feature. Specifically, the line-of-sight recognition model 123 recognizes whether there is a user in the image sequence through a neural network-like algorithm, and obtains the line-of-sight characteristics of the user in the presence of the user. If the line of sight feature does not face the specified direction or the line of sight feature meets the line of sight fatigue feature, the second score is calculated for it through the risk determination model 125.

Specifically, after obtaining the user's line of sight feature, the line of sight recognition model 123 determines whether the user is not facing a designated direction based on the line of sight feature. For example, it is determined whether the driver is facing straight ahead based on the direction of the driver's line of sight. When it is determined that the user is not facing the designated direction, the second score of the behavior is calculated through the risk determination model 125. For example, every second judgment time (for example, 1 second), the second score is added to the designated score. For example, when it is judged that the driver’s line of sight is not in the front (specified direction), after 1 second, 15 minutes are added to the second score, and after 2 seconds, the second score is 30 minutes (15+15). And so on. When it is detected that the direction of the line of sight returns to the specified direction, the second score is returned to zero.

On the other hand, when the gaze recognition model 123 determines that the behavioral gaze feature is consistent with the gaze fatigue feature, the method of calculating the corresponding second score through the hazard determination model 125 is as follows: when the gaze feature is determined to match the gaze fatigue feature, set the second The score is the preset score C1 corresponding to the fatigue level of 1. After the preset time D1 has elapsed when the fatigue level is 1, the second score is set to correspond to the fatigue level when it is determined that the line-of-sight feature still meets the line-of-sight fatigue feature The default score of 2 is C2 (C2>C1); when the fatigue level is 2 after the preset time D2, when it is determined that the line of sight feature still meets the line of sight fatigue feature, the second score is set to correspond to the fatigue level of 3 The default score is C3 (C3>C2).

Specifically, in the training of the gaze recognition model 123, the characteristic pattern of the gaze fatigue feature is further set. In addition, multiple fatigue levels are further set, and different preset scores are set for different fatigue levels. For example, fatigue levels 1 to 3 are set from a light fatigue level to a heavy fatigue level, and the higher the fatigue level, the higher the corresponding preset score.

Assuming that the preset score C1 corresponding to fatigue level 1 is 60 points, the preset score C2 corresponding to fatigue level 2 is 80 points, the preset score C3 corresponding to fatigue level 3 is 100 points, and the preset time D1 is 5 seconds. The preset time D2 is 3 seconds. In the case where the initial judgment is consistent with the line-of-sight fatigue characteristics, the second score is set to 60 points. In the case where the normal state is not restored after 5 seconds, the second score is set to 80 points. If the normal state is not restored after 3 seconds, the second score is set to 100 points. This is only an example, and it is not limited to this. When the line of sight feature returns to a normal state, the second score is returned to zero. Here, it is considered that the line of sight feature does not meet the line of sight fatigue feature as a normal state.

After calculating the first score and the second score, in step S330, the risk determination model 125 is used to calculate the risk score based on the first score and the second score. In one embodiment, the risk determination model 125 adds the first score and the second score to obtain the risk score. In other embodiments, the calculation of the risk score may also be as follows: risk score=first score×first weight+second score×second weight. Among them, the first weight+the second weight=1.

After the risk score is obtained, in step S335, a corresponding warning signal is issued according to the risk score. In addition, it can be further set to: continue to send out a warning signal until the detected behavior posture returns to the normal state and the detected line of sight direction returns to the specified direction. Here, the warning levels can be divided into multiple levels, and different warning levels will issue different warning signals.

For example, there are 4 warning levels below. The hazard score is between 60 and 80, and the current hazard level is determined to be the first warning level. The corresponding warning signal includes a flashing light signal and a short sound (such as one short sound or two short sounds). The hazard score is between 80 and 100, and the current hazard level is determined to be the second warning level, and the corresponding warning signal includes the flashing light signal and the continuous first warning tone. The hazard score is between 100 and 200, and the current hazard level is determined to be the third warning level, and the corresponding warning signal includes the flashing light signal and the continuous second long sound (such as a sharp long sound). If the danger score is greater than 200, the current danger level is determined to be the fourth warning level, a notification signal is sent to the remote control center, and a flashing light signal and a second long tone can also be issued at the same time.

For example, if driving fatigue is detected for 8 seconds through the behavior recognition model 121 (the first score is 100 points), and at the same time it is detected through the gaze recognition model 123 that the line of sight characteristics match the line of sight fatigue characteristics for 5 seconds (the second score is 80 points), then The risk score is 180 points. Therefore, it is determined that the current danger level is the third warning level, and the corresponding warning signals include the flashing light signal and the continuous second long sound (such as a sharp long sound).

In one embodiment, when the risk score is greater than 200, the processor 110 may send a notification signal to the remote control center through wireless communication technologies such as WiFi (wireless fidelity) to facilitate subsequent tracking and correction. Or provide immediate relief. In addition, the image sequence captured by the image capturing device 130 can be further transmitted to the remote control center.

In addition, in addition to issuing a warning signal, it can further directly lock the operation of the center console (such as navigation screen, music control, etc.).

In summary, the present invention uses the behavior recognition model and the line of sight recognition model line to simultaneously recognize the behavior posture and the line of sight direction to obtain more accurate recognition results, and issue corresponding warning signals based on the hazard score, thereby reminding driving and Cars coming from behind. In addition, further use of depth features and vector analysis to separately train the behavior recognition model and the line of sight recognition model can improve the accuracy of recognition.

100: Image recognition device 110: processor 120: storage device 121: Behavior Recognition Model 123: Sight recognition model 125: Dangerous Judgment Model 130: Image capture device N0～N17: Feature points S305～S335: Steps of image recognition method

FIG. 1 is a block diagram of an image recognition device according to an embodiment of the invention. Fig. 2 is a schematic diagram of feature points identified based on the OpenPose model according to an embodiment of the present invention. FIG. 3 is a flowchart of an image recognition method according to an embodiment of the invention.

S305~S335: Steps of image recognition method

Claims (18)

An image recognition method includes: obtaining an image sequence from an image capturing device; and executing the following steps by an electronic device, including: inputting the image sequence to a behavior recognition model to obtain a behavior posture of a user Calculate a first score corresponding to the behavior posture; input the image sequence to a line of sight recognition model to obtain a line of sight direction of the user; calculate a second score corresponding to the line of sight feature; based on the first score and the The second score is used to calculate a risk score; and a corresponding warning signal is issued based on the risk score. The image recognition method according to claim 1, wherein after obtaining the behavior posture of the user, it further includes: in the case of determining that the behavior posture is a distraction behavior, calculating the first score corresponding to the behavior posture , Including: setting the first score as a main score corresponding to the distraction behavior; and in the process of continuously determining that the behavior posture is the distraction behavior, each time a first determination time elapses, the first score Add an auxiliary score corresponding to the main score. The image recognition method according to claim 1, wherein after obtaining the behavior posture of the user, it further includes: setting the first score in the case of determining that the behavior posture is a fatigue behavior Is a first preset score corresponding to a fatigue level of 1; when the fatigue level is 1, after a first preset time has elapsed, it is determined that the behavior posture is still the fatigue behavior. A score is set as a second preset score corresponding to the fatigue level of 2, wherein the second preset score is greater than the first preset score; and when the fatigue level is 2, it passes through a second preset After time, if it is determined that the behavior posture is still the fatigue behavior, the first score is set as a third preset score corresponding to the fatigue level of 3, wherein the third preset score is greater than the second prediction Set the score. The image recognition method according to claim 1, wherein the step of inputting the image sequence to the gaze recognition model to obtain the gaze direction of the user includes: using the user's face recognized by the gaze recognition model The multiple feature points are used as a line of sight feature, and the line of sight direction is determined based on the line of sight feature; wherein, after obtaining the line of sight direction of the user, it further includes: judging whether the user is not facing a line of sight based on the line of sight feature Designated direction; and when it is determined that the user is not facing the designated direction, calculating the second score includes: adding a designated score to the second score every time a second judgment time has elapsed. The image recognition method according to claim 1, wherein the step of inputting the image sequence to the line-of-sight recognition model to obtain the line-of-sight direction of the user includes: Use the multiple feature points on the user's face identified by the gaze recognition model as a gaze feature, and determine the gaze direction based on the gaze feature; wherein, after obtaining the gaze direction of the user, It further includes: judging whether the line-of-sight feature meets a line-of-sight fatigue feature based on the line-of-sight feature; when determining that the line-of-sight feature meets the line-of-sight fatigue feature, setting the second score as a first preset score corresponding to a fatigue level of 1; When the fatigue level is 1, after a first preset time has elapsed, when it is determined that the line of sight feature still conforms to the line of sight fatigue feature, the second score is set to a second preset score corresponding to the fatigue level of 2. The second preset score is greater than the first preset score; and when the fatigue level is 2 after a second preset time, when it is determined that the line of sight feature still meets the line of sight fatigue feature, the second The score is set as a third preset score corresponding to the fatigue level of 3, wherein the third preset score is greater than the second preset score. The image recognition method according to claim 1, further comprising: resetting the first score to zero when detecting that the behavior posture returns to a normal state; and when detecting that the line of sight direction returns to a specified direction , Reset the second score to zero. The image recognition method according to claim 1, wherein after the corresponding warning signal is issued according to the risk score, it further comprises: continuously issuing the warning signal until the detected behavior posture returns to a normal state and detection Go to the direction of the line of sight and return to a specified direction. The image recognition method according to claim 1, wherein the step of issuing the corresponding warning signal according to the risk score includes: judging based on the risk score that a current risk level is a first warning level, a second warning level, A third warning level or a fourth warning level; when it is determined that the current danger level is the first warning level, a flashing light signal and a short tone are issued; when the current danger level is determined to be the second warning level, it is issued The flashing light signal and continuously emitting a first long warning tone; when determining that the current danger level is the third warning level, emitting the flashing light signal and continuously emitting a second long warning tone; and when determining that the current danger level is the first warning level Four warning levels, sending a notification signal to a remote control center. The image recognition method according to claim 1, wherein the behavior recognition model is used to mark the depth characteristics of the user in the image sequence, and the line of sight recognition model is used to find out a plurality of the user in the image sequence Feature points. An image recognition device includes: an image capture device that captures an image sequence; a storage device including a behavior recognition model, a line of sight recognition model, and a hazard determination model; and a processor coupled to the image capture The capture device and the storage device are configured to: obtain the image sequence from the image capture device; input the image sequence to the behavior recognition model to obtain a user's row Is posture; input the image sequence to the line of sight recognition model to obtain a line of sight direction of the user; use the hazard determination model to calculate a first score corresponding to the behavior posture, and calculate a second score corresponding to the line of sight feature Then, a risk score is calculated based on the first score and the second score, and a corresponding warning signal is issued according to the risk score. The image recognition device according to claim 10, wherein the processor is configured to: in the case of using the behavior recognition model to determine that the behavior posture is a distracting behavior, calculate the corresponding behavior posture through the danger determination model The first score includes: setting the first score as a main score corresponding to the distraction behavior; The first score plus an auxiliary score corresponding to the main score. The image recognition device according to claim 10, wherein the processor is configured to: in the case of using the behavior recognition model to determine that the behavior posture is a fatigue behavior, set the first score to correspond to the hazard determination model A first preset score with a fatigue level of 1. After a first preset time has elapsed when the fatigue level is 1, the behavior identification model is used to determine that the behavior posture is still the fatigue behavior. Through the risk determination model, the first score is set as a second preset score corresponding to the fatigue level of 2, wherein the second preset score is greater than the first preset score; in the case where the fatigue level is 2 After a second preset time has elapsed, when the behavior identification model is used to determine that the behavior posture is still the fatigue behavior, the first score is set to correspond to the fatigue level of 3 through the risk determination model. Three preset scores, where the third preset score is greater than the second preset score. The image recognition device according to claim 10, wherein the processor is configured to: use a plurality of feature points on the user's face recognized by the gaze recognition model as a gaze feature, and based on the gaze feature To determine the direction of the line of sight; use the line of sight recognition model to determine whether the user is not facing a specified direction based on the feature of the line of sight; when it is determined that the user is not facing the specified direction, calculate the second score through the hazard determination model, including : Every time a second judgment time has elapsed, add a specified score to the second score. The image recognition device according to claim 10, wherein the processor is configured to: use a plurality of feature points on the user's face recognized by the gaze recognition model as a gaze feature, and based on the gaze feature To determine the direction of the line of sight; use the line of sight recognition model to determine whether it meets a line of sight fatigue feature based on the line of sight feature; use the line of sight recognition model to determine that the line of sight feature meets the line of sight fatigue feature When the risk determination model is used, the second score is set as a first preset score corresponding to a fatigue level of 1; when the fatigue level is 1, after a first preset time has elapsed, the sight recognition is used When the model determines that the line-of-sight feature still meets the line-of-sight fatigue feature, the second score is set to a second preset score corresponding to the fatigue level of 2 through the risk determination model setting, wherein the second preset score is greater than the A first preset score; when a second preset time has elapsed when the fatigue level is 2, when the sight recognition model is used to determine that the sight feature still meets the sight fatigue feature, the risk determination model is used to set the first The second score is set as a third preset score corresponding to the fatigue level of 3, wherein the third preset score is greater than the second preset score. The image recognition device according to claim 10, wherein the processor is configured to: when detecting that the behavior posture returns to a normal state, reset the first score to zero through the hazard determination model; and when detecting When the line of sight direction returns to a specified direction, the second score is reset to zero through the danger determination model. The image recognition device according to claim 10, wherein the processor is configured to: continuously send out the warning signal through the hazard determination model until the detection of the behavior and posture returns to a normal state and the detection of the return of the line of sight To a specified direction. The image recognition device according to claim 10, wherein the processor is configured to: determine a current risk level as a first warning level, a second warning level, and a third warning based on the risk score through the risk determination model City level or a fourth warning level; when the current danger level is determined to be the first warning level, a flashing light signal and a short tone are issued; when the current danger level is determined to be the second warning level, the flashing light signal is issued And continuously emit a first long warning tone; when determining that the current danger level is the third warning level, emit the flashing signal and continuously emit a second long warning tone; when determining that the current danger level is the fourth warning level, send A notification signal to a remote control center. The image recognition device according to claim 10, wherein the processor is configured to: use the behavior recognition model to mark the depth characteristics of the user in the image sequence, and to find the depth characteristics of the user in the image sequence through the sight recognition model Multiple characteristic points of the user.

Images