TWI831524B - System and method for abnormal driving behavior detection based on spatial-temporal relationship between objects - Google Patents

Abstract

A system and a method for an abnormal driving behavior detection based on a spatial-temporal relationship between objects are disclosed. The system and method comprises: an in-cockpit device for capturing continuous image in the cockpit and presenting the images on a webpage in real time; a system host for receiving the image and determining whether abnormal driving behavior is present in the image to generate a voice alert and an abnormal event alert notification; multiple cell phones for receiving abnormal event alert; and a monitor screen to instantly play the in-cockpit taken image via the webpage and to instantly display abnormal event alert notification from the system host. The present invention can effectively determine the occurrence of abnormal driving behavior and issue notification and alert.

Description

Dangerous driving behavior detection system and method based on spatio-temporal relationship of objects

The present invention relates to a driving behavior detection system and method, and in particular to a dangerous driving behavior detection system and method based on the spatiotemporal relationship of objects.

Cars are one of the important means of transportation in human daily life. From media reports and multiple survey reports, we can see that driver distraction and fatigue are important causes of accidents. In recent years, car accidents have occurred one after another due to driver distraction. Therefore, the development and deployment of safe driving behavior monitoring systems have received increasing attention. Although traditional traffic surveillance cameras have recording and playback functions, because they only have the recording function for the monitored area, they cannot provide immediate warning and reminder effects for sudden abnormal events. Although there is currently detection of driver's bad driving behavior in the cockpit, due to the large amount of calculation and complex architecture, it can only be limited to the detection of facial expressions, and it cannot be immediately known to third parties other than the driver. This dangerous driving behavior.

In view of this, one of the purposes of the present invention is to provide a dangerous driving behavior detection system and method based on the spatiotemporal relationship of objects to solve the above long-standing problems.

In order to achieve the above purpose, the present invention proposes a dangerous driving behavior detection system based on the spatiotemporal relationship of objects, including: a cockpit device, located inside a cockpit, including at least a miniature camera device with a network transmission function; A microprocessor with a wireless network transmission function. The micro-photography device with a network transmission function has a camera lens to capture an image in a cockpit. The image is a continuous frame, and the micro-photography device has a wireless network. The microprocessor with the transmission function displays the image on a web page using an MJPG (Motion JPG) real-time image streaming technology; a system host receives the image and uses a deep learning model architecture to target the image Performing an abnormal driving behavior detection step, the dangerous driving behavior detection step includes defining a plurality of spatial events of a plurality of objects based on their positional relationships and overlapping situations in the cockpit, and Based on the time relationship of the spatial events, it is determined whether a dangerous driving behavior occurs in the image. If the dangerous driving behavior occurs, the system host issues a dangerous event warning notification and transmits the type of the dangerous driving behavior to the The microprocessor with wireless network transmission function in the cockpit enables a voice playback unit in the cockpit to generate a corresponding voice warning reminder; a plurality of mobile phones are respectively owned by a driver and the driver. Possessed by a supervisor of a company, it is used to receive the dangerous event warning notification sent by the system host through a communication software with a message warning function; and a monitoring screen is used to play the real-time broadcast through the web page. The image captured by the micro camera device with network transmission function and the dangerous event warning notice issued by the system host are displayed in real time.

Among them, the dangerous driving behavior detection step uses the YOLOv5 architecture and adopts Mosaic (mosaic) data enhancement technology to train multiple object detection models to detect the objects, and by defining the spaces of the objects The time relationship between the event and the spatial events is used to determine whether the dangerous driving behavior appears in the image.

Among them, the system host records the time and type of the dangerous driving behavior in the image in a database for the assessment and management of the driver.

Among them, the dangerous driving behavior detection step is based on whether the positional relationship and overlap of the objects in the cockpit exceed the corresponding first preset value, thereby defining the spatial events of the objects, and the dangerous driving The behavior detection step determines whether the dangerous driving behavior appears in the image based on whether the duration of the spatial events exceeds the corresponding second preset value.

Among them, these objects include multiple items in the cockpit and multiple personal characteristics of the driver. These items are food, seat belts, steering wheels, and mobile phones. These human characteristics are faces, eyes, and hands. .

In order to achieve one of the aforementioned objectives, the present invention proposes a dangerous driving behavior detection method based on the spatio-temporal relationship of objects, which includes the following steps: performing an image shooting and real-time streaming step, which uses a network transmission device with a photographic lens. The functional miniature camera device captures an image in the cockpit, the image is a continuous frame, and uses a microprocessor with a wireless network transmission function to present the image on a web page using an MJPG real-time image streaming technology ; Carry out an abnormal driving behavior detection step, which uses a system host to receive the image, and uses a deep learning model architecture to perform the abnormal driving behavior detection step on the image. , the dangerous driving behavior detection step includes defining a plurality of spatial events of a plurality of objects in the cockpit based on their positional relationships and overlapping conditions, and based on the time relationship of these spatial events, in order to determine what is in the image. Whether a dangerous driving behavior occurs, and if the dangerous driving behavior occurs, the system host sends a dangerous event warning notification and transmits the type of the dangerous driving behavior to the microprocessor with wireless network transmission function in the cockpit device, thereby causing a voice playback unit in the cockpit to generate a corresponding voice warning reminder; performing a communication software warning step, which is used by a driver and a supervisor of a company to which the driver belongs. A plurality of mobile phones receive the dangerous event warning notification sent by the system host through a communication software with a message warning function; and perform a monitoring step, which uses a monitoring screen to play the dangerous event warning notification through a web page in real time. The image captured by the micro camera device with network transmission function and the dangerous event warning notice issued by the system host are displayed in real time.

Among them, the dangerous driving behavior detection step uses the YOLOv5 architecture and adopts Mosaic data enhancement technology to train a plurality of object detection models to detect the objects, and by defining the spatial events of the objects and the The temporal relationship between these spatial events is used to determine whether the dangerous driving behavior occurs in the image.

Among them, the system host records the time and type of the dangerous driving behavior in the image in a database for the assessment and management of the driver.

Among them, the dangerous driving behavior detection step is based on whether the positional relationship and overlap of the objects in the cockpit exceed the corresponding first preset value, thereby defining the spatial events of the objects, and the dangerous driving The behavior detection step determines whether the dangerous driving behavior appears in the image based on whether the duration of the spatial events exceeds the corresponding second preset value.

Among them, the objects include a plurality of objects in the cockpit and a plurality of personal characteristics of the driver.

Based on the above, the dangerous driving behavior detection system and method based on the spatiotemporal relationship of objects of the present invention has the following advantages:

(1) Feasibility: The lightweight system has a simple structure and can achieve real-time identification effects. This system only uses the deep learning model of YOLOv5 when detecting objects. When identifying dangerous driving behaviors, , because the spatial and temporal relationships of objects are used to judge, and due to the small amount of calculation, real-time recognition can be achieved, greatly improving the feasibility of the system.

(2) Scalable: New types of dangerous driving behaviors can be easily expanded to make the system’s identification capabilities more comprehensive. This system can easily add new types of dangerous driving behaviors by simply training the object detection model, and Just define the spatio-temporal relationship between the objects of the event.

(3) Practical: This system has the functions of real-time monitoring, real-time dangerous driving behavior identification and real-time notification. It can notify the driver to stop dangerous behavior in the shortest possible time, effectively reducing the incidence of traffic accidents. In addition, this system will also record each driver's dangerous events in the database, and these records can be used by the company for driver management and assessment.

In order to enable Jun Shen to have a further understanding of the technical features and technical effects of the present invention, preferred embodiments and accompanying detailed descriptions are provided below.

In order to facilitate understanding of the technical features, content and advantages of this invention and the effects it can achieve, this invention is described in detail below with diagrams and in the form of expressions of embodiments. The purpose of the diagrams used is only They are for illustration and auxiliary instructions, and may not represent the true proportions and precise configurations of the creation after its implementation. Therefore, the proportions and configurations of the attached drawings should not be interpreted to limit the scope of rights in the actual implementation of this creation. In addition, to facilitate understanding, the same elements in the following embodiments are labeled with the same symbols for explanation.

In addition, unless otherwise noted, the terms used throughout the specification and patent application generally have the ordinary meanings of each term used in the field, the content disclosed herein, and the specific content. Certain terms used to describe the invention are discussed below or elsewhere in this specification to provide those skilled in the art with additional guidance in describing the invention.

The use of "first", "second", "third", etc. in this article does not specifically refer to the order or sequence, nor is it used to limit the present invention. It is only used to distinguish components described by the same technical terms. Or just an operation.

Secondly, if the words "include", "includes", "have", "contains", etc. are used in this article, they are all open terms, which means including but not limited to.

The dangerous driving behavior detection system based on the spatio-temporal relationship of objects of the present invention includes four parts. The first part is a photography device installed in the cockpit and a microprocessor with a wireless network transmission function. Its function is to photograph the cockpit. The image is transmitted to the system host in the second part through the 5G wireless network. When the system host detects a dangerous event, it will send the category of the dangerous behavior to the cockpit and issue a voice warning message to remind the driver to stop the dangerous behavior immediately. The second part is the system host placed in the computer room. The system host is mainly responsible for two tasks: object detection and abnormal driving behavior detection.

Before the dangerous driving behavior detection system is deployed, the present invention will first conduct a series of training processes so that the system host can identify dangerous driving behaviors from continuous images. Once a dangerous driving behavior is identified, in addition to recording the time and type of the incident in the database, a message will also be sent to the driver and the preset company supervisor's mobile phone through Line Notify. In addition, the incident will also be sent to to the company-side monitoring screen so that relevant personnel can take necessary reminders in a timely manner.

The third and fourth parts are the driver's and company supervisor's mobile phones and the company's monitoring screen. Their function is to receive warning notifications of dangerous events sent by the system host. The monitoring screen on the company side can also monitor the images returned from the cockpit in real time, allowing the company to achieve effective management.

For details, please refer to Figures 1 and 2. Figure 1 is a schematic diagram of the dangerous driving behavior detection system based on the spatio-temporal relationship of objects according to the present invention, and Figure 2 is a dangerous driving behavior detection method based on the spatio-temporal relationship of objects according to the present invention. Schematic diagram. The dangerous driving behavior detection system 10 based on the spatio-temporal relationship of objects of the present invention mainly consists of four parts, including the equipment in the cockpit 20, the system host 30, a plurality of mobile phones 40 and the monitoring screen 50.

The equipment 20 in the cockpit is used to perform an image capturing and real-time streaming step S100, and is installed inside the cockpit of a vehicle such as a car, and at least includes a micro camera device 22 with network transmission function and a wireless network. The microprocessor 24 has a network transmission function, and the micro photography device 22 with a network transmission function has a camera lens to capture images in the cockpit, and the images are continuous images. Moreover, the microprocessor 24 with wireless network transmission function presents the above-mentioned images on the web page using MJPG real-time image streaming technology. Web page frameworks are preferably lightweight, concise and highly scalable. Therefore, combined with network transmission technology, executing streaming web pages can significantly reduce the burden that traditional technologies cannot solve. MJPG real-time image streaming technology is a good open source project that can access cameras compatible with wireless network transmission technology (i.e., micro camera devices 22 with network transmission functions) through HTTP, thereby To achieve the effect of remote video transmission, the effect of real-time streaming of images can be achieved by setting the action parameter of the web page to Stream. Among them, the equipment in the cockpit 20 also optionally includes a voice playback unit 26. The voice playback unit 26 is, for example, a broadcasting device such as a speaker, and is electrically connected to a microprocessor 24 with a wireless network transmission function. The voice playing unit 26 receives voice commands transmitted by the microprocessor 24 with wireless network transmission function, thereby generating corresponding voice.

In the dangerous driving behavior detection system 10 based on the spatio-temporal relationship of objects of the present invention, the system host 30 is used to perform an abnormal driving behavior detection (abnormal driving behavior detection) step S200, and is, for example, located in the company where the driver belongs. place. The system host 30 receives the above-mentioned image transmitted by the device 20 in the cockpit, and uses a deep learning model architecture to perform abnormal driving behavior detection (abnormal driving behavior detection) on the image in step S200. Among them, the dangerous driving behavior detection step S200 includes defining a plurality of spatial events (spatial events) of a plurality of objects in the image based on their positional relationships and overlapping conditions in the cockpit, and based on the temporal relationship of the spatial events ( temporal relation) to determine whether dangerous driving behavior occurs in the image. If a dangerous driving behavior occurs, the system host 30 issues a dangerous event warning notification and transmits the type of dangerous driving behavior to the microprocessor 24 with a wireless network transmission function in the cockpit, whereby the microprocessor 24 with a wireless network transmission function The processor 24 can generate a corresponding voice command according to the type of dangerous driving behavior, so that the voice playing unit 26 in the cockpit generates a corresponding voice warning reminder. The image of the cockpit is composed of a series of continuous pictures. After a frame is detected by YOLOv5 objects, it may contain a variety of objects (objects) such as faces, eyes, hands, seat belts, mobile phones, and food. For example, these objects can be as shown in Figure 3: object a-left hand, object b-right hand, object c-steering wheel, object d-mobile phone, object e-left eye, object f-right eye, etc.

For example, examples of dangerous driving behaviors and corresponding voice warning reminders are (1) Dozing off (eyes closed), that is, if the driver's eyes are recognized to be closed, the driver will be voice reminded "Don't fall asleep." oh". (2) Using a mobile phone, that is, if it is recognized that the driver is using a mobile phone, the driver will be reminded by voice to "Do not use the mobile phone while driving." (3) The seat belt is not fastened, that is, if it is recognized that the driver has not fastened the seat belt, that is, the seat belt is not recognized, the driver will be reminded by voice "Please fasten the seat belt." (4) With both hands off the steering wheel, that is, if it is recognized that the driver is not holding the steering wheel, that is, his hands are not recognized, the driver will be reminded by voice to "please hold the steering wheel tightly." (5) The driver is not detected, that is, if the driver's face cannot be recognized, the driver will be reminded by voice "No driver detected, please look straight ahead." Moreover, the above examples are only examples and are not intended to limit the present invention.

In addition, the above-mentioned dangerous driving behavior detection step of the present invention uses, for example, the YOLOv5 architecture and Mosaic data enhancement technology to train multiple object detection models to detect these objects, and by defining these objects These spatial events and the time relationship between these spatial events are used to determine whether the dangerous driving behavior occurs in the image.

The present invention is based on the positional relationship between various objects (for example, object a-left hand, object b-right hand, object c-steering wheel, object d-mobile phone, object e-left eye, object f-right eye, etc.) in space and The overlapping situation defines many different spatial events. For example, object a-left hand (Object-LHand), object b-right hand (Object-RHand) and object c-steering wheel (Object-Steering Wheel) do not have any overlap. It will be regarded as an event when the hand leaves the steering wheel; for another example, if object a-left hand (Object-LHand) or object b-right hand b (Object-RHand) overlaps with object d-mobile phone (Object-Mobile Phone), it will be Considered as an event of swiping the phone or answering the phone. The present invention can easily obtain the overlapping situation of objects (Object) by calculating the overlapping situation of the bounding boxes of each object (Object). As shown in Figure 4, the present invention divides the area of the overlapping area of two objects by the two objects. The numerical percentage of the total area of the bounding box of each object is used to determine the overlap of the two objects, and the above numerical percentage can be determined according to the requirements. The numerical percentage can, for example, be any value between 1% and 100%. In addition, the present invention also considers the temporal relation of spatial events. When a spatial event is identified, the duration of the event (frame count) will be calculated. Once the duration exceeds the limit of each event, The defined establishment time (the number of frames of the space event exceeds the defined value), the event will be considered as dangerous driving behavior, for example, holding a mobile phone for more than 5 seconds (125 frames), as shown in Figure 4, or closing eyes for more than 2 seconds (50 frames) (as shown in Figure 5). For example, each dangerous driving behavior has a program count. If the dangerous driving behavior is caught, the action will be reset to zero. When the number of frames meets the conditions, a screenshot will be taken and the voice will be played.

The above-mentioned objects include a plurality of items appearing in the cockpit, such as food, seat belts, steering wheels, mobile phones and other items, as well as a plurality of the driver's personal characteristics, such as the face, eyes, hands and other human organs. . The above-mentioned dangerous driving behavior detection step is based on whether the positional relationship of the objects in the cockpit and the overlapping situation exceed the corresponding preset value (for example, the first preset value, which can be a numerical percentage of the above-mentioned overlapping situation). any value in the range), thereby defining the spatial events of the objects. The dangerous driving behavior detection step is based on whether the duration of the spatial events exceeds the corresponding preset value (such as the second preset value) , to determine whether the dangerous driving behavior appears in the image.

In the dangerous driving behavior detection system based on the spatio-temporal relationship of objects of the present invention, a plurality of mobile phones 40 are held by the driver and the manager of the company to which the driver belongs respectively. These mobile phones 40 are used to carry out a communication software warning step. S300, for example, receiving a dangerous event warning notification sent by the system host 30 through communication software with a message warning function. Among them, the above-mentioned communication software is, for example, the Line instant messaging platform, and the above-mentioned message alert function is, for example, the Line Notify function of the Line instant messaging platform. In this way, when dangerous driving behavior is identified, the dangerous driving behavior detection system of the present invention will immediately send out the picture and warning message of the dangerous driving behavior through Line Notify, so the mobile phones 40 of the driver and his supervisor will display the above. The dangerous event warning notification is as shown in Figure 5. The dangerous event warning notification may, for example, include image screenshots corresponding to the above dangerous events and the content of the dangerous events.

In addition, the dangerous driving behavior detection system based on the spatio-temporal relationship of objects of the present invention further has a monitoring screen 50 for performing a monitoring step S400. The monitoring screen 50 is, for example, installed at the company premises where the driver belongs, and is used to instantly display the images captured by the micro camera device 22 with network transmission function through the above-mentioned web page and the real-time display of dangerous event warnings issued by the system host 30 Notification, in which the dangerous event warning notification may, for example, include image screenshots corresponding to the above-mentioned dangerous events and the content of the dangerous events. Moreover, the system host 30 of the dangerous driving behavior detection system of the present invention can also selectively record the time and type of the dangerous driving behavior in the images captured by the micro camera device 22 with network transmission function in a database. 32 in order to assess and manage the driver.

In one embodiment, in order to detect the environment and the driver, the present invention uses the entire image in the image as training. The production of this invention is based on the front seat of the cockpit as the main benchmark. The driver's eyes are clear and square as the training benchmark. The seat belt part is trained on different styles of clothes, and the steering wheel must be held on both sides or above. . The YOLOv5 architecture mainly consists of input, backbone, neck network and output. YOLOv5 mainly has four models, including s, m, l and x. For the input end, the improvement strategies proposed by the present invention during the model training phase mainly include Mosaic data enhancement, adaptive anchor frame calculation, and adaptive picture scaling. That is to say, the input end uses Mosaic data enhancement technology to input images. Mosaic data enhancement is divided into random scaling, random cropping, and random arrangement for splicing, which has good detection results for small targets. Backbone systems form graphic features. As far as the operation of the backbone is concerned, the present invention integrates some new strategies in other detection algorithms, mainly including the Focus structure and the CSP structure. Assume that after inputting 640 x 640 x 3, it will first pass through a Focus module. The Focus module then slices the image, stacks the four adjacent positions of the graphics, and samples them adjacently. In this way, four pictures are obtained. Complementary, without distortion, the input becomes 4 times the original, and the spliced picture becomes 12 channels compared to the original RGB three-channel mode. Finally, the new picture obtained is passed through the convolution layer operation, and finally the two images without distortion are obtained. times sampling feature map. The neck network mixes and combines graphic features to generate a feature pyramid. The YOLOv5 object detection network used in the present invention adds FPN and PAN structures between the backbone and the final output layer to further enhance the diversity and robustness of features. For example, it is divided into two sampling modes, which re-aggregate the feature parameters of different layers, down-sampling (FPN), and up-sampling (Botton-up), to strengthen the structured distinguishing features and form a new set of feature pyramids. The output end predicts the graphical features, applies the frame, and generates the final output vector of the confidence value and the bounding box. The anchor frame mechanism of the output layer of the present invention is the same as that of YOLOv4. The main improvements are the loss function GIOU_Loss during training and the DIOU_nms for prediction frame screening. For example, in network training, the network will output the prediction frame on the basis, and then compare it with the ground truth, calculate the difference between the two, and then update the iterative calculation parameters in reverse. YOLOv5 will perform the calculation in each training During the process, the best anchor box value is calculated by itself.

To sum up, the dangerous driving behavior detection system and method based on the spatiotemporal relationship of objects of the present invention has the following advantages:

(1) Feasibility: The lightweight system has a simple structure and can achieve real-time identification effects. This system only uses the deep learning model of YOLOv5 when detecting objects. When identifying dangerous driving behaviors, , because the spatial and temporal relationships of objects are used to judge, and due to the small amount of calculation, real-time recognition can be achieved, greatly improving the feasibility of the system.

(2) Scalable: New types of dangerous driving behaviors can be easily expanded to make the system’s identification capabilities more comprehensive. This system can easily add new types of dangerous driving behaviors by simply training the object detection model, and Just define the spatio-temporal relationship between the objects of the event.

(3) Practical: This system has the functions of real-time monitoring, real-time dangerous driving behavior identification and real-time notification. It can notify the driver to stop dangerous behavior in the shortest possible time, effectively reducing the incidence of traffic accidents. In addition, this system will also record each driver's dangerous events in the database, and these records can be used by the company for driver management and assessment.

The above is only illustrative and not restrictive. Any equivalent modifications or changes that do not depart from the spirit and scope of the present invention shall be included in the appended patent scope.

10: Dangerous driving behavior detection system

20: Equipment in the cockpit

22: Micro photography device with network transmission function

24: Microprocessor with wireless network transmission function

26: Voice playing unit

30: System host

32:Database

40:Mobile phone

50:Monitor screen

a, b, c, d, e, f: objects

S100, S200, S300, S400: steps

Figure 1 is a schematic diagram of the dangerous driving behavior detection system based on the spatio-temporal relationship of objects according to the present invention.

Figure 2 is a schematic diagram of the dangerous driving behavior detection method based on the spatio-temporal relationship of objects according to the present invention.

FIG. 3 is a schematic diagram of objects in the image captured by the dangerous driving behavior detection system of the present invention.

Figure 4 is a schematic diagram of the dangerous driving behavior detection system of the present invention determining the dangerous driving behavior of using a mobile phone.

Figure 5 is a schematic diagram of the dangerous driving behavior detection system of the present invention using the communication software of a mobile phone to present warning notifications of dangerous events.

10: Dangerous driving behavior detection system

20: Equipment in the cockpit

22: Micro photography device with network transmission function

24: Microprocessor with wireless network transmission function

26: Voice playing unit

30: System host

32:Database

40:Mobile phone

50:Monitor screen

Claims (10)

A dangerous driving behavior detection system based on the spatio-temporal relationship of objects, including: a cockpit device, located inside a cockpit, including at least a miniature camera device with a network transmission function and a wireless network transmission function. Microprocessor, the micro-photography device with network transmission function is equipped with a photographic lens to capture an image in the cockpit, the image is a continuous frame, and the microprocessor with wireless network transmission function is to The image is presented on a web page using MJPG real-time image streaming technology; a system host receives the image and uses a deep learning model architecture to perform abnormal driving behavior detection on the image ) step, the dangerous driving behavior detection step includes defining a plurality of spatial events of the plurality of objects in the image based on the positional relationship and overlap of the objects in the cockpit, and based on the time relationship of the spatial events, This is used to determine whether a dangerous driving behavior occurs in the image. If the dangerous driving behavior occurs, the system host issues a dangerous event warning notification and transmits the type of the dangerous driving behavior to the wireless network in the cockpit. The microprocessor with the transmission function enables a voice playback unit in the cockpit to generate a corresponding voice warning reminder; a plurality of mobile phones are held by a driver and a manager of a company to which the driver belongs. , used to receive the dangerous event warning notification sent by the system host through a communication software with a message warning function; and a monitoring screen used to instantly play the micro photography with network transmission function through the web page The image captured by the device and the dangerous event warning notification issued by the system host are instantly displayed. The dangerous driving behavior detection system based on the spatio-temporal relationship of objects as described in claim 1, wherein the dangerous driving behavior detection step uses the YOLOv5 architecture and adopts Mosaic data enhancement technology to train multiple object detection models to detect these objects, and by defining the spatial events of the objects and the temporal relationship between the spatial events, it is determined whether the dangerous driving behavior occurs in the image. The dangerous driving behavior detection system based on the spatio-temporal relationship of objects as described in claim 1, wherein the system host records the time and type of the dangerous driving behavior in the image in a database to serve as the driver assessment and management. The dangerous driving behavior detection system based on the spatio-temporal relationship of objects as described in claim 1, wherein the dangerous driving behavior detection step is based on whether the positional relationship and overlap of the objects in the cockpit exceed the corresponding first prediction Values are set to define the spatial events of the objects. The dangerous driving behavior detection step is based on whether the duration of the spatial events exceeds the corresponding second preset value to determine whether the hazard appears in the image. driving behavior. The dangerous driving behavior detection system based on the spatio-temporal relationship of objects as described in claim 1, wherein the objects include a plurality of items in the cockpit and a plurality of personal characteristics of the driver, and the items are food, safety Belt, steering wheel, mobile phone, these human body features are face, eyes, hands. A dangerous driving behavior detection method based on the spatio-temporal relationship of objects, including the following steps: performing an image shooting and real-time streaming step, which uses a miniature photography device with a photography lens and a network transmission function to capture a cockpit an image, which is a continuous image, and use a microprocessor with a wireless network transmission function to present the image on a web page using an MJPG real-time image streaming technology; perform an abnormal driving behavior detection step, which uses A system host receives the image and uses a deep learning model architecture to perform the abnormal driving behavior detection step on the image. The abnormal driving behavior detection step includes detecting the abnormal driving behavior based on a plurality of objects in the image. The positional relationship and overlapping situation in the cabin define a plurality of spatial events of these objects, and based on the time relationship of these spatial events, it is judged whether a dangerous driving behavior occurs in the image. If the dangerous driving behavior occurs, then The system host issues a dangerous event warning notification and transmits the type of dangerous driving behavior to the microprocessor with wireless network transmission function in the cockpit, thereby causing a voice playback unit in the cockpit to generate a corresponding A voice warning reminder; carrying out a communication software warning step, which uses multiple mobile phones respectively held by a driver and a manager of a company to which the driver belongs, through communication with a message warning function The software receives the dangerous event warning notification issued by the system host; and performs a monitoring step, which uses a monitoring screen to instantly play the image captured by the micro camera device with network transmission function through the web page; and Immediately display the dangerous event warning notification issued by the system host. The dangerous driving behavior detection method based on the spatio-temporal relationship of objects as described in request 6, wherein the dangerous driving behavior detection step uses the YOLOv5 architecture and adopts Mosaic data enhancement technology to train multiple object detection models to detect these objects, and by defining the spatial events of the objects and the temporal relationship between the spatial events, it is determined whether the dangerous driving behavior occurs in the image. The dangerous driving behavior detection method based on the spatio-temporal relationship of objects as described in claim 6, wherein the system host records the time and type of the dangerous driving behavior in the image in a database to serve as the driver assessment and management. The dangerous driving behavior detection method based on the spatio-temporal relationship of objects as described in claim 6, wherein the dangerous driving behavior detection step is based on whether the positional relationship and overlap of the objects in the cockpit exceed the corresponding first prediction Values are set to define the spatial events of the objects. The dangerous driving behavior detection step is based on whether the duration of the spatial events exceeds the corresponding second preset value to determine whether the hazard appears in the image. driving behavior. The dangerous driving behavior detection method based on the spatio-temporal relationship of objects as described in claim 6, wherein the objects include a plurality of objects in the cockpit and a plurality of personal characteristics of the driver.

Images