WO2022222174A1

WO2022222174A1 - Dangerous goods supervision system and method based on video image analysis

Info

Publication number: WO2022222174A1
Application number: PCT/CN2021/090657
Authority: WO
Inventors: 彭泳; 韩圩娟
Original assignee: 彭泳
Priority date: 2021-04-21
Filing date: 2021-04-28
Publication date: 2022-10-27
Also published as: CN113343754A

Abstract

A dangerous goods supervision system and method based on video image analysis. The system comprises: a video monitoring device and a server, wherein the video monitoring device comprises at least one camera and a processor; the video monitoring device is mounted in a cab of a dangerous goods transportation vehicle and is used for collecting and storing video information of a driver and/or an escort in the cab, and analyzing, according to the video information, whether the driver and/or escort exhibits a dangerous driving behavior; and the server stores and analyzes a short video sent by the video monitoring device, determines whether the driver and/or the escort has a driving qualification and/or escort qualification for the dangerous goods transportation vehicle, and performs hierarchical early warning according to the determination result. According to the present solution, during the whole process of dangerous goods transportation, a qualification and an operation behavior of a practitioner can be monitored in real time without blind spots, thereby ensuring the safety of dangerous goods transportation.

Description

Dangerous goods supervision system and dangerous goods supervision method based on video image analysis

This application claims the priority of the Chinese patent application with the application number 202110433162.4 and the title of "Dangerous Goods Supervision System and Supervision Method for Dangerous Goods Based on Video Image Analysis" filed with the China Patent Office on April 21, 2021, the entire contents of which are approved by Reference is incorporated in this application.

technical field

The invention relates to the field of information technology, in particular to a dangerous goods supervision system and a dangerous goods supervision method based on video image analysis.

Background technique

Dangerous goods generally refer to goods that are easy to cause combustion, explosion, corrosion, poisoning or radioactivity, and are likely to cause personal casualties and property losses during transportation and storage. Special protective facilities and measures must be adopted.

Due to the special nature of dangerous goods, necessary controls must be carried out during their transportation. The qualifications of the driver/escort and the behavior of the driver/escort are critical to the safety of the transport of dangerous goods. In the prior art, it is still impossible to supervise the qualifications of employees and their business behaviors in real time during the entire process of transportation of dangerous goods, and there are potential safety hazards during the transportation of dangerous goods.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problem of in-process and post-event supervision and early warning on the qualifications of the practitioners and the business behaviors of the practitioners during the transportation of dangerous goods.

In order to solve the above problems, the first embodiment of the present invention provides a dangerous goods supervision system based on video image analysis, including:

Video surveillance equipment and server; the video surveillance equipment includes at least one camera and a processor; the video surveillance equipment is installed in the cab of the dangerous goods transportation vehicle, and is used to collect and store the information of the driver and/or escort in the cab video information, and analyze whether the driver and/or escort has dangerous driving behavior according to the video information, and when the dangerous driving behavior exists, the short video, The shooting time information of the short video, the unique identification information of the camera that shoots the short video, and the correspondence information between the camera and the vehicle are sent to the server; a business management system is installed in the server; the server monitors the video The short video sent by the device is stored and analyzed to determine whether the driver and/or escort has the driving qualification and/or escort qualification for dangerous cargo transportation vehicles, and give a graded early warning according to the judgment result.

Optionally, the dangerous driving behavior includes smoking, dozing off, making a phone call, and not wearing a seat belt; when the video monitoring device detects that the driver has the dangerous driving behavior, the dangerous driving behavior is recorded as occurring. Short videos within a period of time before and after are stored locally and sent to the server, and a local early warning is issued to the driver and the server at the same time, reminding the driver to regulate driving behavior.

In order to solve the above problem, the second embodiment of the present invention provides a method for supervising dangerous goods by using the above-mentioned dangerous goods supervision system, including the following steps:

The server extracts the front face information of the driver from the short video, which is recorded as the first face information;

Judging whether there is an electronic waybill for dangerous goods corresponding to the short video in the business management system of the server;

If there is a corresponding electronic waybill for dangerous goods, obtain the face information of the first waybill filing practitioner associated with the waybill, and record it as the second face information

Compare the first face information and the second face information;

If the comparison fails, the first face information will be searched in the photo library of the professional qualification certificate corresponding to the professional qualification in the transportation industry. If the search is successful, the server will issue a first medium-risk early warning; if the search fails, the server will Issue the first high-risk warning.

Optionally, the above method further includes:

The server extracts the face information of the escort from the short video, which is recorded as the third face information;

If there is a corresponding electronic waybill for dangerous goods, obtain the face information of the second waybill filing practitioner associated with the waybill, and record it as the fourth face information;

comparing the third face information and the fourth face information;

If the comparison fails, the third face information will be searched in the photo library of the professional qualification certificate with the corresponding professional qualification in the transportation industry. If the search is successful, the server will issue a second medium-risk early warning; if the search fails, the server will A second high-risk warning was issued.

Optionally, the above method further includes: if there is no electronic waybill for dangerous goods corresponding to the short video in the service management system of the server, sending a third medium-risk early warning by the server.

Optionally, the above method further includes: after the server issues a third medium-risk early warning, searching the photo library of the qualification certificate of the first face information or the third face information that has the corresponding qualification in the transportation industry, if searching If it fails, the server issues a third high-risk warning.

Optionally, judging whether there is a dangerous goods electronic waybill corresponding to the short video in the business management system of the server includes:

According to the shooting time information of the short video, the unique identification information of the camera that shoots the short video, the corresponding relationship information between the camera and the vehicle, and the practitioner qualification information in the business management system, business license information, dangerous goods The vehicle road transport certificate information and the dangerous goods electronic waybill information determine whether there is a dangerous goods electronic waybill corresponding to the short video in the business management system of the server.

Optionally, the above method further includes: the server analyzes whether the driver and/or the escort has smoking behavior according to the short video; if there is smoking behavior, and the cargo category information associated with the vehicle is high-risk and flammable category, the server issues a fourth high-risk warning.

Optionally, the above-mentioned method further includes: after the server issues various types of early warnings, generating early warning records and storing them on the server; the early warning records include the time of the early warning, the type of early warning, the license plate number, the color of the license plate, and the company to which it belongs. /The name of the institution, the credit code of the institution, the registered address of the industry and commerce, the name/code of the issuing authority, and the storage address path information of the corresponding short video on the server.

Through the solution provided by the present invention, the professional qualifications and business behaviors of employees can be monitored in real time without dead ends in the whole process of dangerous goods transportation, so as to ensure the safety of dangerous goods transportation.

Description of drawings

1 is a schematic diagram of a dangerous goods supervision system based on video image analysis according to Embodiment 1 of the present invention;

FIG. 2 is a flowchart of a method for supervising dangerous goods according to Embodiment 2 of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

FIG. 1 is a schematic diagram of a dangerous goods supervision system based on video image analysis according to Embodiment 1 of the present invention. Referring to Figure 1, a dangerous goods supervision system based on video image analysis, including:

Video surveillance equipment and server; the video surveillance equipment includes at least one camera and a processor; the video surveillance equipment is installed in the cab of the dangerous goods transportation vehicle, and is used to collect and store the information of the driver and/or escort in the cab Video information, when the camera is installed, it is bound to the dangerous goods transport vehicle. Each camera has a unique binding corresponding to the dangerous goods transport vehicle. There can be multiple cameras in a cab, such as one for shooting driving Each camera is linked with the engine of the dangerous goods transport vehicle. Once the vehicle is started, the camera starts to work, and continuously shoots the video of the driver and/or the escort in the cab. image.

The video monitoring device can perform localized analysis on the video information, and the analysis is performed by a processor, which can be one or more processors, and judges whether the driver/escort is smoking or dozing according to the analysis result. If there are dangerous driving behaviors such as making phone calls or not wearing seat belts, the driver/escort will be reminded by voice or indicator light in the cab to regulate driving behavior and pay attention to driving safety; The video surveillance device also sends a short video within a period of time before and after the occurrence of the dangerous driving behavior, the shooting time information of the short video, the unique identification information of the camera that captured the short video, and the correspondence information between the camera and the vehicle to the said device. server.

The above-mentioned period of time may be several seconds, or ten seconds or several ten seconds. In actual use, it may be set according to the actual situation, such as communication bandwidth or storage capacity of the server. By sending short videos to the server instead of static pictures, it is possible to accurately determine dangerous driving behaviors and improve the supervision efficiency of dangerous cargo transport vehicles.

The server stores and analyzes the short video sent by the video monitoring device, judges whether the driver and/or escort has the driving qualification and/or escort qualification for dangerous goods transport vehicles, and conducts the operation according to the judgment result. Graded warning. The graded early warning may include three-level early warning, ie, low-risk early warning, medium-risk early warning, and high-risk early warning.

A business management system is installed in the server; and the business management system stores information on the qualifications of practitioners, business license information, road transport certificate information for dangerous goods vehicles, and electronic waybill information for dangerous goods. The server is based on the shooting time information of the short video, the unique identification information of the camera that shoots the short video, the correspondence information between the camera and the vehicle, and the practitioner qualification information and enterprise business license information in the business management system. , dangerous goods vehicle road transport certificate information, dangerous goods electronic waybill information to determine whether there is a dangerous goods electronic waybill corresponding to the short video in the business management system of the server, and further judge the driver and the driver according to the judgment result. / or whether the escort has the driving qualification and / or escort qualification of dangerous goods transport vehicles.

Dangerous cargo electronic waybill information includes cargo information, which includes cargo category code, product name, quantity and quality, etc. of the cargo being transported. The cargo categories are low-risk cargo, medium-risk cargo and high-risk cargo.

FIG. 2 is a flowchart of a method for supervising dangerous goods according to Embodiment 2 of the present invention. Referring to Figure 2, a method for monitoring dangerous goods includes the following steps:

If there is a corresponding electronic waybill for dangerous goods, obtain the face information of the first waybill filing practitioner associated with the waybill, and record it as the second face information; in this embodiment, the second face information can be obtained through the first Obtaining the photo on the qualification certificate of the practitioner for waybill filing;

Compare the first face information and the second face information;

The technology of extracting face information from the short video can adopt the commonly used face video detection technology in the prior art, such as multi-layer convolutional neural network, background difference method and multi-task recognition method based on clustering, etc. In the embodiment, the method for extracting the face information of the human face is: for each frame in the short video, calculate the triangular area identified by the eyes and the mouth of the human face; the frame with the largest triangular area in the short video is the short video. The frontal frame in the video.

The first waybill filing practitioner is the driver who has filed the current waybill. If the driver's face information extracted from the short video matches the driver's face information filed on the current waybill, it means that the driver who carries the current waybill is the current waybill. If there is no match, it means that the driver who is carrying the current waybill is not the driver who is on file with the current waybill. It is necessary to further search for the driver's face information in the photo library of the qualification certificate that has the corresponding qualifications in the transportation industry. Determine whether the driver has the qualifications to practice, and if the search is successful, the server will issue a first medium-risk warning, reminding that although the actual driver of the current waybill has the qualifications, the driver is inconsistent with the driver on file with the waybill, that is, The current waybill does not match the person and the waybill; if the search fails, the server will issue a first high-risk warning, reminding that the actual driver of the current waybill is inconsistent with the driver of the waybill, and the actual driver of the current waybill is not qualified to work Eligible high-risk situations.

The method may also include:

If there is a corresponding electronic waybill for dangerous goods, obtain the face information of the second waybill filing practitioner associated with the waybill, and record it as the fourth face information; in this embodiment, the fourth face information can be obtained through the second Obtaining the photo on the qualification certificate of the practitioner for waybill filing;

comparing the third face information and the fourth face information;

The second waybill filing practitioner is the escort registered with the current waybill. If the escort’s face information extracted from the short video matches the escort’s face information filed on the current waybill, it means that the escort who carries the current waybill is the current waybill. If the registered escort does not match, it means that the escort carrying the current waybill is not the escort registered with the current waybill. It is necessary to further search the escort's face information in the photo library of the qualification certificate of the corresponding professional qualification in the transportation industry. Determine whether the escort has the qualifications to practice, and if the search is successful, the server will issue a second medium-risk warning, reminding that although the actual escort of the current waybill has the qualifications, the escort is inconsistent with the escort recorded on the waybill, that is If the search fails, the server will issue a second high-risk warning, reminding that the actual escort of the current waybill is inconsistent with the escort of the waybill, and the actual escort of the current waybill is not qualified Eligible high-risk situations.

The above method further includes: if the electronic waybill for dangerous goods corresponding to the short video does not exist in the service management system of the server, the server sends a third medium-risk early warning. The third medium-risk warning indicates that the current waybill has not been filed and is a waybill that should not exist.

The following methods can be used to determine whether there is an electronic waybill for dangerous goods corresponding to the short video in the business management system of the server:

The above method also includes: after the server issues a third medium-risk early warning, searching the photo library with the corresponding professional qualifications for the first face information or the third face information in the transportation industry, and if the search fails, the server sends the information. The third high-risk warning. The issuance of the third high-risk warning indicates the existence of the following dangerous situations, that is, the driver or escort of the current, unrecorded waybill does not have the vehicle driving qualification or escort qualification, and this situation poses a huge hidden danger to the transportation safety of dangerous goods transport vehicles , needs to be given special attention.

The above method further includes: the server analyzes whether the driver and/or the escort has smoking behavior according to the short video; The above server issued a fourth high-risk warning.

In the actual driving process of dangerous goods transport vehicles, if the goods transported are high-risk, flammable and explosive goods, neither the driver nor the escort can smoke. Therefore, when the driver or the escort smokes Not only needs to be reminded locally, but the server also needs to determine whether the vehicle cargo category associated with the current waybill is a high-risk level. If it is a high-risk level, the server should immediately issue a high-risk warning.

The method further includes: after the server issues various types of early warnings, generating early warning records and storing them on the server; the early warning records include the time of the early warning, the type of early warning, the license plate number, the color of the license plate, and the name of the company/institution to which it belongs. , unit credit code, industrial and commercial registration address, issuing address and the storage address information of the corresponding short video on the server.

In addition, in specific practice, the above-mentioned medium and high-risk warnings can further support cross-industry and inter-departmental verification and joint supervision through the deep learning coding of frontal faces or frontal faces in short videos to discover more clues of major problems. Moreover, different types/levels of early warnings can be mapped to the credit system of enterprises/practitioners to realize credit supervision.

Although the present invention has been described in detail above with general description, specific embodiments and tests, some modifications or improvements can be made on the basis of the present invention, which is obvious to those skilled in the art . Therefore, these modifications or improvements made without departing from the spirit of the present invention fall within the scope of the claimed protection of the present invention.

Claims

A dangerous goods supervision system based on video image analysis, characterized in that it includes: video surveillance equipment and a server; the video surveillance equipment includes at least one camera and a processor; the video surveillance equipment is installed in the cab of the dangerous goods transportation vehicle , used to collect and store the video information of the driver and/or the escort in the cab, and analyze whether the driver and/or the escort has dangerous driving behavior according to the video information, when there is the dangerous driving behavior When the dangerous driving behavior occurs, the short video within a period of time before and after the occurrence of the dangerous driving behavior, the shooting time information of the short video, the unique identification information of the camera that shoots the short video, and the correspondence information between the camera and the vehicle are sent to the server; A business management system is installed in the server; the server stores and analyzes the short video sent by the video monitoring device, and judges whether the driver and/or escort has the driving qualification for dangerous goods transport vehicles and / or escort qualifications, and graded early warning based on the judgment results.
The system according to claim 1, wherein the dangerous driving behavior includes smoking, dozing off, making a phone call and not wearing a seat belt; the video monitoring device detects that the driver has the dangerous driving behavior When the dangerous driving behavior occurs, the short video within a period of time before and after the occurrence of the dangerous driving behavior is stored locally and sent to the server, and a local warning is issued to the driver and the server at the same time, reminding the driver to regulate the driving behavior.
A method for supervising dangerous goods using the system according to any one of claims 1-2, characterized in that it comprises the following steps:

The server extracts the front face information of the driver from the short video, which is recorded as the first face information;

Judging whether there is an electronic waybill for dangerous goods corresponding to the short video in the business management system of the server;

If there is a corresponding electronic waybill for dangerous goods, obtain the face information of the first waybill filing practitioner associated with the waybill, and record it as the second face information

Compare the first face information and the second face information;

If the comparison fails, the first face information will be searched in the photo library of the professional qualification certificate corresponding to the professional qualification in the transportation industry. If the search is successful, the server will issue a first medium-risk early warning; if the search fails, the server will Issue the first high-risk warning.
The method of claim 3, further comprising:

The server extracts the face information of the escort from the short video, which is recorded as the third face information;

Judging whether there is an electronic waybill for dangerous goods corresponding to the short video in the business management system of the server;

If there is a corresponding electronic waybill for dangerous goods, obtain the face information of the second waybill filing practitioner associated with the waybill, and record it as the fourth face information;

comparing the third face information and the fourth face information;

If the comparison fails, the third face information will be searched in the photo library of the professional qualification certificate with the corresponding professional qualification in the transportation industry. If the search is successful, the server will issue a second medium-risk early warning; if the search fails, the server will A second high-risk warning was issued.
The method according to claim 3 or 4, wherein if there is no electronic waybill for dangerous goods corresponding to the short video in the business management system of the server, the server sends a third medium-risk early warning.
The method according to claim 5, wherein after the server issues a third medium-risk early warning, the first face information or the third face information in the transportation industry has a corresponding qualification certificate photo library Search, if the search fails, the server issues a third high-risk early warning.
The method according to claim 3 or 4, wherein judging whether there is an electronic waybill for dangerous goods corresponding to the short video in the business management system of the server comprises:

According to the shooting time information of the short video, the unique identification information of the camera that shoots the short video, the corresponding relationship information between the camera and the vehicle, and the practitioner qualification information in the business management system, business license information, dangerous goods The vehicle road transport certificate information and the dangerous goods electronic waybill information determine whether there is a dangerous goods electronic waybill corresponding to the short video in the business management system of the server.
The method according to claim 3 or 4, wherein the server analyzes whether the driver and/or the escort has smoking behavior according to the short video; if there is smoking behavior, and the goods associated with the vehicle If the category information is a high-risk flammable category, the server issues a fourth high-risk early warning.
The method according to any one of claims 3-8, characterized in that, after the server issues various types of early warning, an early warning record is generated and stored on the server; the early warning record includes the time of the early warning, the type of the early warning , license plate number, license plate color, name of the company/institution, credit code of the unit, industrial and commercial registration address, name/code of the issuing authority, and the storage address path information of the corresponding short video on the server.