CN112329753B - Method for intelligently auditing safety inspection brake video of motor vehicle - Google Patents

Method for intelligently auditing safety inspection brake video of motor vehicle Download PDF

Info

Publication number
CN112329753B
CN112329753B CN202110015470.5A CN202110015470A CN112329753B CN 112329753 B CN112329753 B CN 112329753B CN 202110015470 A CN202110015470 A CN 202110015470A CN 112329753 B CN112329753 B CN 112329753B
Authority
CN
China
Prior art keywords
image
brake
vehicle
video
color space
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110015470.5A
Other languages
Chinese (zh)
Other versions
CN112329753A (en
Inventor
熊信信
熊奎
秦浩
刘耀祖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanchang Vkeline Information Technology Co ltd
Original Assignee
Nanchang Vkeline Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanchang Vkeline Information Technology Co ltd filed Critical Nanchang Vkeline Information Technology Co ltd
Priority to CN202110015470.5A priority Critical patent/CN112329753B/en
Publication of CN112329753A publication Critical patent/CN112329753A/en
Application granted granted Critical
Publication of CN112329753B publication Critical patent/CN112329753B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G06V20/42—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/08—Detecting or categorising vehicles

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention relates to the technical field of motor vehicle security check, and discloses a method for intelligently checking a motor vehicle security check brake video, which comprises the following two steps after a video to be checked is obtained: firstly, checking a vehicle to be checked in a video, decomposing the video into image frames, detecting a license plate in the image frames by using a neural network model, and identifying the license plate so as to obtain vehicle identity information; and secondly, checking the brake inspection in the video, wherein the checking comprises the vehicle brake inspection time length, the brake wheel rotation time length and the brake tail lamp lighting time length. And after the results of all the tests are obtained, comparing the results with the national standard, judging that the products are qualified if the national standard is met, and otherwise, judging that the products are unqualified. Compared with the traditional manual inspection mode, the invention uses the manual intelligent technology for auditing, thereby greatly saving the human resources; the vehicle-checking system has higher checking speed, greatly improves checking efficiency, reduces waiting time of vehicle owners, and improves vehicle-checking experience of the vehicle owners.

Description

Method for intelligently auditing safety inspection brake video of motor vehicle
Technical Field
The invention belongs to the technical field of motor vehicle security check, and particularly relates to a motor vehicle security check braking video intelligent checking method based on deep learning.
Background
In recent years, with the improvement of the economic level of China, the number of motor vehicles in China is greatly increased year by year, and the security inspection of the motor vehicles can be a life line of the motor vehicles. In particular, the brake test of a motor vehicle is an important part of the test of a motor vehicle. The qualification of the brake device of a motor vehicle is often linked to the lives of the driver and the passengers of the motor vehicle. Therefore, the country pays great attention to the security inspection of the motor vehicles, and a management department is specially set up to supervise and check the security inspection. With the rapid increase of the number of motor vehicles in China, the task of a traffic control supervision department is increasingly heavy.
The requirements for the video supervision of the brake inspection are provided according to general technical conditions of a motor vehicle safety technical inspection supervision system (GA 1186-2014) and motor vehicle safety technical inspection projects and methods (GB 21861-2014): 1. automatically identifying the license plate number of the detected vehicle in the video, and comparing and judging the license plate number with the registration data of a vehicle inspection system; 2. the brake test duration is required to meet the requirements in GB 21861-2014, and the brake test durations are different according to different vehicle types, use properties and use properties; 3. during brake inspection, the wheels of the corresponding axle can be seen in a rotating state in an inspection video, and the duration time is not less than 3 seconds. 4. The inspection video should be able to see that the brake tail lamp of the corresponding shaft is lighted, and the lighting time should not be less than 3 seconds.
The traditional video auditing needs manual auditing for auditors with familiar auditing standards, the auditors need to carefully watch videos uploaded by an inspection mechanism, whether places which do not accord with the relevant standards exist in the videos or not is checked, and if the places do not accord with the relevant standards, the videos are returned to the inspection mechanism to require re-inspection. The method has low efficiency and high requirement on auditors, and the requirements of vehicle security inspection video audit must be familiar with. Generally speaking, the video recording time is long and the number of motor vehicles is large, so that the problem of low efficiency of manual review generally results in the need of larger human resources. Therefore, aiming at the situation, the invention provides a method for intelligently auditing the safety inspection brake video of the motor vehicle according to the industrial standard and by combining the deep learning and artificial intelligence technology which is currently developed, thereby realizing the automatic and intelligent supervision and audit of the brake inspection video.
Disclosure of Invention
When the motor vehicle carries out safety technical inspection, it is for short: and (6) security inspection. The management department needs to check the detection video of the vehicle, and requires the detection station to recall the re-inspection for the vehicle which does not meet the checking requirement. The invention provides an intelligent auditing method based on deep learning aiming at auditing of a motor vehicle brake inspection video in security inspection, which assists an auditor in auditing, thereby greatly saving human resources.
The technical scheme of the invention is realized as follows:
a method for intelligently auditing a security check brake video of a motor vehicle comprises the following steps:
s1, acquiring a video to be audited, and decomposing the video into image frames;
s2, checking the vehicle to be checked in the video: detecting and identifying the license plate in the image frame by using a neural network model, and comparing the identification result with the registered license plate so as to obtain vehicle identity information;
s3, checking braking inspection in the video, comprising the following steps:
s31, detecting and identifying the vehicle position and the wheel position in the image frame by using the neural network model, and judging the states of the vehicle and the wheel by using a frame difference method;
the frame difference method specifically comprises the following steps: recording the first in a video sequence
Figure DEST_PATH_IMAGE002
Frame and second
Figure DEST_PATH_IMAGE004
The frame images are respectively
Figure DEST_PATH_IMAGE006
And
Figure DEST_PATH_IMAGE008
the gray values of the corresponding pixel points of the two frames are respectively recorded as
Figure DEST_PATH_IMAGE010
And
Figure DEST_PATH_IMAGE012
subtracting the gray values of the corresponding pixel points of the two frames of images, and taking the absolute value of the gray values to obtain a difference image
Figure DEST_PATH_IMAGE014
Namely:
Figure DEST_PATH_IMAGE016
setting a threshold value
Figure DEST_PATH_IMAGE018
For difference image
Figure DEST_PATH_IMAGE014A
Performing binarization processing to each pixel point in the difference image and threshold value
Figure DEST_PATH_IMAGE018A
Comparing, and when the gray value of the pixel point is larger than the threshold value
Figure DEST_PATH_IMAGE018AA
Setting the gray value of the point to be 255, and when the gray value of the pixel point is less than or equal to the threshold value
Figure DEST_PATH_IMAGE018AAA
Setting the gray value of the point to be 0 to finally obtain a binary image
Figure DEST_PATH_IMAGE024
(ii) a Wherein, the point with the gray value of 255 is the foreground point, the point with the gray value of 0 is the background point, namely:
Figure DEST_PATH_IMAGE026
for images
Figure DEST_PATH_IMAGE024A
Performing connectivity analysis to remove some edges and scattered regions, and finally obtaining an image containing a complete moving target
Figure DEST_PATH_IMAGE029
From the images obtained
Figure DEST_PATH_IMAGE029A
The number of the foreground points in the middleThreshold value
Figure DEST_PATH_IMAGE032
The relationship between the vehicle motion static state and the wheel rotation static state, and the threshold value
Figure DEST_PATH_IMAGE032A
Is an empirical value for judging motion and stillness according to the number of foreground spots, the number of the current scenery spots is more than
Figure DEST_PATH_IMAGE032AA
Judging the state of the device to be a motion state when the device is in the motion state, otherwise, judging the device to be a static state;
s32, calculating the brake inspection time length of the vehicle to be detected on the brake table body according to the image frame number from the first rotation start of the wheel to the last rotation stop of the wheel, calculating the wheel rotation time length in a certain process according to the image frame number from the rotation of the wheel to the stop of the wheel, regarding the process as a possible process of braking the wheel rotation, and recording the wheel rotation time length in the wheel rotation process of each time;
when only one possible braking wheel rotation process exists, the process is a real braking wheel rotation process, and the corresponding wheel rotation time length is the braking wheel rotation time length;
when a plurality of possible brake wheel rotation processes exist, screening out a real brake wheel rotation process from the plurality of possible brake wheel rotation processes by combining a first image frame for lighting the brake tail lamp and the lighting time length of the brake tail lamp, wherein the corresponding wheel rotation time length is the brake wheel rotation time length;
s33, detecting and identifying the brake tail lamp in the image frame by using a neural network model, converting the RGB color space image of the brake tail lamp into an LAB color space image or a YUV color space image, extracting an L channel or a Y channel image, judging whether the brake tail lamp is lighted, recording the number of continuously lighted frames of the brake tail lamp, and obtaining the lighting time of the brake tail lamp;
s4, comparing the obtained vehicle brake inspection time length, the brake wheel rotation time length and the brake tail lamp lighting time length with the corresponding national standard time length, judging whether the standards are met, and judging whether the safety inspection brake video of the motor vehicle is qualified or not.
As a further scheme of the invention: the neural network model is any one or combination of a plurality of RNN model, CNN model, CRNN model, YOLO model, SSD model and DenseNet model, and is obtained by training images marked with various license plates, vehicles, wheels and brake tail lamps in a large quantity, so that the license plates, the vehicle positions, the wheel positions and the brake tail lamps can be accurately detected and identified.
As a further scheme of the invention: in S6, extracting an image of an L channel in the image of the LAB color space, analyzing the brightness value of the image, and further judging whether the motor vehicle brake tail lamp is in a lighting or closing state; wherein, converting the image of RGB color space into the image of LAB color space requires converting the image of RGB color space into the image of XYZ color space first, that is:
Figure DEST_PATH_IMAGE036
after the image of the XYZ color space is obtained, the image of the XYZ color space can be converted into the image of the LAB color space, and the conversion formula between the two is as follows:
Figure DEST_PATH_IMAGE038
wherein
Figure DEST_PATH_IMAGE040
，
Figure DEST_PATH_IMAGE042
，
Figure DEST_PATH_IMAGE044
Defaults are 95.047, 100.0, 108.883, function
Figure DEST_PATH_IMAGE046
The following were used:
Figure DEST_PATH_IMAGE048
。
as a further scheme of the invention: in S6, extracting an image of an L channel in the image of the LAB color space, analyzing the brightness value of the image, and further judging whether the motor vehicle brake tail lamp is in a lighting or closing state; wherein, converting the image of RGB color space into the image of LAB color space requires converting the image of RGB color space into the image of XYZ color space first, that is:
Figure DEST_PATH_IMAGE036A
after the image of the XYZ color space is obtained, the image of the XYZ color space can be converted into the image of the LAB color space, and the conversion formula between the two is as follows:
Figure DEST_PATH_IMAGE038A
wherein
Figure DEST_PATH_IMAGE040A
，
Figure DEST_PATH_IMAGE042A
，
Figure DEST_PATH_IMAGE044A
Defaults are 95.047, 100.0, 108.883, function
Figure DEST_PATH_IMAGE046A
The following were used:
Figure DEST_PATH_IMAGE048A
。
as a further scheme of the invention: calculating project inspection by acquiring frame rate of video and total frame number of project inspectionThe time length of the inspection is obtained first, and the frame rate of the inspection video is obtained first
Figure DEST_PATH_IMAGE057
Total number of frames for project inspection
Figure DEST_PATH_IMAGE059
Then the duration of the item check is:
Figure DEST_PATH_IMAGE061
wherein, the items are: vehicle braking, brake wheel turning, and brake tail light illumination.
The method for intelligently auditing the safety inspection braking videos of the motor vehicle has the following beneficial effects:
1. compared with the traditional manual inspection mode, the invention uses the manual intelligent technology for auditing, thereby greatly saving the human resources.
2. Compared with the traditional manual inspection mode, the method has the advantages that the auditing speed is higher, the auditing efficiency is greatly improved, the waiting time of the car owner is reduced, and the car inspection experience of the car owner is improved.
3. Usually, when a person carries out a lot of repetitive labor, some errors occur inevitably, and compared with the human, the computer can always keep a long-term efficient and stable state. Therefore, by using the method for auditing, the auditing errors caused by manual auditing can be effectively avoided, and the method has higher accuracy and stability.
Drawings
FIG. 1 is a schematic flow chart of a method for intelligently auditing a security check brake video of a motor vehicle according to the present invention;
FIG. 2 is a schematic diagram illustrating a process of determining a target state by a frame difference method according to the present invention;
FIG. 3 is a schematic flow chart of a brake tail lamp state judgment (LAB) according to the present invention;
fig. 4 is a schematic flow chart of the brake tail lamp state determination (YUV) according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments of the present invention.
Example 1:
the invention discloses an intelligent checking method for a security check braking video of a motor vehicle, which comprises the following steps:
as shown in fig. 1, after a video to be audited is acquired, the method includes two steps: first, the inspection vehicle in the video is inspected: decomposing a video into image frames, detecting license plates in the image frames by using a neural network model, identifying the license plates, comparing an identification result with a registered license plate to obtain vehicle identity information, and confirming a vehicle to be inspected; and secondly, checking braking inspection in the video: the method comprises the steps of vehicle brake inspection time length, brake wheel rotation time length and brake tail lamp lighting time length.
Detecting and identifying a vehicle position and a wheel position in an image frame by using a neural network model, and judging states of the vehicle and the wheel by using a frame difference method;
calculating the brake inspection time length of the vehicle to be detected on the brake table body according to the image frame number from the first rotation starting of the wheel to the last rotation stopping of the wheel, calculating the wheel rotation time length in a certain process according to the image frame number from the rotation of the wheel to the stop of the wheel each time, regarding the process as a possible process of braking the wheel rotation, and recording the wheel rotation time length in the wheel rotation process each time;
when only one possible braking wheel rotation process exists, the process is a real braking wheel rotation process, and the corresponding wheel rotation time length is the braking wheel rotation time length;
when a plurality of possible brake wheel rotation processes exist, the real brake wheel rotation process is screened out from the plurality of possible brake wheel rotation processes by combining the first image frame of the lighting of the brake tail lamp and the lighting time length of the brake tail lamp, and the corresponding wheel rotation time length is the brake wheel rotation time length.
Detecting and identifying the brake tail lamp in the image frame by using a neural network model, converting an image of an RGB color space of the brake tail lamp into an image of an LAB color space or an image of a YUV color space, extracting an image of an L channel or a Y channel, judging whether the brake tail lamp is lightened, recording the number of continuously lightened frames of the brake tail lamp, and obtaining the lightening time of the brake tail lamp.
And finding out a brake wheel rotation process corresponding to the image frame according to the first image frame lighted by the brake tail lamp, wherein the brake wheel rotation process is the real brake wheel rotation process, and the wheel rotation duration corresponding to the image frame is the brake wheel rotation duration.
The vehicle brake inspection time length, the brake wheel rotation time length and the brake tail lamp lighting time length can be calculated through the image frame number and the frame rate of the video. When there is only one possible braking wheel rotation process, the vehicle braking check period is equal to the braking wheel rotation period.
Wherein the above-described vehicle state determination, wheel state determination, and brake tail lamp state determination are respectively expressed as: whether the vehicle is stationary or moving, whether the wheels are rotating, whether the brake tail lights are on or off.
The Neural Network model used in the present application may be any one or a combination of several of RNN (Recurrent Neural Network) model, CNN (Convolutional Neural Network) model, CRNN (Convolutional Recurrent Neural Network) model, YOLO (young Only lok one, which has been updated by 5 versions: YOLO 1-YOLO 5, YOLO 4, which is a fourth version of YOLO Network), ssd (single box detector) model and densneet (denseley Connected weighted Network) model, and the Neural Network model used in the present application includes but is not limited to the above-mentioned ones, and the Neural Network model can be obtained by using a large number of images of vehicle, vehicle license plate, tail lamp, and vehicle position accurately identified by using a vehicle license plate recognition model, and a vehicle tail lamp position recognition model, and accurately detecting the position of the vehicle, Wheel position, brake tail light. At present, a plurality of network models capable of detecting and identifying license plates, vehicles, wheels and brake tail lamps exist in the market, and the detection efficiency can be greatly improved.
As hereinbefore describedThe frame difference method is adopted to judge the state of the vehicle and the wheels, as shown in fig. 2. The frame difference method principle is as follows: recording the first in a video sequence
Figure DEST_PATH_IMAGE002A
Frame and second
Figure DEST_PATH_IMAGE004A
The frame images are respectively
Figure DEST_PATH_IMAGE006A
And
Figure DEST_PATH_IMAGE008A
the gray values of the corresponding pixel points of the two frames are respectively recorded as
Figure DEST_PATH_IMAGE010A
And
Figure DEST_PATH_IMAGE012A
subtracting the gray values of the corresponding pixel points of the two frames of images, and taking the absolute value of the gray values to obtain a difference image
Figure DEST_PATH_IMAGE014AA
Namely:
Figure DEST_PATH_IMAGE016A
setting a threshold value
Figure DEST_PATH_IMAGE018AAAA
For difference image
Figure DEST_PATH_IMAGE014AAA
Performing binarization processing to each pixel point in the difference image and threshold value
Figure DEST_PATH_IMAGE018AAAAA
Comparing, and when the gray value of the pixel point is larger than the threshold value
Figure DEST_PATH_IMAGE018AAAAAA
Setting the gray value of the point to be 255, and when the gray value of the pixel point is less than or equal to the threshold value
Figure DEST_PATH_IMAGE018AAAAAAA
Setting the gray value of the point to be 0 to finally obtain a binary image
Figure DEST_PATH_IMAGE024AA
(ii) a Wherein, the point with the gray value of 255 is the foreground point, the point with the gray value of 0 is the background point, namely:
Figure DEST_PATH_IMAGE026A
；
for images
Figure DEST_PATH_IMAGE024AAA
Performing connectivity analysis to remove some edges and scattered regions, and finally obtaining an image containing a complete moving target
Figure DEST_PATH_IMAGE029AA
From the images obtained
Figure DEST_PATH_IMAGE029AAA
The number and threshold of foreground points
Figure DEST_PATH_IMAGE032AAA
The relationship between the vehicle motion static state and the wheel rotation static state, and the threshold value
Figure DEST_PATH_IMAGE032AAAA
Is an empirical value for judging motion and stillness according to the number of foreground spots, the number of the current scenery spots is more than
Figure DEST_PATH_IMAGE032AAAAA
When the state is in motion, the state is judged, otherwise, the state is in static state.
The scheme obtains the frame rate of the video and the total item inspectionThe number of frames to calculate the duration of project inspection. First obtaining frame rate of inspection video
Figure DEST_PATH_IMAGE057A
Total number of frames for project inspection
Figure DEST_PATH_IMAGE059A
Then the duration of the item check is:
Figure DEST_PATH_IMAGE061A
wherein, the items are: vehicle braking, brake wheel turning, and brake tail light illumination.
For the vehicle brake tail lamp status check, as shown in fig. 3. According to the scheme, after the brake tail lamp is detected and identified by using a neural network model, an image (an image in an RGB color space) of the brake tail lamp is intercepted, and the image is converted into an image in an LAB color space. Extracting a brightness channel (namely L channel) image in the image of the LAB color space, analyzing the brightness value of the image, and further judging whether the motor vehicle brake tail lamp is in a lighting or closing state. Wherein, converting the image of the RGB color space into the image of the LAB color space requires converting the image of the RGB color space into the image of the XYZ color space first, that is:
Figure DEST_PATH_IMAGE036AA
after the image of the XYZ color space is obtained, the image of the XYZ color space can be converted into the image of the LAB color space, and the conversion formula between the two is as follows:
Figure DEST_PATH_IMAGE038AA
wherein
Figure DEST_PATH_IMAGE040AA
，
Figure DEST_PATH_IMAGE042AA
，
Figure DEST_PATH_IMAGE091
General defaults are 95.047, 100.0, 108.883, function
Figure DEST_PATH_IMAGE046AA
The following were used:
Figure DEST_PATH_IMAGE094
。
and checking the checked vehicle in the video through the neural network model, and calculating the vehicle braking checking time, the braking wheel rotation time and the lighting time of the braking tail lamp through the total frame number and the frame rate of the video. And after the results of all the tests are obtained, comparing the results with the national standard, judging that the results meet the national standard as qualified, and otherwise, judging that the results are unqualified.
Example 2:
for the vehicle brake tail lamp status check, as shown in fig. 4. According to the scheme, after the brake tail lamp is detected and identified by using a neural network model, an image (an image in an RGB color space) of the brake tail lamp is intercepted, and the image is converted into an image in a YUV color space. And extracting a brightness channel (namely Y channel) image in the YUV color space image, analyzing the brightness value of the image, and further judging whether the motor vehicle brake tail lamp is in a lighting or closing state. The formula for converting the image of the RGB color space into the image of the YUV color space is as follows:
Figure DEST_PATH_IMAGE096
。
the remaining steps were identical to those in example 1.
Example 3:
the intelligent checking practical operation process of the safety inspection brake video of the motor vehicle comprises the following steps:
1. firstly, a brake detection video of a vehicle to be checked and basic information of the vehicle to be checked are obtained through an interface.
2. The brake detection video is decoded into image frames using a video decoder. For video with a frame rate of 25fps, one frame of image is taken for every 5 frames for detection.
3. Marking the motor vehicles, license plates, tires and brake platforms in about 10 thousands of motor vehicle brake detection images, making the marked image data into a specific data set form, and training by using a Yolov4 network model to obtain a detection model capable of detecting the motor vehicles, license plates, tires and brake platforms in the images.
4. And (3) detecting the motor vehicles, license plates, tires and the braking table bodies in the image frames by using the detection model trained in the step (3), selecting the motor vehicle closest to the braking table body as the vehicle to be detected, and segmenting the license plates on the motor vehicles.
5. And (3) using about 5 thousands of license plate image data as a training data set, and training by using a convolutional neural network to obtain a license plate recognition model.
6. And (5) recognizing the segmented license plate by using the license plate recognition model obtained by training in the step (5), comparing the recognized license plate with the basic information of the vehicle to be checked obtained from the interface initially, and returning a result of inconsistent vehicle information if the recognized license plate is wrong.
7. After checking that the vehicle information is correct, selecting images of two frames before and after checking that the vehicle information is correct, calculating a union set of the two frames of images in which the tire is detected at the image position, segmenting an image of the union set area, and judging the tire state by using a frame error method.
8. When the tire is judged to be in a rotating state, whether the tire is positioned on the brake table body is judged.
9. And when the tire is judged to be in a rotating state, dividing the brake tail lamp, converting the image of the brake tail lamp from the RGB color space into the image of the LAB color space, and judging whether the brake tail lamp is lightened or not by using an image brightness algorithm of the LAB color space.
10. The process of detecting the braking is determined according to the fact that the wheel rotates for the first time to be static for the last time (the wheel is positioned on the braking table body); the wheel rotation process is a wheel rotation process when the wheel rotates to be static; and finding out the real rotation process of the brake wheel according to the image frame of the lighting of the brake tail lamp, wherein the corresponding wheel rotation duration is the brake wheel rotation duration. The duration of the three processes is counted according to the method described above and compared with the duration of the national standard to determine whether the video meets the standard, thereby judging whether the video is qualified.
Thus, the object of the present invention is accomplished.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims (5)

1. A method for intelligently auditing a security check brake video of a motor vehicle is characterized by comprising the following steps:
s1, acquiring a video to be audited, and decomposing the video into image frames;
s2, checking the vehicle to be checked in the video: detecting and identifying the license plate in the image frame by using a neural network model, and comparing the identification result with the registered license plate so as to obtain vehicle identity information;
s3, checking braking inspection in the video, comprising the following steps:
s31, detecting and identifying the vehicle position and the wheel position in the image frame by using the neural network model, and judging the states of the vehicle and the wheel by using a frame difference method;
the frame difference method specifically comprises the following steps: recording the first in a video sequence
Figure 558163DEST_PATH_IMAGE002
Frame and second
Figure 537620DEST_PATH_IMAGE004
The frame images are respectively
Figure 130407DEST_PATH_IMAGE006
And
Figure 587933DEST_PATH_IMAGE008
the gray values of the corresponding pixel points of the two frames are respectively recorded as
Figure 215354DEST_PATH_IMAGE010
And
Figure 100134DEST_PATH_IMAGE012
subtracting the gray values of the corresponding pixel points of the two frames of images, and taking the absolute value of the gray values to obtain a difference image
Figure 180216DEST_PATH_IMAGE014
Namely:
Figure 441433DEST_PATH_IMAGE016
setting a threshold value
Figure 923361DEST_PATH_IMAGE018
For difference image
Figure DEST_PATH_IMAGE019
Performing binarization processing to each pixel point in the difference image and threshold value
Figure 854408DEST_PATH_IMAGE018
Comparing, and when the gray value of the pixel point is larger than the threshold value
Figure 421787DEST_PATH_IMAGE018
Setting the gray value of the point to be 255, and when the gray value of the pixel point is less than or equal to the threshold value
Figure 221116DEST_PATH_IMAGE018
Setting the gray value of the point to be 0 to finally obtain a binary image
Figure DEST_PATH_IMAGE021
(ii) a Wherein, the point with the gray value of 255 is the foreground point, the point with the gray value of 0 is the background point, namely:
Figure DEST_PATH_IMAGE023
for images
Figure 432917DEST_PATH_IMAGE021
Performing connectivity analysis to remove some edges and scattered regions, and finally obtaining an image containing a complete moving target
Figure DEST_PATH_IMAGE025
From the images obtained
Figure 738127DEST_PATH_IMAGE026
The number and threshold of foreground points
Figure DEST_PATH_IMAGE028
The relationship between the vehicle motion static state and the wheel rotation static state, and the threshold value
Figure 917436DEST_PATH_IMAGE028
Is an empirical value for judging motion and stillness according to the number of foreground spots, the number of the current scenery spots is more than
Figure 5609DEST_PATH_IMAGE028
Judging the state of the device to be a motion state when the device is in the motion state, otherwise, judging the device to be a static state;
s32, calculating the brake inspection time length of the vehicle to be detected on the brake table body according to the image frame number from the first rotation start of the wheel to the last rotation stop of the wheel, calculating the wheel rotation time length in a certain process according to the image frame number from the rotation of the wheel to the stop of the wheel, regarding the process as a possible process of braking the wheel rotation, and recording the wheel rotation time length in the wheel rotation process of each time;
when only one possible braking wheel rotation process exists, the process is a real braking wheel rotation process, and the corresponding wheel rotation time length is the braking wheel rotation time length;
when a plurality of possible brake wheel rotation processes exist, screening out a real brake wheel rotation process from the plurality of possible brake wheel rotation processes by combining a first image frame for lighting the brake tail lamp and the lighting time length of the brake tail lamp, wherein the corresponding wheel rotation time length is the brake wheel rotation time length;
s33, detecting and identifying the brake tail lamp in the image frame by using a neural network model, converting the RGB color space image of the brake tail lamp into an LAB color space image or a YUV color space image, extracting an L channel or a Y channel image, judging whether the brake tail lamp is lighted, recording the number of continuously lighted frames of the brake tail lamp, and obtaining the lighting time of the brake tail lamp;
s4, comparing the obtained vehicle brake inspection time length, the brake wheel rotation time length and the brake tail lamp lighting time length with the corresponding national standard time length, judging whether the standards are met, and judging whether the safety inspection brake video of the motor vehicle is qualified or not.
2. The method for intelligently auditing the security check brake videos of motor vehicles according to claim 1, wherein the neural network model is any one or a combination of RNN model, CNN model, CRNN model, YOLO model, SSD model and DenseNet model, and is obtained by training images labeled with various license plates, vehicles, wheels and brake tail lamps in a large quantity, so that the license plates, the vehicle positions, the wheel positions and the brake tail lamps can be accurately detected and identified.
3. The method for intelligently auditing the safety inspection brake videos of motor vehicles according to claim 1 or 2, characterized in that in S6, an image of an L channel in an image of an LAB color space is extracted, the brightness value of the image is analyzed, and it is further judged whether a brake tail lamp of a motor vehicle is in an on or off state; wherein, converting the image of RGB color space into the image of LAB color space requires converting the image of RGB color space into the image of XYZ color space first, that is:
Figure DEST_PATH_IMAGE030
after the image of the XYZ color space is obtained, the image of the XYZ color space can be converted into the image of the LAB color space, and the conversion formula between the two is as follows:
Figure 524446DEST_PATH_IMAGE032
wherein
Figure DEST_PATH_IMAGE034
，
Figure 734978DEST_PATH_IMAGE036
，
Figure 526217DEST_PATH_IMAGE038
Defaults are 95.047, 100.0, 108.883, function
Figure 429799DEST_PATH_IMAGE040
The following were used:
Figure 990094DEST_PATH_IMAGE042
。
4. the method for intelligently auditing the safety inspection brake videos of motor vehicles according to claim 1 or 2, characterized in that in S6, an image of a Y channel in an image of a YUV color space is extracted, the brightness value of the image is analyzed, and then the motor vehicle brake tail lamp is judged to be in an on or off state; the formula for converting the image of the RGB color space into the image of the YUV color space is as follows:
Figure 574790DEST_PATH_IMAGE044
。
5. the method for intelligently auditing the security inspection and braking videos of motor vehicles according to claim 1 or 2, characterized in that the duration of the project inspection is calculated by acquiring the frame rate of the video and the total frame number of the project inspection, and the frame rate of the inspection video is acquired first
Figure 587745DEST_PATH_IMAGE046
Total number of frames for project inspection
Figure 345617DEST_PATH_IMAGE048
Then the duration of the item check is:
Figure 511150DEST_PATH_IMAGE050
wherein, the items are: vehicle braking, brake wheel turning, and brake tail light illumination.
CN202110015470.5A 2021-01-07 2021-01-07 Method for intelligently auditing safety inspection brake video of motor vehicle Active CN112329753B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110015470.5A CN112329753B (en) 2021-01-07 2021-01-07 Method for intelligently auditing safety inspection brake video of motor vehicle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110015470.5A CN112329753B (en) 2021-01-07 2021-01-07 Method for intelligently auditing safety inspection brake video of motor vehicle

Publications (2)

Publication Number Publication Date
CN112329753A CN112329753A (en) 2021-02-05
CN112329753B true CN112329753B (en) 2021-04-16

Family

ID=74302347

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110015470.5A Active CN112329753B (en) 2021-01-07 2021-01-07 Method for intelligently auditing safety inspection brake video of motor vehicle

Country Status (1)

Country Link
CN (1) CN112329753B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114693278B (en) * 2022-05-06 2022-10-04 刘建鑫 Intelligent auditing system and method for motor vehicle inspection supervision

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006014571A1 (en) * 2006-03-29 2007-10-04 Di Ke Opto-Electronics Co., Ltd Auxiliary testing device using method for visually testing e.g. brake system of vehicle, involves attaching auxiliary testing device with sensor head of image recording video recorder for recording images in testing device
CN108352125A (en) * 2015-09-21 2018-07-31 Cw与Sr投资有限公司 Method and apparatus for emergency braking performance test, assessment and/or driver training
CN110765961A (en) * 2019-10-29 2020-02-07 上海眼控科技股份有限公司 Vehicle braking state judgment method and device, computer equipment and storage medium
CN111696361A (en) * 2020-05-15 2020-09-22 公安部道路交通安全研究中心 Remote security check method, device and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018146346A1 (en) * 2017-02-07 2018-08-16 Centro De Experimentación Y Seguridad Vial Mapfre S.A. (Cesvimap) Test device

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102006014571A1 (en) * 2006-03-29 2007-10-04 Di Ke Opto-Electronics Co., Ltd Auxiliary testing device using method for visually testing e.g. brake system of vehicle, involves attaching auxiliary testing device with sensor head of image recording video recorder for recording images in testing device
CN108352125A (en) * 2015-09-21 2018-07-31 Cw与Sr投资有限公司 Method and apparatus for emergency braking performance test, assessment and/or driver training
CN110765961A (en) * 2019-10-29 2020-02-07 上海眼控科技股份有限公司 Vehicle braking state judgment method and device, computer equipment and storage medium
CN111696361A (en) * 2020-05-15 2020-09-22 公安部道路交通安全研究中心 Remote security check method, device and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
机动车安全技术检验监管系统存在的问题与对策;戴晓锋;《汽车与安全》;20180115;全文 *

Also Published As

Publication number Publication date
CN112329753A (en) 2021-02-05

Similar Documents

Publication Publication Date Title
CN107766821B (en) Method and system for detecting and tracking full-time vehicle in video based on Kalman filtering and deep learning
CN104732235B (en) A kind of vehicle checking method for eliminating the reflective interference of road at night time
CN111080620A (en) Road disease detection method based on deep learning
CN105913041A (en) Pre-marked signal lights based identification method
CN110991221B (en) Dynamic traffic red light running recognition method based on deep learning
US11657599B2 (en) Method for detecting appearance of six sides of chip multi-layer ceramic capacitor based on artificial intelligence
CN110348360A (en) A kind of examining report recognition methods and equipment
CN111861990B (en) Method, system and storage medium for detecting bad appearance of product
CN111080607B (en) Rolling bearing oil slinging fault detection method based on image recognition
CN111080614A (en) Method for identifying damage to rim and tread of railway wagon wheel
CN112001299B (en) Tunnel vehicle finger device and lighting lamp fault identification method
CN112329753B (en) Method for intelligently auditing safety inspection brake video of motor vehicle
CN112330961B (en) Traffic indication red, yellow and green light image identification method based on time sequence rule
CN106355180A (en) Method for positioning license plates on basis of combination of color and edge features
CN107967445A (en) A kind of car installs the intelligent checking system and method for skylight additional
CN111008554B (en) Deep learning-based method for identifying pedestrians without giving away in dynamic traffic zebra stripes
CN113657305B (en) Video-based intelligent detection method for black smoke vehicle and ringeman blackness level
CN115424217A (en) AI vision-based intelligent vehicle identification method and device and electronic equipment
CN103034997A (en) Foreground detection method for separation of foreground and background of surveillance video
KR20190119801A (en) Vehicle Headlight Alignment Calibration and Classification, Inspection of Vehicle Headlight Defects
CN112990350B (en) Target detection network training method and target detection network-based coal and gangue identification method
CN112132088B (en) Inspection point missing inspection identification method
CN114973169A (en) Vehicle classification counting method and system based on multi-target detection and tracking
CN116152674A (en) Dam unmanned aerial vehicle image crack intelligent recognition method based on improved U-Net model
CN107976319B (en) Intelligent detection system and method for car additionally provided with pedal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant