CN112991348A

CN112991348A - Vehicle chassis foreign matter detection method and device and storage medium

Info

Publication number: CN112991348A
Application number: CN202110565840.2A
Authority: CN
Inventors: 路通; 王文卓; 薛明龙; 赵智玉; 徐梅娟
Original assignee: Nanjing Soan Electronics Co ltd
Current assignee: Nanjing Soan Electronics Co ltd
Priority date: 2021-05-24
Filing date: 2021-05-24
Publication date: 2021-06-18

Abstract

The invention discloses a vehicle chassis foreign matter detection method, a vehicle chassis foreign matter detection device and a storage medium, wherein the detection method comprises the following steps: shooting to obtain an image of the vehicle bottom as a sample image, screening and establishing a data set of the image of the foreign matter at the vehicle bottom; converting an RGB mode channel into an HSL channel mode for the sample image, and extracting a brightness channel in the sample image; determining the grid size and the contrast limited threshold value of the two important hyperparameter histograms in balance at the maximum curvature of the image entropy curve; obtaining an adjusted brightness channel L' by utilizing two super-parameter optimal values; replacing the brightness channel in the original image by using the adjusted brightness channel L', and converting the image back to the RGB mode; and inputting the training picture into a detection model for training, and testing to obtain a vehicle bottom foreign matter detection result. The invention has high robustness, better adaptability to images of hundred million pixels, good image effect, noise reduction without influencing the image color, and better vehicle bottom foreign matter detection effect.

Description

Vehicle chassis foreign matter detection method and device and storage medium

Technical Field

The invention relates to the field of vehicle chassis foreign matter and image processing, in particular to a vehicle chassis foreign matter detection method and device based on hundred million pixel image self-adaptive dim light enhancement optimization.

Background

Over 20 years, the road infrastructure of China is rapidly developed, the road transport capacity is greatly improved, and the road transport capacity plays more and more important roles in the aspects of national economy growth and improvement of the living standard of people. Road vehicle transport still suffers from an inefficient supply compared to the ever increasing transport demand. With the further development of our country's economy, the demand for road vehicle transportation will continue to grow rapidly. The share of bulk goods and primary products in road vehicle transportation is in a descending trend, and the requirements on the transportation service quality and the service level are increasingly increased. The development of regional economy and the continuous improvement of highway infrastructure and vehicles have increased the demand for medium and long distance highway transportation, and highway freight is developing towards the direction of high speed, long distance and heavy load. The large-tonnage heavy-duty special transport vehicle is the main force of road transport vehicles in the future in China due to high speed safety and low unit transport cost. Vehicle security is very important to ensure transport safety. There is a need to detect whether a vehicle is loaded with a prohibited article of the state law provisions, such as a hazardous article like a firearm. In the conventional security inspection, whether contraband exists is judged mainly by generating images of the material, volume, quantity and the like of loaded goods through X-ray perspective scanning and according with a check list. However, the situation that the vehicle bottom carries contraband cannot be dealt with. Therefore, the detection of foreign matters at the bottom of the vehicle is an effective method for detecting the safety of the road.

The environment for detecting foreign bodies under the vehicle is usually not an ideal condition. Because the quantity of light entering the bottom of the vehicle is small, the scanning photographing is mostly carried out under the condition of dark light. In this case, the contrast of the image is low, which is not favorable for target detection, and therefore, it should be considered that the image is subjected to contrast equalization processing first. Histogram equalization is considered to be the most effective method for improving image contrast, and its basic idea is to mathematically readjust the luminance distribution of pixels so that the adjusted histogram has the maximum dynamic range. For images with lower contrast, the span of the image luminance histogram is smaller, and there is room for stretching to the two poles. Generally the image after equalization has a better visual effect. However, when the distribution of the pixel values of the image is not uniform, such as a dark area or a bright area in the image, the effect of processing these image areas is not ideal. Moreover, histogram equalization has the obvious disadvantage that noise of some dark areas is amplified, and information distribution of an image is influenced. Therefore, the requirement of image processing of complex environment at the bottom of the vehicle is not met. In addition, the samples collected by the vehicle bottom image are large images at the hundred million pixel level, and a flexible method which can adapt to complex scenes is needed to process the vehicle bottom image, so that the accuracy of vehicle bottom foreign matter detection is improved.

Disclosure of Invention

Aiming at the defects of the prior art, the invention provides a vehicle chassis foreign matter detection method, a vehicle chassis foreign matter detection device and a storage medium, wherein the vehicle chassis foreign matter detection method and the vehicle chassis foreign matter detection device have good vehicle bottom foreign matter detection effect.

In order to solve the technical problems, the invention adopts the following technical scheme:

a vehicle chassis foreign matter detection method comprises the following steps:

step 1, shooting and obtaining an image of the bottom of a vehicle as a sample image, screening and establishing a data set of a vehicle bottom foreign body image;

step 2, carrying out channel conversion on the sample image, converting an RGB mode channel into an HSL channel mode, and extracting a brightness channel in the HSL channel mode;

step 3, determining the optimal values of the grid size TileGridSize and the contrast limited threshold ClipLimit of the two important super-parameter histograms in the maximum curvature of the image entropy curve;

step 4, adjusting the brightness channel L of the image by using the two super-parameter optimal values to obtain an adjusted brightness channel L';

step 5, replacing the brightness channel in the original image by using the adjusted brightness channel L', and converting the image back to the RGB mode;

step 6, performing image layer color filtering mixing operation on the processed image and the original image, performing bilateral filtering, and outputting the processed image for foreign matter detection;

and 7, inputting the training picture into a YOLO-v3 detection model for training, and testing to obtain a vehicle bottom foreign matter detection result.

The step 2 comprises the following steps: the sample image is converted from the original RGB mode to the HSL mode using the following formula, and the luminance channel L therein is extracted.

And step 3, realizing self-adaptive determination of the important super parameters. The method comprises the following steps:

and 3-1, changing the contrast limited threshold ClipLimit from 0 to 0.1 while the grid size TileGridSize of histogram equalization is 8x8, and calculating an entropy curve. Fitting methods such as Fitoutputfun and fminsearch in Matlab are used for carrying out optimal fitting on the nonlinear entropy curve. Since the entropy curve is not monotonic, the fitting equation is as follows:

step 3-2, setting the contrast limited threshold value as

Entropy of y₁Suppose that

And y₁Is twice differentiable, curvature kappa₁Is defined as:

in the formula (I), the compound is shown in the specification,

is composed of

The first derivative of (a);

is composed of

The second derivative of (a);

step 3-3, taking a point with the maximum curvature on the entropy and contrast limited threshold curve as a limited threshold ClipLimit:

step 3-4, taking the contrast limited threshold determined in step 3-3, changing the block size from 2x2 to 32x32 to calculate entropy:

in the formula (I), the compound is shown in the specification,

grid size, y, for histogram equalization₂Is entropy, κ₂Is a curvature;

is composed of

The first derivative of (a);

is composed of

The second derivative of (a);

taking the point with the maximum curvature on the grid size curve of entropy and histogram equalization as the grid size of histogram equalization:

and 4, automatically acquiring the optimal value of the grid size TiLEGridSize and the contrast limited threshold ClipLimit by utilizing the histogram equalization, and adjusting the L channel of the brightness channel.

And 4-1, counting the proportion of each gray level in the total number of the pixels on the L channel extracted in the step 2 by taking the grid size TileGridSize of the hyperparametric histogram equalization in the step 3-4 as a unit, and marking as Pi.

And 4-2, dividing the three gray intervals according to the gray level according to the calculation result of the 4-1, so that the number of the pixel points in the three intervals is approximately equal.

And 4-3, calculating an accumulated value S (i) of the histogram probability until the last gray level, wherein the sum is 1. And (5) counting to obtain a cumulative distribution function.

And 4-4, cutting the gray level histogram into a preset value to limit amplification according to the contrast limited threshold determined in the step 3-3, uniformly distributing the cut part to the gray level with the amplitude value smaller than the contrast limited threshold ClipLimit in the grid interval balanced by the histogram, realizing local contrast enhancement through the following formula, and limiting the slope of the neighborhood cumulative distribution function so as to limit the slope of the conversion function.

Wherein, let W be the current window, m_i,jAverage gray scale of pixels in grid for histogram equalization in window W, k is coefficient set by experiment, x_i,jM and n are the length and width of the window W, respectively, for the pixel at the coordinate (i, j) position.

And 4-5, calculating a neighborhood cumulative distribution function CDF according to the clipped gray level histogram.

And 4-6, traversing and carrying out the following operation on each image block taking the grid size TiLEGridSize of histogram equalization as a unit, and carrying out linear interpolation between blocks so as to map the image blocks to a gray space. For example, a pixel of a point P is obtained, and interpolation is performed in the x-axis direction to obtain:

and then carrying out interpolation in the y-axis direction to obtain:

wherein x and y are the horizontal and vertical coordinates of the pixel position,

the coordinates are

，

The coordinates are

，

The coordinates are

，

The coordinates are

，

The coordinates are

，

The coordinates are

F (P) after interpolation between blocks for P point, the value of the position on L channel is recorded as

Therefore, the luminance channel L is converted into the luminance channel L after the interpolation between the whole picture blocks is finished

。

After the whole picture is calculated by the calculating method, the adjusted image brightness channel L' is obtained.

And 5, replacing the brightness channel in the original image by using the adjusted brightness channel L', and converting the adjusted image back to the RGB mode by using the following formula:

for each color vector

，

Wherein (h, s, l) are hue, saturation and brightness in HSL space, h is in intellectual education [0, 360 ], and s, l is in value range [0, 1 ]]Where (r, g, b) are the three primary colors red, green, blue in RGB space, and r, g, and b are also in the value range [0, 1 ]]In (1). The ratio of p, q,

，

，

，

is the intermediate variable calculated by (h, s, l) for finding (r, g, b).

And 6, performing brightness fusion on the image output in the step 5 and the original image, performing bilateral filtering, protecting edge information, and filtering noise signals in the image to obtain an input image finally used for the target detection model. A

In step 7, the image output in step 6 is input into a YOLO-v3 model for training. Through the test, can realize finally detecting out the position and the classification of vehicle bottom foreign matter.

Compared with the prior art, the invention has the following effective effects:

1. since the two super-parameters of the grid size and the contrast-limited threshold value mainly control the quality in the contrast-limited adaptive histogram equalization process, the setting of the values is very important. We are different from the former histogram equalization on R, G, B channels, but do histogram equalization adjustment on L (luminance) channel after converting it to HSL space, because the modification of luminance channel is more suitable for the brightening enhancement requirement of dark image. Adjusting a brightness channel L of the image by using two super-parameter optimal values of the grid size and the contrast limited threshold value to obtain an adjusted brightness channel L'; these two super-parameters mainly control the image quality during histogram equalization.

2. The former method needs manual setting and testing to select two super-parameter optimal values, and here, the two super-parameter optimal values are selected through an entropy curve

And the hyper-parameter value corresponding to the maximum curvature point automatically acquires the optimal hyper-parameter value, so that the contrast-limited adaptive histogram equalization process is quicker and more convenient.

3. And performing layer color filtering mixing operation on the processed RGB mode image and the original image, performing bilateral filtering, and outputting the processed image for foreign matter detection. On the basis of not losing other detailed information of the original image, the characteristic of brightness enhancement is fused into the original image through the processed RGB mode image. Through bilateral filtering, the noise is suppressed while edge information is kept. The output picture is clearer in visibility and can be better used for foreign matter detection.

5. According to the low illumination characteristic of the foreign matter detection sample, the YOLO foreign matter detection is not directly carried out, but the sample is subjected to dark light enhancement firstly, so that the sample is clearer in visibility. The bright image after the dark light enhancement is more suitable as an input to the YOLO to detect a foreign substance than the dark image before the dark light enhancement. The increased dim light enhancement operation before foreign matter detection improves the final detection effect position accuracy and classification accuracy.

Drawings

The foregoing and other advantages of the invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings.

FIG. 1 is a comparison diagram before and after a vehicle bottom picture is converted from an RGB mode channel to an HSL channel mode, wherein a is an original RGB channel mode diagram, and b is an HSL channel mode diagram;

FIG. 2 is a flow chart of a vehicle chassis foreign object detection method of the present invention;

FIG. 3 is a process of adaptively determining two important hyperparameters of a contrast-limited adaptive histogram equalization method;

FIG. 4 is a graph of original RGB channel pattern, HSL channel pattern, and finally processed image contrast for final detection of alien materials, wherein a is the graph of original RGB channel pattern, b is the graph of HSL channel pattern, and c is the graph of processed RGB channel pattern;

FIG. 5 shows the detection effect before and after processing the picture, wherein a is the detection effect before processing and b is the detection effect after processing;

fig. 6 is a diagram illustrating linear interpolation between blocks.

Detailed Description

The invention is further explained below with reference to the drawings and the embodiments.

The invention relates to a method for detecting foreign matters on a vehicle chassis, which is suitable for vehicles such as trucks, trains and the like.

Example 1

Taking a truck as an example, referring to a flow chart of the method of the invention shown in FIG. 2, the specific method comprises the following steps:

1. collecting vehicle bottom image

Shooting a picture of the foreign matters at the bottom of the vehicle, and storing the picture of the foreign matters at the bottom of the vehicle in an image library; and screening the acquired images of the vehicle bottom foreign matters in the image library, and acquiring the images of the vehicle bottom sample to be used for establishing a vehicle bottom foreign matter image sample library.

2. The sample image is channel converted from an RGB mode channel to an HSL channel mode as shown in fig. 1. And extracts a luminance channel therein.

3. Two important hyperparameters of the contrast-limited adaptive histogram equalization method are determined adaptively.

As shown in fig. 3, the optimal value of the grid size TileGridSize contrast limited threshold ClipLimit for two important hyperreference histogram equilibria is determined at the maximum curvature of the image entropy curve. The method comprises the following specific steps:

step 3-2, setting the contrast limited threshold value as

Entropy of

Suppose that

And

is twice differentiable, curvature kappa₁Is defined as:

in the formula (I), the compound is shown in the specification,

is composed of

The first derivative of (a);

is composed of

The second derivative of (a);

in the formula (I), the compound is shown in the specification,

for the grid size of the histogram equalization,

is entropy, κ₂Is a curvature;

is composed of

The first derivative of (a);

is composed of

The second derivative of (a);

2. and adjusting the brightness channel L of the image by using the two super-parameter optimal values to obtain an adjusted brightness channel L'.

Adjusting the L channel of the brightness channel by using the grid size TiLEGridSize and the contrast limited threshold ClipLimit optimal value of the histogram equalization automatically acquired in the step 3, wherein the specific steps of the adjustment comprise:

Wherein m is_i,jFor the mean gray level of the pixels in the histogram equalized grid, k is the contrast limited threshold ClipLimit, x determined in step 3-4_i,jIs the pixel at coordinate (i, j) position.

and then carrying out interpolation in the y-axis direction to obtain:

and after the whole picture is calculated by the calculating method, obtaining an adjusted image brightness channel.

2. And replacing the brightness channel in the original image by using the adjusted brightness channel L', and converting the image back to the RGB mode.

3. And performing image layer color filtering mixing operation on the processed image and the original image, performing bilateral filtering, and outputting the processed image for foreign matter detection. As shown in fig. 4, the original RGB channel pattern diagram, the HSL channel pattern diagram, and the finally processed image contrast for the foreign object detection are shown.

4. And (4) inputting the image output in the step (6) into a YOLO-v3 model for training. Through the test, can realize finally detecting out the position and the classification of vehicle bottom foreign matter. As shown in fig. 5, the detection effect before and after the picture processing is demonstrated. Where knife represents the detected foreign object classified as a knife, axe represents the detected foreign object classified as an axe, and gun represents the detected foreign object classified as a gun. The percentage represents the accuracy of the detection. As can be seen from fig. 5a, the picture before the dark light enhancement is performed is taken as an input, the positions of the axe and the gun are not detected by the foreign object detection, only the position of the knife is detected, and the detected position of the knife has a classification error, the knife is wrongly classified as the axe, and the position of the knife is also wrongly detected (actually, no knife is left). After dark light enhancement, as shown in fig. 5b, the foreign object detection successfully detects all three foreign object positions: the knife, the gun and the axe are classified correctly, the detection accuracy is 99%, 99% and 98%, and the detection accuracy is improved. Compared with the original image, the detection area of the processed image is more accurate, and the foreign matter classification is more accurate.

Example 2

The present invention also provides a vehicle chassis foreign matter detection apparatus, including:

the image acquisition module is used for acquiring an image of the bottom of the vehicle;

and the foreign matter detection module adopts the detection model trained by the detection method in the embodiment 1 to detect foreign matters in the vehicle bottom image acquired by the image acquisition module.

Example 3

The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium, and which may also be a volatile computer-readable storage medium, having stored therein instructions, which, when executed on a computer, cause the computer to perform the steps of the method for detecting a foreign object under a vehicle bottom according to embodiment 1.

It is clear to those skilled in the art that the technical solution of the present invention, which is essential or part of the technical solution contributing to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Claims

1. A vehicle chassis foreign matter detection method is characterized by comprising the following steps:

step 1, shooting and obtaining an image of the vehicle bottom as a sample image, screening and establishing a data set of a vehicle bottom foreign body image;

step 3, determining the grid size and the contrast limited threshold value of the histogram equalization at the maximum curvature of the image entropy curve, wherein the two important optimal values exceed the parameters;

step 4, adjusting the brightness channel L of the image by using two super-parameter optimal values of the grid size and the contrast limited threshold value to obtain an adjusted brightness channel L';

step 5, replacing the brightness channel in the original image by using the adjusted brightness channel L', and converting the image from the HSL channel mode to the RGB mode;

step 6, performing layer color filtering mixing operation on the processed RGB mode image and the original image, performing bilateral filtering, and outputting the processed image for foreign matter detection;

step 7, inputting the training pictures into the detection model for training;

and 8, testing by using the trained detection model to obtain a vehicle bottom foreign matter detection result.

2. The method of claim 1, wherein step 2 converts the sample image from RGB mode channel to HSL channel mode and extracts the luminance channel L therein using the following formula:

wherein r, g and b are red, green and blue coordinates of a color, respectively, and the values of r, g and b are real numbers between 0 and 1; h. s and l are values in HSL space; h is the hue angle of the angle; s and l are saturation and brightness, respectively.

3. The method of claim 1, wherein step 3 comprises:

step 3-1, fitting an entropy curve in a limited threshold range of grid size and contrast of the set histogram equalization;

in the formula (I), the compound is shown in the specification,c ₁、c ₁、λ ₁、λ ₂are all fitting parameters;

step 3-2, setting the contrast limited threshold value asx ₁Entropy ofy ₁Curvature kappa₁Is defined as:

in the formula (I), the compound is shown in the specification,

is composed ofx ₁The first derivative of (a);

is composed of x ₁The second derivative of (a);

step 3-3, taking the point with the maximum curvature on the entropy and contrast limited threshold curve as a limited threshold:

in the formula (I), the compound is shown in the specification,x ₂grid size, y, for histogram equalization₂Is entropy, κ₂Is a curvature;

is composed ofx ₂The first derivative of (a);

is composed ofx ₂The second derivative of (a);

。

4. the method of claim 3, wherein step 4 comprises:

step 4-1, taking the grid size of the hyperparametric histogram equalization in the step 3-4 as a unit, counting the proportion of each gray level in the total number of the pixels on the L channel extracted in the step 2, and marking as Pi;

step 4-2, dividing the three gray intervals according to the gray level according to the calculation result of the step 4-1, so that the number of pixel points in the three intervals is approximately equal;

4-3, calculating an accumulated value S (i) of the histogram probability until the last gray level, wherein the sum is 1, and counting to obtain an accumulated distribution function;

4-4, cutting a gray level histogram into a preset value to limit amplification according to the contrast limited threshold determined in the step 3-3, and uniformly distributing the cut part to the gray level with the amplitude value smaller than the contrast limited threshold in the grid interval balanced by the histogram;

4-5, calculating a neighborhood cumulative distribution function according to the clipped gray level histogram;

step 4-6, traversing each image block with the grid size of histogram equalization as a unit, performing inter-block linear interpolation on each image block, and obtaining an adjusted image brightness channel after the whole image is calculated

。

5. The method of claim 4, wherein the inter-block linear interpolation is performed on each image block by: for a pixel P (x, y), its image block is its image block ofQ ₁₁、 Q ₁₂ 、Q ₂₁AndQ ₂₂an enclosing region; firstly, interpolation is carried out in the x-axis direction to obtain:

and then carrying out interpolation in the y-axis direction to obtain:

wherein x and y are the horizontal and vertical coordinates of the pixel position,Q ₁₁the coordinates are (x ₁，y ₁），Q ₁₂The coordinates are (x ₁，y ₂），Q ₂₁The coordinates are (x ₂，y ₁），Q ₂₂The coordinates are (x ₂，y ₂），R ₁The coordinates are (x，y ₁），R ₂The coordinates are (x，y ₂) And f (P) after the P point is subjected to the inter-block interpolation, the value of the position on the L channel is marked as P ', so that the brightness channel L is converted into L' after the inter-block interpolation of the whole picture is finished.

6. The method of claim 4, wherein step 5 replaces the luminance channel in the original image with the adjusted luminance channel L' and converts the adjusted image back to RGB mode using the following formula:

for each color vector

，

In the formula (I), the compound is shown in the specification,

finger-shaped

One of (1)

Is the intermediate variable calculated by (h, s, l) for finding (r, g, b).

7. The method of claim 1, wherein in step 7, the detection model is a YOLO-v3 model; and (4) inputting the image output in the step (6) into a YOLO-v3 model for training.

8. The method according to claim 4, characterized in that in step 4-4, the local contrast enhancement is achieved by the following formula:

wherein, let W be the current window,

the average gray level of the pixels in the grid for histogram equalization in the window W, k is a coefficient set experimentally,

is a coordinate (i，j) The enhanced pixels of the location are then compared to the reference pixel,

is a coordinate (i，j) The pixels before enhancement of the position, m and n are the length and width of the window W, respectively.

9. A vehicle chassis foreign matter detection apparatus, comprising: the image acquisition module is used for acquiring an image of the bottom of the vehicle;

the foreign matter detection module adopts a detection model trained by the detection method according to any one of claims 1 to 8 to detect foreign matters in the vehicle bottom image acquired by the image acquisition module.

10. A computer-readable storage medium storing thereon a program or instructions which, when executed by a processor, implement the steps of the underbody foreign matter detection method of any one of claims 1 to 8.