CN113421230A - Vehicle-mounted liquid crystal display light guide plate defect visual detection method based on target detection network - Google Patents

Abstract

The invention relates to the technical field of image recognition, and particularly discloses a vehicle-mounted liquid crystal display light guide plate defect visual detection method based on a target detection network, which comprises the following steps: collecting an image of the light guide plate; preprocessing an image; establishing and training a first-stage target detection network, wherein the first-stage target detection network comprises a main feature extraction sub-network, a feature enhancement sub-network and a classification and regression sub-network; and (3) defect detection: and obtaining four effective characteristic layers by using a characteristic fusion mode through an image pyramid of a characteristic fusion sub-network, transmitting the four effective characteristic layers through a classification and regression sub-network to obtain a prediction result of the defects of the light guide plate, and displaying the prediction result in an upper computer. The invention solves the problems of unbalanced positive and negative samples, micro defect detection rate, detection efficiency and the like, simultaneously completes the task of positioning the defects and classification of the defects, simultaneously provides visual results, and achieves and realizes industrial application.

Description

Vehicle-mounted liquid crystal display light guide plate defect visual detection method based on target detection network

Technical Field

The invention relates to the technical field of image recognition, in particular to a defect visual detection method for a vehicle-mounted liquid crystal display light guide plate based on a target detection network.

Background

The light guide plate (light guide plate) is made of optical acrylic/PC plate, and then high-tech material with high reflectivity and no light absorption is used to print light guide points on the bottom surface of the optical acrylic plate by laser engraving, V-shaped cross grid engraving and UV screen printing technology. The light guide plate is also an important component of the liquid crystal display backlight module, and can inevitably generate defects such as bright spots, scratches, pressure damages and the like in the production process of the light guide plate, so that the display effect is directly influenced. Defects are classified into two main categories according to their shapes: point defects and line defects. The point defect mainly refers to a point defect formed inside the light guide plate, and mainly comprises a bright point and a pressure injury. In the plasticizing process, the plastic raw material cannot be completely melted due to too low temperature, the dust around the molding machine is heavy, or the plastic raw material is not clean, and white impurities are doped, so that bright spot defects can be presented. The line defect refers to a linear defect formed on the surface of the light guide plate, and mainly refers to a scratch mark on the surface of the light guide plate. The formation reason is mainly scratch on the surface of the mold core, or the contact surface of the light guide plate is not clean in the production process of the light guide plate, such as a polishing machine and a roller, so that the contact surface and the light guide plate generate large friction in the movement process, and strip-shaped scratches are formed on the surface of the light guide plate.

At present, domestic light guide plate defect detection mainly relies on manual operation to remove to accomplish, and under the condition of polishing of inspection tool, light guide plate is lighted, and whether defect such as bright spot, fish tail appear in some or many places of light guide plate of measurement personnel's eye-measurement to judge whether there is the defect in the light guide plate, the limitation that manual detection defect exists is very obvious, mainly lies in: (1) the manual detection environment is not good, and workers face the light guide plate for a long time, so that the eyesight of the workers can be seriously damaged; (2) the defect detection of the light guide plate mainly depends on the judgment and identification of human eyes, and human subjective factors exist, so that a quantifiable quality standard is difficult to form; (3) the manual operation is easily interfered by various factors, such as external environment, eye fatigue and the like, so that the actual detection efficiency and precision can be influenced to a certain extent; (4) the light guide plate has high detection complexity, high difficulty and various defects, and staff can hardly master related detection technologies. Due to various limitations of manual detection of defects, the precision, efficiency, stability and the like of manual bright point detection are difficult to adapt to the requirements of enterprises. The quality detection precision requirement of the vehicle navigation light guide plate is high, the defect of more than 10um needs to be detected, and the industrial area-array camera is difficult to meet the requirement. The industrial area-array camera adopts a 16K linear array camera to present clear images of the vehicle-mounted navigation light guide plate, the size of the acquired high-resolution images is 10084 multiplied by 14500, and enterprises require to complete defect detection on one light guide plate within 6 seconds in an industrial field, which also puts higher requirements on the defect detection efficiency. The existing defect detection method based on deep learning cannot effectively detect small point defects and shallow line defects in the vehicle-mounted navigation light guide plate, and has serious false detection and missing detection which are difficult to meet the precision requirement of industrial detection.

Disclosure of Invention

The technical problem to be solved by the invention is to provide a vehicle-mounted liquid crystal display light guide plate defect visual detection method based on a target detection network, which is used for automatically detecting and classifying defects of the light guide plate, solves the problems of unbalance of positive and negative samples, micro defect detection rate, detection efficiency and the like, simultaneously completes the task of positioning the defects and classification of the defects, simultaneously provides a visual result, and achieves and realizes industrial application.

In order to solve the technical problem, the invention provides a defect visual detection method of a vehicle-mounted liquid crystal display light guide plate based on a target detection network, which comprises the following steps:

s01, collecting the light guide plate image: collecting the light guide plate image by using a 16K linear array camera and transmitting the light guide plate image to an upper computer for processing;

s02, image preprocessing: obtaining a region image of the light guide plate ROI by using a threshold segmentation technology, and then cutting the region image of the light guide plate ROI into a group of small images with the size of H multiplied by W multiplied by 1, wherein adjacent small images are overlapped left and right by 1/10 image width;

s03, establishing and training a one-stage target detection network, wherein the one-stage target detection network comprises a main feature extraction sub-network, a feature enhancement sub-network and a classification and regression sub-network;

s04, defect detection:

inputting the H multiplied by W multiplied by 1 small image preprocessed by the S02 image into a one-stage target detection network trained by the S03, extracting four multi-scale feature maps from the H multiplied by W multiplied by 1 small image by using a feature extraction sub-network, then obtaining four effective feature layers by using a feature fusion mode through an image pyramid of a feature fusion sub-network, transmitting the four effective feature layers through a classification and regression sub-network to obtain a prediction result of the defect of the light guide plate, and displaying the prediction result in an upper computer.

The invention relates to an improvement of a defect visual detection method of a vehicle-mounted liquid crystal display light guide plate based on a target detection network, which comprises the following steps:

the trunk feature extraction sub-network comprises a batch residual error network ResNeXt50 network with 5 convolutional layers as a base line network, and the convolution of the lower half part 1X1 in each ResNeXt _ block of convolutional layers Conv2, Conv3, Conv4 and Conv5 in the base line ResNeXt50 network is replaced by Ghost _ Module; the input of the trunk feature extraction sub-network is H multiplied by W multiplied by 1 small images obtained by S02, and the output is multi-scale feature maps p1, p2, p3 and p4 output by convolutional layers Conv2, Conv3, Conv4 and Conv5 respectively;

the input of the feature fusion sub-network is a multi-scale feature map P1, P2, P3 and P4, channel adjustment is carried out through a 1x1 convolution module to be changed into P1_ in, P2_ in, P3_ in and P4_ in, and then four effective feature layers P1_ out, P2_ out, P3_ out and P4_ out are obtained through a feature pyramid in a feature fusion mode;

the classification and regression subnetwork comprises four class + box subnet structures, each class + box subnet structure comprises a class subnet and a box subnet, the class subnet comprises convolution of 4 times 256 channels and 1 time of num _ priors xnum _ classes, the box subnet comprises convolution of 4 times 256 channels and 1 time of num _ priors x 4, four effective feature layers P1_ out, P2_ out, P3_ out and P4_ out are respectively transmitted through one class + box subnet structure, and finally, each class + box subnet structure outputs a prediction result: target position information and a category corresponding to each prediction box at each grid point on the effective feature layer.

The defect visual detection method of the vehicle-mounted liquid crystal display light guide plate based on the target detection network is further improved as follows:

the training stage target detection network comprises:

s0301, establishing a training data set: collecting light guide plate images to establish a data set, manually marking the images in the data set, selecting a defect frame by using a rectangular frame during marking, and inputting a corresponding defect name; then, after data enhancement is carried out on the data set image by using a background replacement and brightness conversion method, the data set image is processed according to the following steps of 7: 2: 1, dividing the ratio into a training set, a verification set and a test set;

s0302, establishing a loss function, Focal loss:

Fl(p_t)＝-α_t(1-p_t)^γlog(p_t)

wherein p is_tRepresenting the probability of prediction being correct, alpha_tIs a weight, α_tTaking 0.25, wherein gamma represents an adjustable focusing parameter and 2;

s0303, training

The optimizer adopts an Adam optimizer, and the learning rate is 1x10^-5(ii) a Inputting the training set image established by S0301 into the one-stage target detection network for training, calculating the loss of the current one-stage target detection network through the loss function Focal loss of the output prediction result and manual annotation of the one-stage target detection network, and adjusting the network parameters through a back propagation algorithm and a gradient descent algorithm to enable the training to be carried outThe training set loss is continuously reduced, if the verification set loss in the current training round is lower than that in the previous round, the model of the current round is saved, and a stage of target detection network trains for 200 rounds in total; and then testing and obtaining the trained one-stage target detection network through the test set.

The defect visual detection method of the vehicle-mounted liquid crystal display light guide plate based on the target detection network is further improved as follows:

after each convolution operation of the trunk feature extraction sub-network, a normalization BN and an activation function ReLU operation are carried out, wherein the normalization BN is defined as: will input pixel point x_iSubtract mean μ and divide by mean variance

To obtain a normalized value x_iThen carrying out scale conversion and offset to obtain a value y after batch normalization processing_iWherein:

n is the batch size; epsilon is a fixed value; γ and β are network learned parameters;

the activation function ReLU is defined as:

the invention has the following beneficial effects:

1. the main feature extraction sub-network is improved and optimized based on a ResNeXt50 network, and the Ghost _ Module is used for replacing the convolution of the lower half part 1X1 in the ResNeXt _ block, so that resource parameters and resource consumption are reduced, and the training and reasoning speed is increased;

2. compared with the traditional detection algorithm, the traditional detection algorithm usually needs to perform partition processing aiming at the characteristics of different sparse and dense areas of the light guide plate image, different processing strategies are adopted aiming at different partitions, the problem of sparse and dense light guide points is not needed to be distinguished, and the problem can be solved by a one-stage target detection network in a learning mode;

3. the invention improves the feature fusion network in RetinaNet by using the proposed feature fusion sub-network, more effectively fuses the main features to extract the shallow semantic information and the high semantic information in the sub-network, and further improves the detection capability of small target defects.

4. The detection algorithm of the invention has strong universality and strong stability, reduces erroneous judgment and missing detection and improves the detection precision.

Drawings

The following describes embodiments of the present invention in further detail with reference to the accompanying drawings.

FIG. 1 is a schematic flow chart of a vehicle-mounted LCD screen light guide plate defect visual detection method based on a target detection network according to the present invention;

FIG. 2 is a schematic diagram of a stage target detection network according to the present invention;

FIG. 3 is a schematic structural diagram of a backbone feature extraction sub-network of the stage target detection network of FIG. 2;

FIG. 4 is a diagram of an exemplary configuration of a feature fusion subnetwork of the stage one target detection network of FIG. 2;

FIG. 5 is a schematic diagram of the classification and regression sub-networks of the stage one target detection network of FIG. 2;

FIG. 6 is a graph showing the results of comparative experiment 1.

Detailed Description

The invention will be further described with reference to specific examples, but the scope of the invention is not limited thereto:

embodiment 1, a method for visually inspecting defects of a light guide plate of a vehicle-mounted liquid crystal display screen based on a target detection network, as shown in fig. 1 to 5, includes the following steps:

step 1, collecting light guide plate images

Arranging a light guide plate image acquisition device at the tail end of a vehicle-mounted navigation light guide plate production line, wherein the light guide plate image acquisition device adopts a 16K linear array camera to acquire images and then transmits the acquired light guide plate images to an upper computer for processing;

step 2, image preprocessing

The collected light guide plate image contains a background area, the background contained in the collected light guide plate image is removed by utilizing a threshold segmentation technology, an area image of a light guide plate ROI is obtained, and the detection efficiency is further improved;

then, cutting the extracted regional image of the light guide plate ROI into a group of small images with the size of H multiplied by W multiplied by 1, wherein the adjacent small images are overlapped left and right by 1/10 image width, so that the integrity of the edge of the defect image can be ensured, and the complete defect can not be segmented due to image segmentation; this group of cropped out hxwx 1 small images, where H is 600, W is 600, and H, W appearing below is the same size as it is input to the subsequent steps;

step 3, establishing a stage target detection network

The first-stage target detection network comprises a main feature extraction sub-network, a feature enhancement sub-network and a classification and regression sub-network; step 3.1, establishing a backbone feature extraction sub-network

The trunk feature extraction sub-network takes a bulk residual error network ResNeXt50 network as a baseline network and comprises 5 convolutional layers Conv1, Conv2, Conv3, Conv4 and Conv5 which are connected in sequence, wherein the input of the trunk feature extraction sub-network, namely the input of a convolutional layer Conv1, is a small image of H multiplied by W multiplied by 1 obtained in step 2, the inputs of the convolutional layers Conv2, Conv3, Conv4 and Conv5 are the outputs of the previous convolutional layers respectively, the outputs of the trunk feature extraction sub-network are the outputs of the convolutional layers Conv2, Conv3, Conv4 and Conv5, and are respectively a multi-scale feature map P1, a multi-scale feature map P2, a multi-scale feature map P3 and a multi-scale feature map P4, namely the multi-scale feature map P1 is the input of the first layer of the convolutional layer Conv2 and a feature fusion sub-network (IFPN),

replacing the lower half of the 1X1 convolution in each ResNeXt _ block of the convolutional layers Conv2, Conv3, Conv4 and Conv5 in the baseline ResNeXt50 network with Ghost _ Module, so as to establish a backbone feature extraction sub-network; the improved ResNeXt50 network, namely the backbone feature extraction sub-network, reduces resource parameters and resource consumption, accelerates training and reasoning speed, and the specific network hierarchical structure is shown in Table 1;

table 1: backbone feature extraction subnetwork hierarchy table

Wherein the convolutional layer Conv1 is a 7 × 7 convolutional kernel with 128 channels, and connects a largest pooling layer with 3 pooling size of 3 × 3 in step size, and each of the convolutional layers Conv2, Conv3, Conv4 and Conv5 has a similar structure, taking convolutional layer Conv2 as an example, as shown in fig. 3, the input is a multi-scale feature map P1 of the output of the convolutional layer Conv1, and then sequentially a 1 × 1 convolutional kernel with 128 channels, and connects 32 groups of parallel stacked convolutional layers of 3 × 3 convolutional kernels with 128 channels, then a Ghost _ module of the 1 × 1 convolutional kernel with 256 channels, and the output is a multi-scale feature map P2;

the convolutional layers Conv2, Conv3, Conv4 and Conv5 in table 1 contain a convolution of 1X1, and the formula of the amount of calculation of the convolution is as follows:

computation＝n×h×w×c×k×k

wherein: n is the number of channels of the output characteristic diagram, h and w are the height and width of the characteristic diagram respectively, c is the number of input channels, and k is the size of a convolution kernel;

and Ghost _ Module, defining init _ channels as the number of output channels, output _ channels as the number of output1 channels, new _ channels as the number of output (2) channels, and ratio as the ratio of the number of output (1) and output (2) channels to the number of output (1) channels. The implementation of the Ghost _ Module is carried out in two steps, and the specific operation is as follows:

(1) performing convolution in advance, wherein the obtained output channel number is output _ channel/ratio, and the ratio is 2;

(2) performing simple linear transformation, namely, performing convolution once on each feature graph of the pre-convolution module, wherein the group number of the feature graphs is k, and the output channel number of the feature graphs is new _ channels; the channel number generated by adding the output (1) and the output (2) together is the final channel number init _ channels of the Ghost _ Module;

the backbone feature extraction sub-network uses batch normalization BN (batch normalization) operation to outputPixel point x_iSubtract mean μ and divide by mean variance

To obtain a normalized value x_iThen carrying out scale conversion and offset to obtain a value y after batch normalization processing_iWherein:

n is the batch size; ε is a fixed value to prevent divide by 0 errors; gamma and beta are parameters learned by the network, normalization and activation functions (BN + ReLU) are required to be carried out after each convolution operation in the trunk feature extraction sub-network, and one activation function is used after the BN operation by default, so that the regularization of the network is facilitated; the activation function adopts a ReLU activation function, wherein:

step 3.2 construction of feature fusion sub-networks

The specific operation of the feature fusion sub-network (IFPN) can be divided into three parts, and four effective feature layers are obtained by using a feature fusion mode through an image pyramid, as shown in fig. 4, the process is as follows:

(1) the output multi-scale feature map P1, the multi-scale feature map P2, the multi-scale feature map P3 and the multi-scale feature map P4 of the trunk feature extraction sub-network cannot be directly used as the input of a feature fusion sub-network (IFPN), and a 1x1 convolution module is respectively adopted to adjust channels to become P1_ in, P2_ in, P3_ in and P4_ in, so that the feature pyramid can be input; stacking the p4_ in after upsampling with p3_ in to obtain p3_ td, stacking the p3_ td with p2_ in after upsampling is continuously performed on the p3_ td to obtain p2_ td, stacking the p2_ td with p1_ in to obtain p1_ tg after upsampling is also performed on the p2_ td, and completing the first part of a feature fusion sub-network (IFPN);

(2) the multi-scale feature map P1, the multi-scale feature map P2, the multi-scale feature map P3 and the multi-scale feature map P4 are feature layers which are obtained from the trunk feature extraction sub-networks and rich in information resources, with the continuous convolution operation of the trunk network, the receptive fields of the feature layers are increased, but the fine granularity in the pictures is in a descending trend; compared with the characteristic layers of the P multi-scale characteristic diagram P3 and the multi-scale characteristic diagram P4, the multi-scale characteristic diagram P1 and the multi-scale characteristic diagram P2 are less in convolution, richer in fine granularity, more sensitive to small defects such as light guide plate points and lines and the like, beneficial to quickly completing positioning and helping to reduce missed detection; in order to more fully utilize the shallow semantic information, p1_ tg is downsampled and stacked with p2_ td and p2_ in to obtain p2_ tg, and an attention mechanism is introduced during stacking to judge which channel information should be paid attention to specifically; the p2_ tg is subjected to down sampling once and then stacked with the p3_ td to obtain p3_ tg, and the p3_ tg is subjected to down sampling once and then stacked with the p4_ in once to obtain p4_ tg;

(3) for point defects and partially shallow line defects, more feature extraction is obtained from p1_ tg and p2_ tg, and the difference between p1_ tg and p1_ in is small, so that more sufficient feature information is difficult to obtain, and the method is not helpful for improving the detection accuracy of the defects; for this purpose, a third feature fusion is added, p4_ out is upsampled and then stacked with p3_ tg to obtain p3_ out, p3_ out is upsampled once and then stacked with p2_ tg to obtain p2_ out, and p2_ out is upsampled further and then stacked with p1_ tg and p1_ in with attention mechanism to obtain p1_ out; in order to eliminate the aliasing effect brought by feature fusion, feature layer addition is adopted during stacking, then 3X3 depth separable convolution is carried out, and 1X1 convolution is used for channel adjustment;

the feature layers, P1_ out, P2_ out, P3_ out and P4_ out, are obtained by the feature pyramid of the feature fusion sub-network (IFPN), and are called effective feature layers for distinguishing from common feature layers.

Step 3.3, construction of Classification and regression sub-networks

The classification and regression subnetwork comprises four class + box sub-network structures, four effective feature layers P1_ out, P2_ out, P3_ out and P4_ out are respectively transmitted through one class + box sub-network structure, each class + box sub-network structure comprises a class sub-network and a box sub-network, the class sub-network adopts 4 convolutions of 256 channels and 1 convolution of num _ priors x num _ classes, the convolution of num _ priors x num _ classes is used for predicting the type corresponding to each prediction box on each grid point on the effective feature layer, num _ priors refers to the number of prior boxes owned by the effective feature layer, and num _ classes refers to the number of objects detected by the network; the box subnet adopts 4 convolutions of 256 channels and 1 convolution of num _ priors x 4, num _ priors refers to the number of prior frames owned by the effective feature layer, 4 refers to the adjustment condition of the prior frames, and the convolution of num _ priors x 4 is used for predicting the change condition of each prior frame on each grid point on the effective feature layer; finally, each class + box subnet structure outputs a prediction result: the change condition of each prior frame on each grid point on each effective characteristic layer is the target position information and the corresponding category of each prediction frame on each grid point on the effective characteristic layer.

Step 4, training and testing a stage of target detection network

Step 4.1, establishing a training data set

At the tail end of a vehicle-mounted navigation light guide plate production line, acquiring an image of a light guide plate on an industrial site through a light guide plate image acquisition device comprising a 16K linear array camera, establishing a data set, manually marking the image in the data set, selecting a defect frame by using a rectangular frame during marking, and inputting a corresponding defect name; then, using a background replacement method and a brightness conversion method to carry out data enhancement to obtain 2897 point defects, 2936 line defects and 2856 surface defects, wherein the method comprises the following steps of: 2: 1, dividing a data set according to a proportion, wherein 6082 training sets, 1737 verification sets and 286 testing sets are used, and the training sets, the verification sets and the testing sets are as uniform as possible as light guide plate images containing point defects, line defects and surface defects;

step 4.2, establishing a loss function

The defects of points and lines in the image of the vehicle-mounted LCD light guide plate are small, the occupied image position is small, the image position occupied by the background is large, and the negative sample is far more than the positive sample, the method adopts a Focal loss function for balancing the positive and negative samples to solve the problem that the positive and negative samples are extremely unbalanced in the defect detection of the light guide plate, and the Focal loss is as follows:

Fl(p_t)＝-α_t(1-p_t)^γlog(p_t)

wherein p is_tRepresenting the probability of prediction being correct, alpha_tIs a weight, α_tTaking 0.25, wherein gamma represents an adjustable focusing parameter and 2;

step 4.3, training a stage of target detection network

The method is characterized in that a pytorech 1.2 building network is used for training and testing, an Adam optimizer is adopted as an optimizer, and the learning rate is 1x10^-5(ii) a Inputting the established training set image into the one-stage target detection network established in the step 3 for training, calculating the loss of the one-stage target detection network in the current round through a loss function Focal loss by the output prediction result of the one-stage target detection network and manual labeling, adjusting network parameters through a back propagation algorithm and a gradient descent algorithm to enable the loss of the training set to be continuously descended, if the loss of the verification set in the current training round is lower than that of the verification set in the previous round, saving the model of the current round, and training the one-stage target detection network for 200 rounds;

after training is finished, the parameters of the one-stage target detection network are stored as configuration files to be output, and therefore the trained one-stage target detection network is obtained.

Step 4.4, off-line testing

Inputting the test set into the one-stage target detection network trained in the step 4.3 for defect detection, extracting input H multiplied by W multiplied by 1 images into 4 multi-scale feature maps by using a feature extraction sub-network, then obtaining 4 effective feature layers by using a feature fusion sub-network, respectively transmitting the 4 effective feature layers into a classification and regression sub-network, and respectively obtaining 4 prediction results: outputting and storing the prediction result on an upper computer of a light guide plate production line according to the change condition of each prior frame on each grid point on each effective characteristic layer, namely target position information and the type corresponding to each prediction frame on each grid point on the effective characteristic layer;

the Average Accuracy AP, mAP (mean Average Precision) and Accuracy are adopted to measure the Accuracy of the prediction result:

1) the AP can reflect the average precision of various defects, and a definite integral is made between precision and recall, wherein the specific expression is as follows:

wherein x represents a certain defect,

precision _ rate refers to the accuracy: precision _ rate is TP/(TP + FP),

the call _ rate refers to the recall rate: the Recall _ rate is TP/(TP + FN),

wherein the content of the first and second substances,

the number of correct defect detection results in the class of TP (true Positive) refers to positive samples predicted to be positive,

fp (false positive) false detection, i.e. indicating that the negative samples predicted to be positive refer to the number of defect detection errors of this type,

FN (false negative) missing detection, which refers to positive samples that are predicted to be negative, i.e., the number of no detected results,

TN (true negative) is predicted as a negative sample, and has no practical meaning in the detection task;

2) the mAP is an average value of all types of APs, and is used as a comprehensive performance evaluation index capable of reflecting the model, and the average condition of detection precision of each type in the model is reflected:

wherein X refers to the set of all classes of defects;

3) and the accuracy is as follows:

Accuracy＝(TP+TN)/TP+TN+FP+FN。

the results of the tests are shown in table 2,

TABLE 2 test results

From the experiment, the network structure provided by the invention has higher accuracy, the accuracy reaches 98.6%, and the mAP reaches 96.7%, so that the network structure is proved to have higher accuracy based on one-stage target detection and can obtain excellent effect on the task of detecting the defects of the light guide plate; the training set, the verification set and the test set are all light guide plate images actually obtained from online production, and meanwhile, the method is also suitable for detection of online production.

Step 5, using the trained one-stage target detection network to detect the defects and output the result

Extracting four multi-scale feature maps, namely P1, P2, P3 and P4, from the H multiplied by W multiplied by 1 small images subjected to the image preprocessing in the step 2 by using a feature extraction sub-network; and then, four effective characteristic layers, namely P1_ out, P2_ out, P3_ out and P4_ out, are obtained by using a characteristic fusion mode through an image pyramid of a characteristic fusion sub-network, and the four effective characteristic layers are classified and subjected to a class + box subnet structure of a regression sub-network to obtain a prediction result of the defect of the light guide plate, and the prediction result is displayed in an upper computer.

Comparative experiment 1:

adopting one light guide plate image with line defects, point defects and surface defects, wherein the light guide plate image with the line defects contains one deep line defect and one shallow line defect respectively, as shown in fig. 6, firstly, preprocessing the light guide plate image in the upper computer by the step 2 image described in the embodiment 1 to obtain H multiplied by W multiplied by 1 small images, then respectively inputting the H multiplied by W multiplied by 1 small images into a RetinaNet network and a one-stage target detection network of the invention to carry out a defect detection comparison experiment, and outputting a result to show that the one-stage target detection network of the invention identifies all defects, and the RetinaNet network cannot identify the shallow line defects and the point defects, which shows that the invention can improve the detection capability and the detection precision of the small target defects.

Finally, it is also noted that the above-mentioned lists merely illustrate a few specific embodiments of the invention. It is obvious that the invention is not limited to the above embodiments, but that many variations are possible. All modifications which can be derived or suggested by a person skilled in the art from the disclosure of the present invention are to be considered within the scope of the invention.

Claims (4)

1. The vehicle-mounted liquid crystal display light guide plate defect visual detection method based on the target detection network is characterized by comprising the following steps of:

s01, collecting the light guide plate image: collecting the light guide plate image by using a 16K linear array camera and transmitting the light guide plate image to an upper computer for processing;

s02, image preprocessing: obtaining a region image of the light guide plate ROI by using a threshold segmentation technology, and then cutting the region image of the light guide plate ROI into a group of small images with the size of H multiplied by W multiplied by 1, wherein adjacent small images are overlapped left and right by 1/10 image width;

s03, establishing and training a one-stage target detection network, wherein the one-stage target detection network comprises a main feature extraction sub-network, a feature enhancement sub-network and a classification and regression sub-network;

s04, defect detection:

inputting the H multiplied by W multiplied by 1 small image preprocessed by the S02 image into a one-stage target detection network trained by the S03, extracting four multi-scale feature maps from the H multiplied by W multiplied by 1 small image by using a feature extraction sub-network, then obtaining four effective feature layers by using a feature fusion mode through an image pyramid of a feature fusion sub-network, transmitting the four effective feature layers through a classification and regression sub-network to obtain a prediction result of the defect of the light guide plate, and displaying the prediction result in an upper computer.

2. The vehicle-mounted liquid crystal display light guide plate defect visual detection method based on the target detection network as claimed in claim 1, wherein:

the trunk feature extraction sub-network comprises a batch residual error network ResNeXt50 network with 5 convolutional layers as a base line network, and the convolution of the lower half part 1X1 in each ResNeXt _ block of convolutional layers Conv2, Conv3, Conv4 and Conv5 in the base line ResNeXt50 network is replaced by Ghost _ Module; the input of the trunk feature extraction sub-network is H multiplied by W multiplied by 1 small images obtained by S02, and the output is multi-scale feature maps p1, p2, p3 and p4 output by convolutional layers Conv2, Conv3, Conv4 and Conv5 respectively;

the input of the feature fusion sub-network is a multi-scale feature map P1, P2, P3 and P4, channel adjustment is carried out through a 1x1 convolution module to be changed into P1_ in, P2_ in, P3_ in and P4_ in, and then four effective feature layers P1_ out, P2_ out, P3_ out and P4_ out are obtained through a feature pyramid in a feature fusion mode;

the classification and regression subnetwork comprises four class + box subnet structures, each class + box subnet structure comprises a class subnet and a box subnet, the class subnet comprises convolution of 4 times 256 channels and 1 time of num _ priors x number _ classes, the box subnet comprises convolution of 4 times 256 channels and 1 time of num _ priors x 4, four effective feature layers P1_ out, P2_ out, P3_ out and P4_ out are respectively transmitted through one class + box subnet structure, and finally, each class + box subnet structure outputs a prediction result: target position information and a category corresponding to each prediction box at each grid point on the effective feature layer.

3. The vehicle-mounted liquid crystal display light guide plate defect visual detection method based on the target detection network as claimed in claim 2, wherein:

the training stage target detection network comprises:

s0301, establishing a training data set: collecting light guide plate images to establish a data set, manually marking the images in the data set, selecting a defect frame by using a rectangular frame during marking, and inputting a corresponding defect name; then, after data enhancement is carried out on the data set image by using a background replacement and brightness conversion method, the data set image is processed according to the following steps of 7: 2: 1, dividing the ratio into a training set, a verification set and a test set;

s0302, establishing a loss function, Focal loss:

Fl(p_t)＝-α_t(1-p_t)^γlog(p_t)

wherein p is_tRepresenting the probability of prediction being correct, alpha_tIs a weight, α_tTaking 0.25, wherein gamma represents an adjustable focusing parameter and 2;

s0303, training

The optimizer adopts an Adam optimizer, and the learning rate is 1x10^-5(ii) a Inputting the training set image established in S0301 into the one-stage target detection network for training, calculating the loss of the one-stage target detection network in the current round through a loss function Focal loss according to the output prediction result of the one-stage target detection network and manual labeling, adjusting network parameters through a back propagation algorithm and a gradient descent algorithm to enable the loss of the training set to be continuously descended, if the loss of the verification set in the current round is lower than the loss of the verification set in the previous round, saving the model of the current round, and training the one-stage target detection network for 200 rounds; and then testing and obtaining the trained one-stage target detection network through the test set.

4. The vehicle-mounted liquid crystal display light guide plate defect visual detection method based on the target detection network as claimed in claim 3, wherein:

after each convolution operation of the trunk feature extraction sub-network, a normalization BN and an activation function ReLU operation are carried out, wherein the normalization BN is defined as: will input pixel point x_iSubtract mean μ and divide by mean variance

To obtain a normalized value x_iThen carrying out scale conversion and offset to obtain a value y after batch normalization processing_iWherein:

n is the batch size; epsilon is a fixed value; γ and β are network learned parameters;

the activation function ReLU is defined as:

Images