Multi-scale IOF random cutting data enhancement method for target detection
Technical Field
The invention relates to the technical field of intelligent manufacturing and artificial intelligence, in particular to a data enhancement method for multi-scale IOF random cutting for target detection.
Background
With the popularization of social informatization, display terminals as media for information display and propagation have developed into an indispensable important role. Because of its advantages of low power consumption, small size, high definition, low distortion factor, etc., liquid crystal display screen is becoming most display terminal equipment such as smart phone, personal computer, smart watch, digital camera, large screen television, etc.
In recent years, due to the rapid development of the field of machine vision, the adoption of an Automatic Optical Inspection (AOI) technology to assist the traditional manual quality inspection mode to detect and classify panel defects is an essential link in factory production; in the defect detection link, the classification accuracy of the detail map is low due to the problems of small picture and fine defect of the detail map; meanwhile, in the process of manual quality inspection, the detection result is unstable and the judgment standard is difficult to unify and quantify due to human eye structure difference, emotional state, fatigue and the like; the problems of low accuracy and low efficiency exist in the detection of the detail map.
In the panel manufacturing process, a large number of defects with large scale span can be generated; at present, data enhancement modes (such as common cutting, random cutting and random IOU cutting) in a common deep learning target detection framework are difficult to adapt to the defect of huge scale span, and the accuracy rate of model positioning and classification is reduced.
Disclosure of Invention
The technical problem to be solved by the invention is as follows: in the panel manufacturing process, a large number of defects with large scale span can be generated; at present, data enhancement modes (such as common cutting, random cutting and random IOU cutting) in a common deep learning target detection framework are difficult to adapt to the defect of huge scale span, and the accuracy rate of model positioning and classification is reduced.
To solve the above technical problems.
The invention is realized by the following technical scheme:
the invention provides a data enhancement method for multi-scale IOF random clipping used for target detection, which comprises the following steps:
s1, setting a probability P1, and directly outputting an original image without cutting;
s2, setting an area range of a cutting frame according to the target original image to be cut;
s3, randomly generating a cutting proportion in the area range of the cutting frame, and determining the scale of the candidate cutting frame based on the cutting proportion;
s4, randomly generating a candidate cutting frame on the target original image based on the scale of the candidate cutting frame;
s5, calculating the IOF value according to the candidate cutting frame and the scale of the target original graph
S6, comparing the calculated IOF value with a preset IOF threshold value, returning to S4 for circulation when the IOF value is lower than the preset IOF threshold value, and reserving the candidate cutting frame and jumping out of the circulation when the IOF value is higher than the preset IOF threshold value;
s7, cutting the target original image according to the reserved candidate cutting frame, and correcting the cut image data;
and S8, outputting the cut picture and the corrected data.
The working principle of the scheme is as follows: in the panel manufacturing process, a large number of defects with huge scale span can be generated; after a camera shoots a picture on a production line, the defect position is determined through AOI detection, and a defect picture at the corresponding position needs to be amplified and cut for manual identification and input of a panel defect detection model. The data enhancement mode (such as common cutting, random cutting and random IOU cutting) in the current common deep learning target detection framework is difficult to adapt to the defect of huge scale span, and the accuracy rate of model positioning and classification is reduced.
Firstly, directly outputting an original image in a probabilistic manner, reserving required image defect backboard information, simultaneously cutting a target image by using an IOF random cutting method, setting a cutting range according to the area of a target defect in the target original image, firstly setting the area range of a cutting frame in the target original image according to the position of the target defect, and randomly generating a cutting proportion; and then generating a candidate cutting frame based on the cutting proportion in the range of the cutting frame, calculating an IOF value based on the scales of the candidate cutting frame and the target original image, comparing the IOF value with a preset IOF threshold value to judge whether the candidate cutting frame is reserved, if not, generating another candidate cutting frame corresponding to the cutting proportion in the range of the cutting frame again, repeating the judgment for at least 50 times until the generated candidate cutting frame meets the requirement, and jumping out to circularly reserve the candidate cutting frame to cut the reserved candidate cutting frame.
Setting the area range of the cutting frame to increase the selection of the cutting mode, and if the lower limit of the area range of the cutting frame is too small, the cutting needs to be carried out by using the method for multiple times, the calculated amount is increased, and the model training time is increased; if the lower limit of the area range of the cutting frame is set to be too large, the identification of the ultra-small scale defects can be influenced; if the area range of the cutting frame is set to be too small, the defect that the identification scale of the detection model spans a lot is influenced; therefore, the area range of the optimal crop box needs to be set according to the actual situation test adjustment of the target original drawing.
The multi-scale IOF random cutting method obtains a plurality of pictures cut in different modes through multiple cycles, can generate the cut pictures with different scales of a plurality of IOF values, and ensures the cutting quality. The traditional mode is that a fixed cutting scale is set for cutting, and after the defect cutting, the defect information of a part of scale spanning huge defects or ultra-small scale defects is lost or a defect detection model cannot accurately position and identify the defects; the cutting method provided by the scheme randomly cuts according to a certain interval of the set cutting frame, obtains pictures in the preset interval of the cutting frame after cutting for multiple times, and can accommodate pictures with more scales; by the data enhancement method of multi-scale IOF random cutting, the recognition and classification capabilities of ultra-small scale, huge spanning target and background information in data can be effectively improved.
Further preferably, the default value of the probability P1 is 1/5.
The input defect original image contains respective backboard information, after the defect in the input original image is cut in detail, part of backboard information can be cut off, the original image is output with the probability of 1/5, the part of pictures is not cut, the pictures can completely enter a deep learning target detection model for detection, and the accuracy of the defect detection system is improved.
Further preferably, the method for setting the area range of the crop box comprises the following steps:
when the area of the target defect in the original image is large, setting a large lower boundary of a cutting frame area;
when the area of the target defect in the original image is small, setting a small lower boundary of a cutting frame area;
when the original image has a plurality of target defects and the area of the target defects is small, the area range of the large cutting frame is set, namely the lower limit of the area range of the cutting frame is small and the upper limit of the area range of the cutting frame is large.
When there are a plurality of target defects in the original image and the area distribution of the target defects is large, the area range of the crop box is set to be large.
Further preferably, the S6 cycle is at least 50 times.
Further preferably, the IOF threshold setting mode is: when the area of the defect in the target original image is small, a high IOF threshold value is set, and when the area of the defect in the target original image is large, a low IOF threshold value is set.
If the defect in the target original image is large, setting the IOF threshold value to be high may increase the cropping ratio, which is equivalent to directly outputting the original image.
Further preferably, the data correction of the clipped picture includes: and correcting the numerical value of the target defect bundling box in the cut picture.
The target defect bundling box value is a positioning mark of the target defect in the picture, and after the original picture is cut, the position of the defect cannot be correctly corresponding to the target defect bundling box value before the target defect bundling box value is used, so that the target defect bundling box value of the cut picture needs to be reset.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the invention provides a data enhancement method for multi-scale IOF random cutting for target detection, which adopts a multi-scale random IOF cutting mode to carry out data enhancement processing and improves the recognition capability of a model on ultra-small scale defects, defects with huge scale span and background information.
2. The invention provides a data enhancement method for multi-scale IOF random cutting for target detection, which is used for probabilistically outputting original pictures, not cutting partial original pictures, cutting the target original pictures, ensuring that partial backboard information can enter a defect detection model for detection and improving the accuracy of a defect detection system.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention.
In the drawings:
FIG. 1 is a flow chart of a data enhancement method for multi-scale IOF random clipping for target detection according to the present invention.
FIG. 2 is a schematic diagram of the IOF value calculation process.
FIG. 3 is a diagram illustrating a result of random clipping according to an embodiment.
FIG. 4 is a diagram illustrating a result of random clipping according to an embodiment.
1-candidate crop box, 2-target artwork, 3-intersection of candidate crop box and target artwork, 4-crop box a, 5-crop box B, 6-target defect a, 7-crop box C, 8-crop box D, 9-target defect B.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to examples and accompanying drawings, and the exemplary embodiments and descriptions thereof are only used for explaining the present invention and are not meant to limit the present invention.
Example 1
As shown in fig. 1, the data enhancement method for multi-scale IOF random cropping for target detection provided by the present invention specifically includes the following steps:
s1, setting a probability P1, and directly outputting an original image without cutting;
s2, setting an area range of a cutting frame according to the target original image to be cut;
s3, randomly generating a cutting proportion in the area range of the cutting frame, and determining the scale of the candidate cutting frame based on the cutting proportion;
s4, randomly generating a candidate cutting frame on the target original image based on the scale of the candidate cutting frame;
s5, calculating the IOF value according to the candidate cutting frame and the scale of the target original graph
S6, comparing the calculated IOF value with a preset IOF threshold value, returning to S4 for circulation when the IOF value is lower than the preset IOF threshold value, and reserving the candidate cutting frame and jumping out of the circulation when the IOF value is higher than the preset IOF threshold value;
s7, cutting the target original image according to the reserved candidate cutting frame, and correcting the cut image data;
and S8, outputting the cut picture and the corrected data.
The default value of the probability P1 is 1/5.
The method for setting the area range of the cutting frame comprises the following steps:
when the area of the target defect in the original image is large, setting a large lower boundary of a cutting frame area;
when the area of the target defect in the original image is smaller, setting a small lower boundary of the cutting frame area;
when the original image has a plurality of target defects and the area distribution span of the target defects is large, a large area range of the cutting frame is set, namely, the lower limit of the area range of the cutting frame is small and the upper limit of the area range of the cutting frame is large.
And circulating at least 50 times in the S6.
The IOF threshold is an interval range, when the crossing degree of the target defect is larger, a large IOF threshold interval is set, and when the crossing degree of the target defect is smaller, a small IOF threshold interval is set.
The data correction of the cut picture comprises the following steps: and correcting the numerical value of the target defect bundling box in the cut picture.
Example 2
As shown in fig. 3, a panel factory uses the data enhancement processing method provided by the present invention to crop a picture of a defective panel in the factory, and a random crop frame A4 and a random crop frame B5 are obtained after two times of cropping.
In the process of one-time clipping, one clipping proportion randomly generated in the area range of the clipping frame is 0.8, and the scale of the corresponding candidate clipping frame is as follows: 800x800, calculating an IOF value to be 1 according to the scales of the candidate cutting frame and the target original image, setting an IOF threshold value to be 0.2 according to the characteristics of the target defect, setting the IOF value to be more than 0.2, meeting the requirement, reserving the candidate cutting frame by the system, jumping out of circulation, and then cutting the target original image according to the reserved 800x800 cutting frame; and resetting the clipped target defect bundling box value for output, wherein the random clipping frame A4 can be seen to completely contain the target defect 6 by the picture, so that a group of random clipping pictures and corresponding data are obtained by clipping through the multi-scale IOF random clipping data enhancement method for target detection provided by the invention.
In the second clipping process, after the target original image is input into the clipping system, through multiple cycles, a clipping proportion of 0.5 is randomly generated in the area range of the clipping frame, and the scale of the corresponding candidate clipping frame is as follows: 300x300, calculating an IOF value to be 0.8 according to the scales of the candidate cutting frame and the target original image, setting an IOF threshold value to be 0.2 according to the characteristics of the target defect, setting the IOF value to be more than 0.2 and meeting the requirement, reserving the candidate cutting frame by the system and jumping out of circulation, and then cutting the target original image according to the reserved 800x800 cutting frame; and resetting the clipped target defect bundling box value for output, wherein the random clipping frame B5 can be seen from the picture to completely contain the target defect 6, so that another group of random clipping pictures and corresponding data are obtained by clipping through the multi-scale IOF random clipping data enhancement method for target detection provided by the invention.
The random cutting frames generated twice meet the requirements, the target original image is cut randomly through the data enhancement method of multi-scale IOF random cutting for multiple times, finally, cutting pictures with different scales of multiple IOF values can be generated, the cutting quality is guaranteed, and the capacity of recognizing and classifying ultra-small scales and scales spanning huge targets and background information in data can be effectively improved.
The target original image B is input into the multi-scale IOF random clipping system, and two candidate clipping boxes are obtained as shown in fig. 4, where the IOF value of the clipping box C7 is: 0.5, the IOF value of the clipping frame C7 is smaller than the threshold value 0.6, a candidate clipping frame is generated on the target original image again randomly and circularly to obtain a clipping frame D8, and the IOF value is as follows: 0.8 meeting the threshold requirement ends the loop.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.