A kind of defect inspection method of high density flexible substrate
Technical field
The present invention relates to machine vision surface defects detection technical fields, and in particular to a kind of high density flexible substrate lacks
Fall into detection method.
Background technique
High density flexible substrate (Flexible Integrated Circuit Substrate, abbreviation FICS) is a kind of
It can be used as the high density flexible print wiring board of IC package substrate.In the production process of high density FICS, due to technical process
The precision problem of control inevitably generates apparent defect.By high-precision vision-based detection, to the various appearances defect of FICS
It is quickly positioned and type identification, is the key that realize quality control in high density FICS manufacturing process.
The method that flexible base board manufacturer mainly uses Manual Visual Inspection carries out the inspection of open defect to high density FICS
It surveys.The detection efficiency of Manual Visual Inspection is relatively low, causes the significant wastage of human resources, and this method false detection rate is higher, difficult
To guarantee the quality of detection.Some scholars propose the open defect detection method based on characteristics of image, but these methods detect
Speed is slower, it is difficult to meet the practical application request of FICS detection;Meanwhile these methods can only be carried out for certain special defect
Detection can not identify simultaneously a plurality of types of open defects on FICS.
Summary of the invention
In order to overcome shortcoming and deficiency of the existing technology, the present invention provides a kind of defect inspection of high density flexible substrate
Survey method.
The present invention adopts the following technical scheme:
A kind of defect inspection method of high density flexible substrate, comprising:
Deep neural network model training step, specifically:
Acquisition has the FICS image of open defect, and standard size will be unified for after image preprocessing, marks every image
The defects of position and classification, the training sample as faster R-CNN convolutional neural networks model;
Using the training sample of label number as the input of faster R-CNN convolutional neural networks model, and exports FICS and lack
Sunken position and type information obtains trained based on faster R-CNN convolutional neural networks model;
Defect detection procedure, specifically:
FICS image to be detected input is trained based on faster R-CNN convolutional neural networks model, and output is
It is no defective, if defective, export defective locations and type.
The faster R-CNN convolutional neural networks model includes
Using pretreated FICS image to be detected as input, the shared convolutional neural networks of characteristic pattern are exported
After carrying out sample point coordinate to output characteristic pattern, by bilinear interpolation, the value of sample point coordinate is calculated,
And export the parallel space converting network that characteristic pattern corresponds to target point value;
Input feature vector figure corresponds to target point value and obtains the candidate frame generation network of candidate frame;
Candidate frame is inputted, the area-of-interest pond sorter network of the location information and classification for defect is exported.
The shared convolutional neural networks are made of 5 convolutional layers and 2 pond layers.
The parallel space converting network includes
The positioning network being made of three full articulamentums, for obtaining transition matrix θ by characteristic pattern;
By transition matrix θ, position of each sampled point on input feature vector figure on output characteristic pattern is obtained, it is then right
The pixel positioner that characteristic pattern is converted;
By bilinear interpolation, value is obtained to sample point coordinate and is calculated, exports the sampling of target point value on characteristic pattern
Device.
Pixel positioner includes the mapping mode of scaling, translation and rotation.
The candidate frame generates the mechanism that network uses sliding window combination anchor, determines the target of sliding window.
Area-of-interest pond sorter network includes an area-of-interest layer and two full articulamentums.
The loss function of area-of-interest pond sorter network are as follows:
Lhard=Lclass(p, u)+δ Lregre(t, v)
Lclass(p, u) represents the log loss of classification, Lregre(t, v) represents the recurrence loss of prediction block coordinate;P is represented
The prediction block to it includes object true classification predicted value, 4 set of coordinates that t and v have respectively represented prediction block close
The vector that the vector sum actual position coordinate come combines;
4 coordinates are respectively (x, y, w, h), and x represents the opposite abscissa of candidate frame, and y represents the relatively vertical of candidate frame and sits
Mark, w represent the width of candidate frame, and h represents the height of candidate frame, and δ returns loss for balanced sort loss and coordinate;
In the training process, preceding 128 candidate frames of the largest loss are considered as difficult sample, using under small batch stochastic gradient
Drop method carries out backpropagation, updates network parameter.
Uniform sizes of the present invention are specially the standard size of 224 × 224 pixels.
Beneficial effects of the present invention:
(1) a kind of high density flexible exterior substrate defective vision detection side based on improved faster R-CNN of the present invention
Method, it can be applied to the quick detection of open defect in flexible base board manufacturing process, replace Manual Visual Inspection, avoid artificial mesh
The waste of human resources brought by examining, and greatly improve working efficiency;
(2) detection method can be quickly detected from different types of defect on high density flexible substrate automatically, and will
The position of defect and type mark come out, and the defect solved in the case of the insoluble random site of traditional images method is quick
Positioning and test problems;
(3) without carrying out image characteristic analysis, after the neural network model that picture to be detected input has been trained, Ke Yizhi
Position and the classification information for connecing output defect, to solve traditional image processing method speed based on signature analysis slowly simultaneously
And the problem of multiple types defect can not be detected simultaneously.
Detailed description of the invention
Fig. 1 is work flow diagram of the invention;
Fig. 2 is that the present invention is based on the structure charts of the deep neural network model of faster R-CNN;
Fig. 3 is the schematic diagram of the shared convolutional neural networks in Fig. 2;
Fig. 4 is the schematic diagram of parallel space converting network in Fig. 2;
Fig. 5 is the schematic diagram that candidate frame generates network in Fig. 2;
Fig. 6 is the schematic diagram of area-of-interest pond sorter network in Fig. 2;
Fig. 7 (a)-Fig. 7 (f) is the detection effect figure of FICS in the present invention, is short circuit, open circuit, scratch, pin hole, oxygen respectively
Change and holes defect.
Specific embodiment
Below with reference to examples and drawings, the present invention is described in further detail, but embodiments of the present invention are not
It is limited to this.
Embodiment
As shown in Figure 1, a kind of defect inspection method of high density flexible substrate, including deep neural network model training and
Two steps of defects detection.
The deep neural network model training step, deep neural network model uses fasterR- in the present embodiment
CNN, specific as follows:
S1: the FICS image for largely having variety classes open defect is collected;
S2: the size for the image being collected into is pre-processed, and is unified for 224 × 224 standard size;
S3: marking the defects of every image position and classification, using the image marked as next step faster R-
The training sample of CNN convolutional neural networks;
S4: using the training sample image that has marked as the input of depth convolutional neural networks model, and FICS is lacked
Sunken position and type information is exported as model, is carried out to the neural network detection model based on improved faster R-CNN
Training, obtains the deep neural network model for FICS picture appearance defect location and type identification.
Defect detection procedure, specific as follows:
All FICS images to be detected are unified for standard size 224 × 224;
FICS image is inputted into trained convolutional neural networks model in order, exports whether defective, defect
Position and type information, and by information preservation into database;
The image inputted to next detects, until detection terminates.
As shown in Fig. 2, based on the depth convolutional neural networks model of improved faster R-CNN in the present embodiment.It by
4 parts form, and first part is shared convolutional neural networks, and second part is parallel space converting network, and Part III is
Candidate frame generates network, and Part IV is area-of-interest and pond network.
As shown in figure 3, first part is shared convolutional neural networks.This subnetwork is by 5 convolutional layers and 2 ponds
Layer composition, input are the images that size is 224*224 after pre-processing, and output is characteristic pattern.Pond layer uses the side in maximum pond
Method calculates to reduce a large amount of parameter, and prevents over-fitting.
As shown in figure 4, second part is parallel space converting network.Each spatial alternation network contains 3 portions
Point, it is respectively: 1) positions network, it is made of 3 full articulamentums.Its input is that width is w, and a height of h, port number is the spy of c
Sign figure/, output is a transition matrix θ;2) pixel positioner, it obtains each on output characteristic pattern by transition matrix θ
Position of the point on input feature vector figure, when carrying out becoming privileged processing to transition matrix θ, so that it may produce different types of feature
Figure transformation, the parallel space converting network contain scaling, translate, rotate 3 kinds of mapping modes;3) sampler passes through previous step
Output characteristic pattern is obtained after the sample point coordinate on input feature vector figure, by bilinear interpolation, to the value of sample point coordinate
It is calculated, it is determined that the value of target point is corresponded on output characteristic pattern.
As shown in Figures 5 and 6, it is one image of input that the candidate frame, which generates the effect of network, then output a batch point
The higher candidate frame of number.In the network, the generation of candidate frame uses the mechanism of sliding window combination anchor, to determine each cunning
Whether with the presence of target inside region corresponding to dynamic window.Since the length and width of target are inconsistent, it is therefore desirable to a variety of scales
Window is covered.The mechanism of anchor is on the basis of a benchmark window size, according to length and width and multiple proportional next life
At different candidate frames.The invention generates candidate frame using (8,16,32) three kinds of multiples and (0.5,1,2) three kinds of ratios,
The anchor of available 9 kinds of different scales.
Part IV is the area-of-interest pond sorter network in conjunction with difficult pattern detection.Area-of-interest pondization classification net
Network is made of an area-of-interest layer and two full articulamentums.Its input is a series of candidate frame, by a upper part
Candidate frame generate network generated, output is position and classification information of the neural network for defect, and loss function is as follows:
Lhard=Lclass(p, u)+δ Lregre(t, v)
Lclass(p, u) represents the log loss of classification, Lregre(t, v) represents the recurrence loss of prediction block coordinate;P is represented
The prediction block to it includes object true classification predicted value, 4 set of coordinates that t and v have respectively represented prediction block close
The vector that the vector sum actual position coordinate come combines.4 coordinates are respectively (x, y, w, h), and x represents the opposite of candidate frame
Abscissa, y represent the opposite ordinate of candidate frame, and w represents the width of candidate frame, and h represents the height of candidate frame.δ is for balancing
Classification Loss and coordinate return loss.In the training process, preceding 128 candidate frames of the largest loss are considered as difficult sample, utilized
Small batch stochastic gradient descent method carries out backpropagation, updates network parameter.
The training step of depth convolutional neural networks model of the present embodiment based on improved faster R-CNN are as follows:
1) convolutional neural networks of four parts are initialized respectively;
2) to shared convolutional neural networks, parallel space converting network, candidate frame generates network and is trained, and obtains a system
The candidate frame of column;
3) candidate frame generated using step 2, to shared convolutional neural networks, parallel space converting network and region of interest
Domain pond sorter network is trained, and is picked out the candidate frame of preceding 128 the largest loss every time, is utilized small batch stochastic gradient
Descent algorithm carries out network parameter update;
4) fixed shared convolutional neural networks, parallel space converting network generate network to candidate frame and are individually trained,
Obtain a series of candidate frame;
5) candidate frame generated using step 4, to shared convolutional neural networks, parallel space converting network and region of interest
Domain pond sorter network is trained, and 128 candidate frames of the largest loss is picked out every time, using under small batch stochastic gradient
It drops algorithm and carries out network parameter update;
Open defect detection system of the present invention and method are used for the short circuit, open circuit of route in FICS image, broken
The detection of the defects of damage, holes defect, testing result such as Fig. 7 (a), Fig. 7 (b), Fig. 7 (c), Fig. 7 (d), Fig. 7 (e) and Fig. 7 (f)
It is shown.The method of the present invention is defeated by FICS image to be detected by training the depth convolutional network model based on deep learning
Enter the model trained, can quickly export position and its type information of the defect on FICS image.
The above embodiment is a preferred embodiment of the present invention, but embodiments of the present invention are not by the embodiment
Limitation, other any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present invention,
It should be equivalent substitute mode, be included within the scope of the present invention.