CN104573635B - A kind of little height recognition methods based on three-dimensional reconstruction - Google Patents

A kind of little height recognition methods based on three-dimensional reconstruction Download PDF

Info

Publication number
CN104573635B
CN104573635B CN201410787959.4A CN201410787959A CN104573635B CN 104573635 B CN104573635 B CN 104573635B CN 201410787959 A CN201410787959 A CN 201410787959A CN 104573635 B CN104573635 B CN 104573635B
Authority
CN
China
Prior art keywords
edge
image
color
algorithm
fringe center
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201410787959.4A
Other languages
Chinese (zh)
Other versions
CN104573635A (en
Inventor
杜娟
洪大江
胡跃明
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN201410787959.4A priority Critical patent/CN104573635B/en
Publication of CN104573635A publication Critical patent/CN104573635A/en
Application granted granted Critical
Publication of CN104573635B publication Critical patent/CN104573635B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection

Abstract

The invention discloses a kind of little height recognition methods based on three-dimensional reconstruction, including two-dimensional image data collection, Segmentation of Color Image, coloured image quaternary number mixed filtering algorithm, the detection of coloured image quaternary number vector mixed edge, coloured image quaternary number sub-pix fringe center location algorithm, the fringe center obtained using sub-pixel edge fringe center location algorithm changes difference, substitute into height calculation formula, the height value of characteristic point is tried to achieve, identifies defect type.

Description

A kind of little height recognition methods based on three-dimensional reconstruction
Technical field
The present invention relates to image recognition and defects detection field, more particularly to a kind of little height based on three-dimensional reconstruction is known Other method.
Background technology
Detection in commercial Application at present is mostly to use two-dimensional detection, and the light source of use is mostly that dot laser or line swash Radiant.Three-dimensional values also begin to enter the Preliminary Applications stage now, are mostly projected on light source using high-accuracy projector black White raster or chromatic grating realize, most of related systems are in the reconstruct of three-dimensional color image, and the edge detection of image, On Feature Correspondence Algorithm, mostly it is individually to be handled by tri- kinds of colors of RGB, the association between image color information is artificially shelled From, have impact on the reliability of detection, moreover, essentially all of algorithm be all using solution phase PMP methods it is tested to obtain The final three-dimensional height of object, its precision is difficult to improve.
The content of the invention
In order to overcome shortcoming and deficiency existing in the prior art, the present invention provides a kind of little height based on three-dimensional reconstruction Recognition methods.
The present invention adopts the following technical scheme that：
A kind of little height recognition methods based on three-dimensional reconstruction, includes the following steps：
S1 two-dimensional image datas gather：Specially obtain the RGB structure lights irradiation stripe pattern of chip under test；
S2 image Fast Segmentation Algorithms：Specially the angle point of all pins of chip under test image is positioned, and is made For the benchmark of segmentation；
S3 Algorithm for Color Image Filtering, specially using quaternary number-vector mixed filtering algorithm, to the striped containing elevation information Edge carries out noise filtering and local enhancement；
S4 color images edge detections, specially using the edge detection algorithm of quaternary number-vector, pass through design colours phase Like degree function, adjacent pixel is compared one by one from horizontally and vertically both direction, draws stripe edge and chip side to be measured Edge, the accurate coordinates of altitude feature point are determined by crosspoint, and confirm that the calculating of elevation information edge feature is sat by horizontal edge Punctuate collection；
S5 coloured image sub-pixel edges fringe center positions：Specially use the sub-pix side based on versor Edge stripe centralized positioning algorithm, positions the height fringe center of altitude feature point, draws striped caused by height respectively Center changes difference；
The fringe center change difference that S6 is obtained using sub-pixel edge fringe center location algorithm, substitution height calculates public Formula, tries to achieve the height value of altitude feature point, identifies defect type.
Sub-pix fringe center location algorithm described in the S5 is specially according to versor color similarity letter Number, fringe center coordinate is calculated using interpolation method.
The S2 is specially：
According to the striped reference picture template of 0 contour reference planes with being put into the bar graph collected after chip to be measured The image of object under test is basically separated by difference with background, then carries out threshold binarization treatment to the image after separation, then lead to Cross combinatorial operation and obtain the Basic Contour Line of object under test, and all angle points of Basic Contour Line are obtained with angle point derivation algorithm, And according to reference coordinate of the obtained angle point as image region segmentation.
Quaternary number described in the S3-vector mixed filtering algorithm is right specially based on designed color similarity function Inside striped and borderline region uses different filtering methods, while noise is eliminated, to the striped side containing elevation information Edge carries out local enhancement, and the color similarity function is designed based on quaternary number-vector.
The color similarity function, is specially：
Wherein, fsimilarity(x1,y1,x2,y2) it is pixel (x1,y1) and (x2,y2) similarity function, fQ(x1,y1,x2, y2) be color quaternary number similarity function, T for it is default relatively threshold values, work as fQ>During T, f is usedQ(x1,y1,x2,y2) conduct The final value of similarity function；Work as fQ<During T, f is usedV(x1,y1,x2,y2) final value as similarity function.
Beneficial effects of the present invention：
The present invention is handled the internal association of each colour element R, G, B triple channel as an entirety, to knot The image that structure photosystem collects carries out effective guarantor's side denoising, and carries out precision to height using subpixel method and repair Just, the accuracy and precision of high levels of three-dimensional reconstruction process is substantially increased so that structured light measurement system can apply to precision The defects of electronic manufacture detection field.
Brief description of the drawings
Fig. 1 is a kind of work flow diagram of the little height recognition methods based on three-dimensional reconstruction of the present invention.
Fig. 2 is the Algorithm for Color Image Filtering flow chart of S3 steps of the present invention；
Fig. 3 is the color images edge detection flow chart of S4 steps of the present invention；
Fig. 4 is the work of the sub-pix color fringe centralized positioning algorithm based on versor of S5 steps of the present invention Flow chart；
Fig. 5 is the color similarity function flow chart of the invention based on quaternary number-vector；
Fig. 6 is the present invention based on quaternary number normalization color similarity function algorithm flow chart；
Fig. 7 is three-dimensional height schematic diagram calculation of the invention.
Embodiment
With reference to embodiment and attached drawing, the present invention is described in further detail, but embodiments of the present invention are not It is limited to this.
Embodiment
As shown in Figure 1, a kind of little height recognition methods based on three-dimensional reconstruction, suitable for on-line checking and offline inspection Both of which, specific implementation mainly include following steps：The object under test striped that camera calibration, collection are irradiated through structure light Figure, Fast image segmentation algorithm, cromogram quaternary number-vector mixing are as filtering algorithm, coloured image quaternary number-vector edge inspection Method of determining and calculating, coloured image quaternary number normalization sub-pix fringe center location algorithm and three-dimensional height calculate.
The calibration of camera can be demarcated directly in advance using the camera calibration tool box of matlab, and then export is direct Loading can.
The stripe pattern of chip under test of the collection through structure light irradiation.
Operating range is reduced by Fast image segmentation algorithm first：According to the striped reference picture of 0 contour reference planes The image of object under test is basically separated by template with being put into the difference of the bar graph collected after object under test with background, then right Image after separation carries out threshold binarization treatment, then obtains the Basic Contour Line of object under test by combinatorial operation, and uses angle Point derivation algorithm obtains all angle points of Basic Contour Line, and is sat according to reference of the obtained angle point as image region segmentation Mark；Angle point described in text refers to the rectangular area of chip makes physical form, such as each pin.
The colored quaternary number vector model of image is first established, the Two-dimensional Color Image obtained for coloured image taking module Each pixel with real part be 0 pure quaternion Ri+Gj+Bk represent, in processing procedure afterwards, coloured image RGB tri- Component is uniformly processed as an entirety.
Before relevant treatment algorithm is carried out, also need to design relevant color similarity function as shown in Figure 5：
Wherein, fsimilarity(x1,y1,x2,y2) it is pixel (x1,y1) and (x2,y2) similarity function.fQ(x1,y1,x2, y2) be color quaternary number similarity function, T for it is default relatively threshold values, generally take the numerical value within 10, when numerical value is too big, Different colours are difficult to differentiate between, when numerical value is too small, discrimination is too high, easily produces erroneous judgement.Work as fQ>During T, f is usedQ(x1,y1,x2, y2) final value as similarity function；Work as fQ<During T, f is usedV(x1,y1,x2,y2) taken as the final of similarity function Value.fQ(x1,y1,x2,y2) design be described as follows：
Set hereinAnother function fV(x1,y1,x2,y2) be described as follows：
fV(x1,y1,x2,y2)=| | f (x1,y1)-f(x2,y2)||2
Wherein, fq(x1,y1) with subscript " q " it is colour element (x1,y1) quaternary number represent, f (x1,y1) not subscripting " q " for corresponding vector representation, similar unified representation is also used for other colour elements.fq12gray(x, y) represents distance Colour element fq12(x, y) immediate grey level value.||·||2Euclidean distance between colour element.
The filter enhancement algorithm of coloured image as shown in Figure 2 should filter out noise, while strengthen elevation edge information again. Using the color similarity function designed before, classify to pixel to be filtered, be respectively adopted and directly skip, based on quaternary The medium filtering of number-vector and the bilateral filtering algorithm noise reduction based on quaternary number-vector, and strengthen elevation edge information.It is related Formula is described as follows：
Wherein, fq,new(x, y) is filtered center pixel value, fq(x, y) is the center pixel value before filtering, fQVBF(x, y) is quaternary number filter function, mainly carries out filtering process to border point；fQVMF(x, y) is that quaternary number medium filtering is calculated Method, it is main to handle noise acnode；T is similarity pre-set threshold value.Quaternary number-Vector median filtering algorithmic formula is described as follows：
Wherein, (xc,yc) it is some the pixel point coordinates filtered in neighborhood.Quaternary number-vector bilateral filtering algorithmic formula is retouched State as follows：
Wherein, w (x, y, xk,yk) it is weight coefficient, rq(x,y,xk,yk) it is codomain core, d (x, y, xk,yk), it is domain Core, △ dk 2For pixel f in filtering neighborhoodq(xk,yk) arrive centre of neighbourhood pixel fqThe Euclidean distance square of (x, y).In application, can According to the characteristic distributions of pending object, suitable σ is chosenrqAnd σdParameter, it is possible to carried out to border point at relevant filtering Reason.
Then, as shown in figure 3, performing relevant Edge Detection based on color image in the region filtered.Equally The color similarity function designed before is used, from horizontal and vertical directions, each pixel is identified, is respectively obtained Vertical edge point set and V-Array horizontal edge point sets H-Array.Again by the intersection of two point sets, it is possible to obtain height and become Change the accurate coordinates point set C-Array of characteristic point.
Associated vertical edge detection algorithm is described as follows：
Related levels edge detection algorithm is described as follows：
Wherein, TverticalIt is vertical detection threshold values, ThorizontalIt is horizontal detection threshold.
As shown in figure 4, according to obtained point set V-Array, H-Array and C-Array, relevant sub-pix striped is carried out Centralized positioning algorithm.Design the normalization color similarity function based on quaternary number, as shown in fig. 6, obtain color pixel cell with The versor distance of unit grey level.Related algorithm is described as follows：
Wherein, fnorm-simi(x1,y1) be color pixel cell versor distance, fQ(x1,y1, gray) and it is colour The actual quaternary number distance of pixel and unit grey level, fQ-maxFor color pixel cell and unit grey level maximum quaternary number away from From.
The result of calculation for normalizing color similarity function is updated in sub-pix fringe center ranging formula
Wherein,Y is respectively arranged for height variation feature point the right and left horizontal border linecolFringe center coordinate Value, then weighted average is sought, obtain left altitude feature fringe center rowcenter,leftWith left altitude feature fringe center rowcenter,right
△ row=rowcenter,left-rowcenter,right
Wherein, △ row are the change difference of height fringe center.
The number of pixels △ row of height fringes shift are calculated, and then are converted into the actual range AC of offset (due to being The hardware parameter of system is pre-set, thus one millimeter accounted in the picture at 0 contour how many a pixels be can be advance Measure).
Height calculation formula as shown in Figure 7, for red light, its height calculates as follows：
Wherein, L is vertical range of the camera to reference planes, drIt is corresponding camera lens central point into structure light The distance of heart point,It is the distance of respective heights fringes shift.
Similarly, it is not as follows for green light and blue light, altimeter point counting：
Wherein, dg、dbIt is distance of the camera lens central point to corresponding color structure light central point, It is The distance of respective heights fringes shift.
Finally, by the elevation information being calculated, contrast standard parameter, it is possible to identify the defects of corresponding.
Above-described embodiment is the preferable embodiment of the present invention, but embodiments of the present invention and from the embodiment Limitation, other any Spirit Essences without departing from the present invention with made under principle change, modification, replacement, combine, simplification, Equivalent substitute mode is should be, is included within protection scope of the present invention.

Claims (3)

1. a kind of little height recognition methods based on three-dimensional reconstruction, it is characterised in that include the following steps：
S1 two-dimensional image datas gather：Obtain the RGB structure lights irradiation stripe pattern of chip under test；
S2 image Fast Segmentation Algorithms：The angle point of all pins of chip under test image is positioned, and as the base of segmentation It is accurate；
S3 Algorithm for Color Image Filtering：Specifically utilize quaternary number-vector mixed filtering algorithm, to the stripe edge containing elevation information into Row noise filtering and local enhancement；
S4 color images edge detections, the specific edge detection algorithm for using quaternary number-vector, pass through design colours similarity letter Number, adjacent pixel is compared from horizontally and vertically both direction, stripe edge and chip edge to be measured are drawn, by handing over one by one Crunode determines the accurate coordinates of characteristic point, and the coordinates computed point set of elevation information edge feature is confirmed by horizontal edge；
S5 coloured image sub-pixel edges fringe center positions：Specially use the sub-pixel edge bar based on versor Line centralized positioning algorithm, positions characteristic point height fringe center, draws fringe center difference in change caused by height respectively Value；
Sub-pixel edge fringe center location algorithm described in the S5 is specially according to versor color similarity letter Number, fringe center coordinate is calculated using interpolation method；
The color similarity function is：
Wherein, fsimilarity(x1,y1,x2,y2) it is pixel (x1,y1) and (x2,y2) similarity function, fQ(x1,y1,x2,y2) be The quaternary number similarity function of color, T are default relatively threshold values, work as fQ>During T, f is usedQ(x1,y1,x2,y2) it is used as similarity The final value of function；Work as fQ<During T, f is usedV(x1,y1,x2,y2) final value as similarity function；
The sub-pix fringe center coordinate formula is：
Wherein,Y is respectively arranged for height variation feature point the right and left horizontal border linecolFringe center coordinate value； fsimi-norm(xk,ycol) be color pixel cell versor distance；
The fringe center change difference that S6 is obtained using sub-pixel edge fringe center location algorithm, substitutes into height calculation formula, The height value of characteristic point is tried to achieve, identifies defect type.
2. recognition methods according to claim 1, it is characterised in that the S2 is specially：
According to the striped reference picture template of 0 contour reference planes and the difference for being put into the bar graph collected after chip to be measured The image of object under test is basically separated with background, threshold binarization treatment then is carried out to the image after separation, then pass through group Close computing and obtain the Basic Contour Line of object under test, and all angle points of Basic Contour Line are obtained with angle point derivation algorithm, and root According to reference coordinate of the obtained angle point as image region segmentation.
3. recognition methods according to claim 1, it is characterised in that quaternary number described in the S3-vector mixed filtering is calculated Method is specially based on designed color similarity function, and different filtering methods is used to striped inside and borderline region, While eliminating noise, local enhancement is carried out to the stripe edge containing elevation information, the color similarity function is to be based on Quaternary number-vector design.
CN201410787959.4A 2014-12-17 2014-12-17 A kind of little height recognition methods based on three-dimensional reconstruction Active CN104573635B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201410787959.4A CN104573635B (en) 2014-12-17 2014-12-17 A kind of little height recognition methods based on three-dimensional reconstruction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201410787959.4A CN104573635B (en) 2014-12-17 2014-12-17 A kind of little height recognition methods based on three-dimensional reconstruction

Publications (2)

Publication Number Publication Date
CN104573635A CN104573635A (en) 2015-04-29
CN104573635B true CN104573635B (en) 2018-04-13

Family

ID=53089658

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201410787959.4A Active CN104573635B (en) 2014-12-17 2014-12-17 A kind of little height recognition methods based on three-dimensional reconstruction

Country Status (1)

Country Link
CN (1) CN104573635B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107271445B (en) * 2017-05-16 2020-10-16 广州视源电子科技股份有限公司 Defect detection method and device
CN112701060B (en) * 2021-03-24 2021-08-06 高视科技(苏州)有限公司 Method and device for detecting bonding wire of semiconductor chip
CN113240636B (en) * 2021-05-08 2022-06-21 苏州天准科技股份有限公司 Surface navigation intelligent detection method, system, storage medium and terminal equipment
CN116309589B (en) * 2023-05-22 2023-08-01 季华实验室 Sheet metal part surface defect detection method and device, electronic equipment and storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103543167A (en) * 2013-10-08 2014-01-29 华南理工大学 Knowledge base-based three-dimensional X-ray computed tomography (CT) detection system and method
CN103559499A (en) * 2013-10-09 2014-02-05 华南理工大学 RGB vector matching rapid-recognition system and method
CN104075659A (en) * 2014-06-24 2014-10-01 华南理工大学 Three-dimensional imaging recognition method based on RGB structure light source

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2138091B1 (en) * 2007-04-24 2013-06-19 Olympus Medical Systems Corp. Medical image processing apparatus and medical image processing method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103543167A (en) * 2013-10-08 2014-01-29 华南理工大学 Knowledge base-based three-dimensional X-ray computed tomography (CT) detection system and method
CN103559499A (en) * 2013-10-09 2014-02-05 华南理工大学 RGB vector matching rapid-recognition system and method
CN104075659A (en) * 2014-06-24 2014-10-01 华南理工大学 Three-dimensional imaging recognition method based on RGB structure light source

Also Published As

Publication number Publication date
CN104573635A (en) 2015-04-29

Similar Documents

Publication Publication Date Title
CN107203973B (en) Sub-pixel positioning method for center line laser of three-dimensional laser scanning system
CN106340044B (en) Join automatic calibration method and caliberating device outside video camera
CN105957059B (en) Electronic component missing part detection method and system
CN107179322A (en) A kind of bridge bottom crack detection method based on binocular vision
KR20170139590A (en) Colony contrast collection
CN110930390B (en) Chip pin missing detection method based on semi-supervised deep learning
CN104573635B (en) A kind of little height recognition methods based on three-dimensional reconstruction
CN112651968B (en) Wood board deformation and pit detection method based on depth information
CN104075659B (en) A kind of three-dimensional imaging recognition methods based on RGB structure light source
KR102073468B1 (en) System and method for scoring color candidate poses against a color image in a vision system
CN103679656B (en) A kind of Automated sharpening of images method
CN104899888B (en) A kind of image sub-pixel edge detection method based on Legendre squares
CN104537651B (en) Proportion detecting method and system for cracks in road surface image
US11769274B2 (en) Image processing apparatus, method and storage medium, for object color evaluation
CN105147311A (en) Visual equipment assisted scanning and positioning method and system applied to CT system
CN115100206B (en) Printing defect identification method for textile with periodic pattern
CN106535740A (en) Grading corneal fluorescein staining
CN104574312A (en) Method and device of calculating center of circle for target image
JP2013238449A (en) Crack detection method
CN108601509A (en) Image processing apparatus, image processing method and program
CN115272256A (en) Sub-pixel level sensing optical fiber path Gaussian extraction method and system
CN114331986A (en) Dam crack identification and measurement method based on unmanned aerial vehicle vision
CN114299070A (en) Method and related device for detecting mura defects of display screen
CN116503316A (en) Chip defect measurement method and system based on image processing
JP3661635B2 (en) Image processing method and apparatus

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant