CN104036492B - A kind of fruit image matching process based on spot extraction with neighbor point vector method - Google Patents
A kind of fruit image matching process based on spot extraction with neighbor point vector method Download PDFInfo
- Publication number
- CN104036492B CN104036492B CN201410215892.7A CN201410215892A CN104036492B CN 104036492 B CN104036492 B CN 104036492B CN 201410215892 A CN201410215892 A CN 201410215892A CN 104036492 B CN104036492 B CN 104036492B
- Authority
- CN
- China
- Prior art keywords
- point
- vector
- match point
- reconnaissance
- match
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 235000013399 edible fruits Nutrition 0.000 title claims abstract description 55
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000008569 process Effects 0.000 title claims abstract description 16
- 238000000605 extraction Methods 0.000 title claims abstract description 11
- 238000001514 detection method Methods 0.000 claims abstract description 23
- 230000008878 coupling Effects 0.000 claims abstract description 20
- 238000010168 coupling process Methods 0.000 claims abstract description 20
- 238000005859 coupling reaction Methods 0.000 claims abstract description 20
- 238000012216 screening Methods 0.000 claims abstract description 3
- 239000000284 extract Substances 0.000 claims description 10
- 230000000877 morphologic effect Effects 0.000 claims description 9
- 241001164374 Calyx Species 0.000 claims description 6
- 230000010339 dilation Effects 0.000 claims description 6
- 230000003628 erosive effect Effects 0.000 claims description 6
- 239000012141 concentrate Substances 0.000 claims description 5
- 235000006596 Salacca edulis Nutrition 0.000 claims description 4
- 244000208345 Salacca edulis Species 0.000 claims description 4
- 238000013459 approach Methods 0.000 abstract description 3
- 238000003708 edge detection Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 1
- 241001269238 Data Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000000205 computational method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a kind of extraction and the fruit image matching process of neighbor point vector method based on spot.Obtain the left and right side image of fruit: carry out spot extraction, be respectively adopted extreme point detection method, Harris corner detection approach, Canny edge detection method obtain left figure dot set and right figure dot set；Each of which point is carried out matching judgment, obtains coupling point set to be selected；Match point collection to be selected carries out Mismatching point rejecting, and screening obtains correct coupling point set；Complete the coupling of fruit image.The present invention is extracted by spot and spot carries out vector method and judges to make fruit image coupling achieve excellent stability, accuracy and real-time.
Description
Technical field
The present invention relates to a kind of fruit image matching process, especially relate to a kind of base of technical field of image processing
The fruit image matching process with neighbor point vector method is extracted in spot.
Background technology
Images match refers to identify same point between two width or multiple image by certain matching process.Closely
Nian Lai, images match oneself become object identification, robot map perception and navigation, image are sewed up, 3D model
Foundation, gesture identification, image tracing and action are than the key technology in reciprocity image analysis processing field and research
Focus.
The acquisition of fruit surface information is the index of quality such as the size of fruit, shape, surface color and surface defect
The basis of detection.The degree of accuracy of surface color and surface defects detection depends on obtaining of fruit full surface image
Taking, image mosaic technology is the key realizing fruit full surface Image Acquisition, and image matching technology is figure
Basis as splicing.
SIFT (Scale Invariant Feature Transform) method is that David Lowe proposed in 1999
Local feature description son (David.G.Lowe.Object recognition from local scale-invariant
features.International Conference on Computer Vision,Corfu,Greece,1999:1150-
1157), and in 2004 carried out deeper into development and perfect (David.G.Lowe.Distinctive
image features from scale-invariant keypoints[J].International Journal of Computer
Vision,2004,60(2):91-110).The SIFT feature vector extracted is to rotation, scaling, brightness
Change maintains the invariance, and visual angle change, affine transformation, noise are also kept a certain degree of stability.
But method is required higher with classification by the on-line checking of fruit, on the one hand, SIFT method is raising
Joining adaptability and cause the complexity of the method, the most computationally intensive, the longest, former SIFT method cannot expire
The online requirement of foot；On the other hand, conventional Mismatching point elimination method RANSAC method (Chum O,
Matas J.Optimal randomized RANSAC[J].IEEE Trans.On Pattern Analysis and
Machine Intelligence, 2008,30 (8): 1472-1482) basic assumption is to comprise correct data in sample
(inliers, the data that can be described by model), (Outliers, deviation normal range (NR) is also to comprise abnormal data very much
Far, the data of Mathematical Modeling cannot be adapted to), i.e. containing noise in data set.These abnormal datas be probably by
Produce in wrong measurement, the hypothesis of mistake, the calculating etc. of mistake, and fruit encoded colors feature difference
Less making fruit image based on SIFT method coupling cannot stably obtain match point, once coupling obtains
Coupling to count be 0 to multiple, the Mismatching point that such method therefore cannot be used to carry out fruit image is rejected.
Summary of the invention
In order to solve problem present in background technology, the present invention proposes a kind of to extract and neighbor point based on spot
The fruit image matching process of vector method.
The technical solution adopted for the present invention to solve the technical problems is to comprise the steps:
1) fruit side image is obtained:
Fruit is placed on fruit tray so that the calyx carpopodium line of fruit is substantially vertical with horizontal plane,
Gather a width left surface image from fruit side, then with the calyx carpopodium line of fruit as axle, rotate fruit 60 °
After gather a width right hand view picture again；
2) carry out spot extraction, obtain left figure dot set and right figure dot set；
3) to step 2) each point of the left and right figure dot set that obtains carries out matching judgment, obtains to be selected
Coupling point set；
4) to step 3) match point to be selected that obtains concentrates and each match point to be selected carried out Mismatching point rejecting,
Screening obtains correct match point；
5) coupling of fruit image is completed.
Described step 2) in carry out spot and extract and include step in detail below:
2.1) extreme point detection:
First left surface image and right hand view picture (1) as follows are calculated, obtain initial Gaussian left
Figure, the right figure of initial Gaussian:
L (x, y, σ)=G (x, y, σ) * I (x, y) (1)
Wherein, L (x, y, σ) is calculated initial Gaussian image, I (x, y) is side image to be calculated,
The expression formula of G (x, y, σ) is below equation (2):
Wherein, σ yardstick coordinate, x, y are respectively the transverse and longitudinal coordinate of left surface image or right hand view picture；
Then left for initial Gaussian figure, the right figure of initial Gaussian are put and be twice, obtain the left figure of ground floor Gauss and first
This right figure of floor height；Again left for ground floor Gauss figure and the right figure of ground floor Gauss are calculated by above-mentioned formula (1) again,
To the left figure of second layer Gauss and the right figure of second layer Gauss；
Finally left for second layer Gauss figure is deducted the image that the left figure of ground floor Gauss obtains and obtains the left figure of difference of Gaussian,
Right for second layer Gauss figure is deducted the image that the right figure of ground floor Gauss obtains and obtains the right figure of difference of Gaussian；
Each pixel p in figure left for difference of Gaussian and the right figure of difference of Gaussian, if the gray scale of pixel p
Value is respectively less than or is all higher than the gray value of remaining each pixel in 3 × 3 neighborhoods centered by pixel p,
Then pixel p is labeled as extreme point；
Each pixel of figure left to difference of Gaussian and the right figure of difference of Gaussian travels through, and respectively obtains left figure
Difference of Gaussian extreme value point set and right figure difference of Gaussian extreme value point set；
2.2) Harris Corner Detection:
To step 1) the left surface image that obtains and right hand view picture carry out Harris Corner Detection, respectively respectively
Obtain left figure Harris angle point collection and right figure Harris angle point collection；
2.3) Canny rim detection:
By step 1) the left surface image that obtains and right hand view picture carry out Canny rim detection respectively, obtain
Left figure edge image and right figure edge image, carry out a form to left figure edge image and right figure edge image
Learn dilation operation, then the image obtained after morphological dilations computing made contours extract and fills all profiles,
Reject wherein area more than the morphological erosion computing that tries again after the profile of 100 pixels, the most right
The image obtained after morphological erosion computing is made contours extract and is exported by all profile center point coordinates, respectively
Obtain left map contour center point set and right map contour center point set；
Then left figure difference of Gaussian extreme value point set, left figure Harris angle point collection and left map contour center point set are entered
Row merges, the point repeated for coordinate, retains one of them and rejects the point that remaining coordinate repeats, obtaining a left side
Figure dot set；By right figure difference of Gaussian extreme value point set, right figure Harris angle point collection and right map contour center point set
Merge, the point that coordinate is repeated, retain one of them and reject the point that remaining coordinate repeats, obtaining
Right figure dot set.
Described step 3) in each point of left figure dot set is carried out with each point of right figure dot set
Join judgement and include step in detail below:
3.1) in left figure dot set treat reconnaissance A centered by, select 70 × 70 rectangular area, search
Point closest with treating reconnaissance A in being in this rectangular area in left figure dot set, is designated as a B, and calculates
Treat the reconnaissance A vector to some B, be designated as
3.2) search in right figure dot set and within 6 pixels, treat reconnaissance A ' with treating reconnaissance A Diff N,
Centered by treating reconnaissance A ', select the rectangular area of 70 × 70, search in right figure dot set and be in this rectangle region
With some B Diff N point within 6 pixels and the point nearest with treating reconnaissance A ' in territory, it is designated as a little
B ', calculates the vector treating reconnaissance A ' to some B 'If can not find this B ', giving up and treating that reconnaissance A is with to be selected
Point A ' is as a pair match point, and terminates to treat reconnaissance A and the remaining steps treating reconnaissance A ' matching judgment；
3.3) search in left figure dot set and be in 70 × 70 rectangular areas centered by treating reconnaissance A and treat
The point that reconnaissance A distance time is near, is designated as a C, calculates the vector treating reconnaissance A to some CIf can not find this
Point C then gives up and treats reconnaissance A and treat reconnaissance A ' as a pair match point, and terminates treat reconnaissance A and treat reconnaissance
The remaining steps of A ' matching judgment；
3.4) search in right figure dot set and be in 70 × 70 rectangular areas centered by treating reconnaissance A ' and put C
Diff N point within 6 pixels and the point nearest with treating reconnaissance A ', be designated as a C ', and calculate and treat
Reconnaissance A ' is to the vector of a C 'If can not find this C ', giving up and treating reconnaissance A and treat reconnaissance A ' conduct
A pair match point, and terminate to treat reconnaissance A and the remaining steps treating reconnaissance A ' matching judgment；
3.5) formula (3) is used to calculate vector respectivelyWith vectorAngle α and vectorWith vector
Angle β, use formula (4) calculate vectorWith vectorDifference dis of modulus value1And vectorWith vectorDifference dis of modulus value2:
Wherein, vectorA, b are respectively vectorTransverse and longitudinal coordinate, vector
C, d are respectively vectorTransverse and longitudinal coordinate；
If it is vectorialWith vectorAngle α < 15 ° andWith vectorDifference dis of modulus value1< 8 and to
AmountWith vectorAngle β < 18 ° and vectorWith vectorDifference dis of modulus value2< 10, then will treat
Reconnaissance A with treat that reconnaissance A ', as a pair match point to be selected, and terminates to treat in reconnaissance A and right figure dot set surplus
The matching judgment of remaining point, otherwise gives up and treats reconnaissance A and treat reconnaissance A ' as a pair match point to be selected.
Described step 4) in carry out Mismatching point and reject and specifically use following steps:
4.1) marginal point is rejected:
Match point to be selected concentration is a certain respectively to be treated in left surface image and right hand view picture match point to be selected
Select match point P and match point P ' to be selected, and centered by match point P to be selected and match point P ' to be selected, respectively
Choose the rectangular area of 20 × 20, if exist in any one rectangular area in two rectangular areas red, green,
Blue three-component value is all higher than the pixel of 200, then give up match point P to be selected and match point P ' to be selected as one
To correct match point；
4.2) first round vector determination is carried out:
4.2.1) if the above-mentioned logarithm of match point to be selected concentration match point that obtains is less than 3, step is the most directly carried out
4.3)；
4.2.2) in left surface image, centered by match point P to be selected, the rectangular area of 70 × 70 is selected,
Match point to be selected closest with some P in searching for this rectangular area, is designated as match point Q to be selected, and calculating is treated
Select match point P to the vector of match point Q to be selected
If can not find this match point Q to be selected, giving up match point P to be selected and match point P ' to be selected and aligning as one
True match point, and terminate the remaining of match point P to be selected and match point P ' to be selected is judged step；
4.2.3) in right hand view picture corresponding for match point Q to be selected with left surface image, match point to be selected is Q ',
Calculate the vector of match point P ' to be selected to match point Q ' to be selected
4.2.4) in left surface image, in the search 70 × 70 rectangular areas centered by match point P to be selected and
The match point to be selected that match point P to be selected distance time is near, is designated as match point R to be selected, calculates match point P to be selected
Vector to match point R to be selected
If can not find this match point R to be selected, give up match point P to be selected and match point P ' to be selected as a pair
Join a little, and terminate the remaining of match point P to be selected and match point P ' to be selected is judged step；
4.2.5) in right hand view picture corresponding for match point R to be selected with left surface image, match point to be selected is R ',
Calculate the vector of match point P ' to be selected to match point R ' to be selected
4.2.6) formula (3) is used to calculate vector respectivelyWith vectorAngle ρ and vectorWith vectorAngle μ, use formula (4) calculate vectorWith vectorDifference dis of modulus value3And vectorWith
VectorDifference dis of modulus value4；
4.2.7) if meeting vectorWith vectorAngle ρ < 15 ° and vectorWith vectorModulus value
Difference dis3< 8, or meet vectorWith vectorAngle μ < 18 ° and vectorWith vectorMould
Difference dis of value4< 10, then proceed below step, otherwise give up match point P to be selected and match point P ' to be selected
As a pair correct match point；
4.3) second vector determination is taken turns；
To step 4.2.7) the match point P to be selected that obtains and match point P ' repeat the above steps 4.2.1 to be selected)
～4.2.6), again obtain vectorWith vectorAngle ρ, vectorWith vectorThe difference of modulus value
dis3, vectorWith vectorAngle μ, vectorWith vectorDifference dis of modulus value4If, full
Foot vectorWith vectorAngle ρ < 15 ° and vectorWith vectorDifference dis of modulus value3< 8 and to
AmountWith vectorAngle μ < 18 ° and vectorWith vectorDifference dis of modulus value4< 10, this is treated
Selecting match point P and match point P ' to be selected is correct match point, otherwise gives up match point P to be selected and to be selected
Join a P ' as a pair correct match point.
Described fruit is snake fruit.
Described left surface image and the resolution ratio of right hand view picture are 0.146mm/pixel.
Usefulness of the present invention is:
The present invention is extracted by spot and spot carries out vector method and judges fruit image coupling is obtained
Excellent stability, accuracy and real-time.
Accompanying drawing explanation
Fig. 1 is the matching process broad flow diagram of the present invention.
Fig. 2 is two width fruit original images of the embodiment of the present invention.
Fig. 3 is the left figure of ground floor Gauss and the left figure of second layer Gauss of the embodiment of the present invention.
Fig. 4 is the left figure of difference of Gaussian of the embodiment of the present invention.
Fig. 5 is that the embodiment of the present invention detects the spot coordinate diagram obtained based on difference of Gaussian image extreme point.
Fig. 6 is the spot coordinate diagram that the embodiment of the present invention obtains based on Harris Corner Detection.
Fig. 7 is that the embodiment of the present invention obtains the step of spot coordinate based on Canny rim detection.
Fig. 8 is the spot coordinate diagram that the embodiment of the present invention obtains based on Canny rim detection.
Fig. 9 is that in embodiment of the present invention spot matching process, closest approach chooses schematic diagram with time near point.
Figure 10 is embodiment of the present invention spot matching judgment process realization figure.
Figure 11 is that embodiment of the present invention marginal point rejects schematic diagram.
Figure 12 is that embodiment of the present invention Mismatching point rejects process realization figure.
Figure 13 is the coupling point diagram that the embodiment of the present invention obtains.
Detailed description of the invention
The present invention is further illustrated with embodiment below in conjunction with the accompanying drawings.
As it is shown in figure 1, the inventive method comprises the steps:
1) fruit side image is obtained:
Fruit is placed on fruit tray so that the calyx carpopodium line of fruit is substantially vertical with horizontal plane,
Gather a width left surface image from fruit side, then with the calyx carpopodium line of fruit as axle, rotate fruit 60 °
After gather a width right hand view picture again.
2) carrying out spot extraction, spot extracts and includes three below method, obtains left figure dot set and right figure spot
Point set:
2.1) extreme point detection:
First left surface image and right hand view picture (1) as follows are calculated, obtain initial Gaussian left
Figure, the right figure of initial Gaussian:
L (x, y, σ)=G (x, y, σ) * I (x, y) (1)
Wherein, L (x, y, σ) is calculated initial Gaussian image, I (x, y) is side image to be calculated,
The expression formula of G (x, y, σ) is below equation (2):
Wherein, σ yardstick coordinate, x, y are respectively the transverse and longitudinal coordinate of left surface image or right hand view picture；
Then left for initial Gaussian figure, the right figure of initial Gaussian are put and be twice, obtain the left figure of ground floor Gauss and first
This right figure of floor height；Again left for ground floor Gauss figure and the right figure of ground floor Gauss are calculated by above-mentioned formula (1) again,
To the left figure of second layer Gauss and the right figure of second layer Gauss；
Finally left for second layer Gauss figure is deducted the image that the left figure of ground floor Gauss obtains and obtains the left figure of difference of Gaussian,
Right for second layer Gauss figure is deducted the image that the right figure of ground floor Gauss obtains and obtains the right figure of difference of Gaussian；
Each pixel p in figure left for difference of Gaussian and the right figure of difference of Gaussian, if the gray scale of pixel p
Value is respectively less than or is all higher than the gray value of remaining each pixel in 3 × 3 neighborhoods centered by pixel p,
Then pixel p is labeled as extreme point；
Each pixel of figure left to difference of Gaussian and the right figure of difference of Gaussian travels through, and respectively obtains left figure
Difference of Gaussian extreme value point set and right figure difference of Gaussian extreme value point set.
2.2) Harris Corner Detection:
To step 1) the left surface image that obtains and right hand view picture carry out Harris Corner Detection (Chris respectively
Harris,Mike Stephens,A Combined Corner and Edge Detector,4th Alvey Vision
Conference, 1988, pp147-151), respectively obtain left figure Harris angle point collection and right figure Harris angle point
Collection.
2.3) Canny rim detection:
By step 1) the left surface image that obtains and right hand view picture carry out respectively Canny rim detection (Canny,
J., A Computational Approach To Edge Detection, IEEE Trans.Pattern Analysis
And Machine Intelligence, 8:679-714,1986.), obtain left figure edge image and right figure edge image,
Left figure edge image and right figure edge image are carried out again a morphological dilations computing (Rafael C.Gonzalez,
Richard E.Woods, Digital Image Processing, Third Edition, 2010, pp402-442), then
The image obtained after morphological dilations computing made contours extract and fills all profiles, rejecting wherein area big
The morphological erosion that tries again after the profile of 100 pixels computing (Rafael C.Gonzalez, Richard E.
Woods, Digital Image Processing, Third Edition, 2010, pp402-442), the most again to form
The image obtained after learning erosion operation is made contours extract and is exported by all profile center point coordinates, respectively obtains
Left map contour center point set and right map contour center point set；
Then left figure difference of Gaussian extreme value point set, left figure Harris angle point collection and left map contour center point set are entered
Row merges, the point repeated for coordinate, retains one of them and rejects the point that remaining coordinate repeats, obtaining a left side
Figure dot set；By right figure difference of Gaussian extreme value point set, right figure Harris angle point collection and right map contour center point set
Merge, the point that coordinate is repeated, retain one of them and reject the point that remaining coordinate repeats, obtaining
Right figure dot set.
3) spot coupling is carried out, to step 2) each point of the left figure dot set that obtains and right figure dot set
Each point carry out matching judgment and include step in detail below:
3.1) in left figure dot set treat reconnaissance A centered by, select 70 × 70 rectangular area, search
Point closest with treating reconnaissance A in being in this rectangular area in left figure dot set, is designated as a B, and calculates
Treat the reconnaissance A vector to some B, be designated as
3.2) search in right figure dot set and within 6 pixels, treat reconnaissance A ' with treating reconnaissance A Diff N,
Centered by treating reconnaissance A ', select the rectangular area of 70 × 70, search in right figure dot set and be in this rectangle region
With some B Diff N point within 6 pixels and the point nearest with treating reconnaissance A ' in territory, it is designated as a little
B ', calculates the vector treating reconnaissance A ' to some B '
If can not find this B ', giving up and treating reconnaissance A and treat reconnaissance A ' as a pair match point, and terminating point
A to be selected and the remaining steps treating reconnaissance A ' matching judgment；
3.3) search in left figure dot set and be in 70 × 70 rectangular areas centered by treating reconnaissance A and treat
The point that reconnaissance A distance time is near, is designated as a C, calculates the vector treating reconnaissance A to some CIf can not find this
Point C then gives up and treats reconnaissance A and treat reconnaissance A ' as a pair match point, and terminates to mate an A with putting A '
Judge remaining steps.
3.4) search in right figure dot set and be in 70 × 70 rectangular areas centered by treating reconnaissance A ' and put C
Diff N point within 6 pixels and the point nearest with treating reconnaissance A ', be designated as a C ', and calculate and treat
Reconnaissance A ' is to the vector of a C 'If can not find this C ', giving up and treating reconnaissance A and treat reconnaissance A ' conduct
A pair match point, and terminate to treat reconnaissance A and the remaining steps treating reconnaissance A ' matching judgment.
3.5) formula (3) is used to calculate vector respectivelyWith vectorAngle α and vectorWith vector
Angle β, use formula (4) calculate vectorWith vectorDifference dis of modulus value1And vectorWith vectorDifference dis of modulus value2:
Wherein vectorA, b are respectively vectorTransverse and longitudinal coordinate, vector
C, d are respectively vectorTransverse and longitudinal coordinate；During substitution, vectorVectorPoint
Wei vectorAnd vector
If it is vectorialWith vectorAngle α < 15 ° andWith vectorDifference dis of modulus value1< 8 and to
AmountWith vectorAngle β < 18 ° and vectorWith vectorDifference dis of modulus value2< 10, then will treat
Reconnaissance A with treat that reconnaissance A ', as a pair match point to be selected, and terminates to treat in reconnaissance A and right figure dot set surplus
The matching judgment of remaining point, otherwise gives up and treats reconnaissance A and treat reconnaissance A ' as a pair match point to be selected.
4) Mismatching point is rejected, to step 3) match point to be selected that obtains concentrate each to match point to be selected by
Following steps are screened respectively, obtain correct match point:
4.1) marginal point is rejected:
Match point to be selected concentration is a certain respectively to be treated in left surface image and right hand view picture match point to be selected
Select match point P and match point P ' to be selected, and centered by match point P to be selected and match point P ' to be selected, respectively
Choose the rectangular area of 20 × 20, if exist in any one rectangular area in two rectangular areas red, green,
Blue three-component value is all higher than the pixel of 200, then give up match point P to be selected and match point P ' to be selected as one
To correct match point.
4.2) first round vector determination is carried out:
4.2.1) if the above-mentioned logarithm of match point to be selected concentration match point that obtains is less than 3, step is the most directly carried out
4.3)；
4.2.2) in left surface image, centered by match point P to be selected, the rectangular area of 70 × 70 is selected；
Match point to be selected closest with some P in searching for this rectangular area, is designated as match point Q to be selected, meter
Calculate the match point P to be selected vector to match point Q to be selected
If can not find this match point Q to be selected, giving up match point P to be selected and match point P ' to be selected and aligning as one
True match point, and terminate the remaining of match point P to be selected and match point P ' to be selected is judged step；
4.2.3) in right hand view picture corresponding for match point Q to be selected with left surface image, match point to be selected is Q ',
Calculate the vector of match point P ' to be selected to match point Q ' to be selected
4.2.4) in left surface image, in the search 70 × 70 rectangular areas centered by match point P to be selected and
The match point to be selected that match point P to be selected distance time is near, is designated as match point R to be selected, calculates match point P to be selected
Vector to match point R to be selected
If can not find this match point R to be selected, giving up match point P to be selected and match point P ' to be selected and aligning as one
True match point, and terminate the remaining of match point P to be selected and match point P ' to be selected is judged step；
4.2.5) in right hand view picture corresponding for match point R to be selected with left surface image, match point to be selected is R ',
Calculate the vector of match point P ' to be selected to match point R ' to be selected
4.2.6) formula (3) is used to calculate vector respectivelyWith vectorAngle ρ and vectorWith vectorAngle μ, use formula (4) calculate vector respectivelyWith vectorDifference dis of modulus value3And vector
With vectorDifference dis of modulus value4；
Vector during substitution, in formula (3) and formula (4)VectorIt is respectively
VectorAnd vector
4.2.7) if meeting vectorWith vectorAngle ρ < 15 ° and vectorWith vectorModulus value
Difference dis3< 8, or meet vectorWith vectorAngle μ < 18 ° and vectorWith vectorMould
Difference dis of value4< 10, then proceed below step, otherwise give up match point P to be selected and match point P ' to be selected
As a pair correct match point；
4.3) second vector determination is taken turns；
To step 4.2.7) the match point P to be selected that obtains and match point P ' repeat the above steps 4.2.1 to be selected)
～4.2.6), again obtain vectorWith vectorAngle ρ, vectorWith vectorThe difference of modulus value
dis3, vectorWith vectorAngle μ, vectorWith vectorDifference dis of modulus value4If, full
Foot vectorWith vectorAngle ρ < 15 ° and vectorWith vectorDifference dis of modulus value3< 8 and to
AmountWith vectorAngle μ < 18 ° and vectorWith vectorDifference dis of modulus value4< 10, this is treated
Selecting match point P and match point P ' to be selected is correct match point, otherwise gives up.
5) coupling of fruit image is completed.
Described fruit is snake fruit.
Described left surface image and the resolution ratio of right hand view picture are 0.146mm/pixel.
When left surface image and right flank Image Acquisition, adjustable object distance is 970mm, the zoom of regulation camera
Camera lens makes focal length be 25mm, camera CCD a size of 1/3 inch so that the image resolution ratio collected
For 0.146mm/pixel.
Embodiments of the invention use snake fruit as experimental subjects, particularly as follows:
Step (1), left surface image that original image acquires and right hand view picture are as shown in Figure 2.
Step (2), spot extracts.
2.1) extreme point detection.
Generate the Gaussian image of 4 layers.First left surface image and right hand view picture are pressed formula (1) calculate, this
Place takes σ=0.5, obtains the left figure of initial Gaussian, the right figure of initial Gaussian, the initial Gaussian that will be obtained by up-sampling
The right figure of Zuo Tu, initial Gaussian is put and is twice, and obtains the left figure of ground floor Gauss and the right figure of ground floor Gauss；Again will
The left figure of ground floor Gauss and the right figure of ground floor Gauss are pressed formula (1) and are calculated, and now take σ=1, obtain the second floor height
This left figure and the right figure of second layer Gauss.The left figure of ground floor Gauss obtained and the left figure of second layer Gauss are respectively such as Fig. 3
Left and right two figure shown in.The left figure of difference of Gaussian finally obtained is as shown in Figure 4.
Then extreme point is marked；Figure left to difference of Gaussian and the right figure of difference of Gaussian travel through, and respectively obtain a left side
Figure difference of Gaussian extreme value point set and right figure difference of Gaussian extreme value point set, left figure difference of Gaussian extreme value point set coordinate is such as
Shown in Fig. 5.
2.2) left surface image and right hand view picture are carried out Harris Corner Detection respectively, respectively obtain left figure
Harris angle point collection and right figure Harris angle point collection, left figure Harris angle point collection coordinate is as shown in Figure 6.
2.3) as it is shown in fig. 7, left surface image and right hand view picture are carried out respectively Canny rim detection and
Related operation, respectively obtains left map contour center point set and right map contour center point set, left map contour central point
Collection coordinate is as shown in Figure 8.
Then carry out point set merging, obtain left figure dot set and right figure dot set.
Step (3), spot mates, as it is shown in figure 9, the left figure dot set that step (2) is obtained and right figure spot
Concentrate each point carry out matching judgment, wherein deterministic process realize effect as shown in Figure 10.
Step (4), Mismatching point is rejected.The match point to be selected obtaining step (3) is concentrated each to coupling to be selected
Point screens respectively:
4.1) marginal point rejecting.As shown in figure 11, remember that match point to be selected concentrates in left figure and right figure to be selected
Match point is respectively P and P ', and centered by P and P ', chooses the rectangular area of 20 × 20 respectively, investigates
Whether there is red, green, blue three-component value in two rectangular areas and be all higher than the point of 200.In the left figure of Figure 11,
It is positioned at the match point to be selected in the lower left corner near fruit edge, there is white background in its rectangular area, this white
The red, green, blue three-component value of background is all higher than 200, and therefore giving up this to match point is correct match point；
4.2) carry out first round vector determination and second successively and take turns vector determination.
Step (5), travels through coupling point set to be selected and obtains correct coupling point set.
In said process match point deterministic process to be selected realize effect as shown in figure 12, the inventive method is final
The match point schematic diagram obtained is as shown in figure 13.
By experimental verification, result shows, rate that the match is successful reaches 100%, and Mismatching point rate is 4.4%.Right
Being the fruit image of 0.15mm/pixel in image resolution ratio, whole coupling flow process total used time is 0.53 second.
Thus the present invention is proposed to be extracted and carry out spot vector method judgement by spot and carries out fruit image
Join so that fruit image coupling obtains excellent stability, accuracy and real-time.And by test
Demonstrate the reliability of match point computational methods of the present invention.
Above-mentioned detailed description of the invention is used for illustrating the present invention rather than limiting the invention, at this
In the spirit of invention and scope of the claims, any modifications and changes that the present invention is made, all fall
Enter protection scope of the present invention.
Claims (5)
1. one kind is extracted and the fruit image matching process of neighbor point vector method based on spot, it is characterised in that bag
Include following steps:
1) fruit side image is obtained:
Fruit is placed on fruit tray so that the calyx carpopodium line of fruit is substantially vertical with horizontal plane,
Gather a width left surface image from fruit side, then with the calyx carpopodium line of fruit as axle, rotate fruit 60 °
After gather a width right hand view picture again；
2) carry out spot extraction, obtain left figure dot set and right figure dot set；
Described step 2) in carry out spot and extract and include step in detail below:
2.1) extreme point detection:
First left surface image and right hand view picture (1) as follows are calculated, obtain initial Gaussian left
Figure, the right figure of initial Gaussian:
L (x, y, σ)=G (x, y, σ) * I (x, y) (1)
Wherein, L (x, y, σ) is calculated initial Gaussian image, I (x, y) is side image to be calculated,
The expression formula of G (x, y, σ) is below equation (2):
Wherein, σ yardstick coordinate, x, y are respectively the transverse and longitudinal coordinate of left surface image or right hand view picture；
Then left for initial Gaussian figure, the right figure of initial Gaussian are put and be twice, obtain the left figure of ground floor Gauss and first
This right figure of floor height；Again left for ground floor Gauss figure and the right figure of ground floor Gauss are calculated by above-mentioned formula (1) again,
To the left figure of second layer Gauss and the right figure of second layer Gauss；
Finally left for second layer Gauss figure is deducted the image that the left figure of ground floor Gauss obtains and obtains the left figure of difference of Gaussian,
Right for second layer Gauss figure is deducted the image that the right figure of ground floor Gauss obtains and obtains the right figure of difference of Gaussian；
Each pixel p in figure left for difference of Gaussian and the right figure of difference of Gaussian, if the gray scale of pixel p
Value is respectively less than or is all higher than the gray value of remaining each pixel in 3 × 3 neighborhoods centered by pixel p,
Then pixel p is labeled as extreme point；
Each pixel of figure left to difference of Gaussian and the right figure of difference of Gaussian travels through, and respectively obtains left figure
Difference of Gaussian extreme value point set and right figure difference of Gaussian extreme value point set；
2.2) Harris Corner Detection:
To step 1) the left surface image that obtains and right hand view picture carry out Harris Corner Detection, respectively respectively
Obtain left figure Harris angle point collection and right figure Harris angle point collection；
2.3) Canny rim detection:
By step 1) the left surface image that obtains and right hand view picture carry out Canny rim detection respectively, obtain
Left figure edge image and right figure edge image, carry out a form to left figure edge image and right figure edge image
Learn dilation operation, then the image obtained after morphological dilations computing made contours extract and fills all profiles,
Reject wherein area more than the morphological erosion computing that tries again after the profile of 100 pixels, the most right
The image obtained after morphological erosion computing is made contours extract and is exported by all profile center point coordinates, respectively
Obtain left map contour center point set and right map contour center point set；
Then left figure difference of Gaussian extreme value point set, left figure Harris angle point collection and left map contour center point set are entered
Row merges, the point repeated for coordinate, retains one of them and rejects the point that remaining coordinate repeats, obtaining a left side
Figure dot set；By right figure difference of Gaussian extreme value point set, right figure Harris angle point collection and right map contour center point set
Merge, the point that coordinate is repeated, retain one of them and reject the point that remaining coordinate repeats, obtaining
Right figure dot set；
3) to step 2) each point of the left and right figure dot set that obtains carries out matching judgment, obtains to be selected
Coupling point set；
4) to step 3) match point to be selected that obtains concentrates and each match point to be selected carried out Mismatching point rejecting,
Screening obtains correct match point；
5) coupling of fruit image is completed.
A kind of fruit image based on spot extraction with neighbor point vector method the most according to claim 1
Method of completing the square, it is characterised in that: described step 3) in left figure dot set each point with right figure dot set
Each point carry out matching judgment and include step in detail below:
3.1) in left figure dot set treat reconnaissance A centered by, select 70 × 70 rectangular area, search
Point closest with treating reconnaissance A in being in this rectangular area in left figure dot set, is designated as a B, and calculates
Treat the reconnaissance A vector to some B, be designated as
3.2) search in right figure dot set and within 6 pixels, treat reconnaissance A ' with treating reconnaissance A Diff N,
Centered by treating reconnaissance A ', select the rectangular area of 70 × 70, search in right figure dot set and be in this rectangle region
With some B Diff N point within 6 pixels and the point nearest with treating reconnaissance A ' in territory, it is designated as a little
B ', calculates the vector treating reconnaissance A ' to some B 'If can not find this B ', giving up and treating that reconnaissance A is with to be selected
Point A ' is as a pair match point, and terminates to treat reconnaissance A and the remaining steps treating reconnaissance A ' matching judgment；
3.3) search in left figure dot set and be in 70 × 70 rectangular areas centered by treating reconnaissance A and treat
The point that reconnaissance A distance time is near, is designated as a C, calculates the vector treating reconnaissance A to some CIf can not find this
Point C then gives up and treats reconnaissance A and treat reconnaissance A ' as a pair match point, and terminates treat reconnaissance A and treat reconnaissance
The remaining steps of A ' matching judgment；
3.4) search in right figure dot set and be in 70 × 70 rectangular areas centered by treating reconnaissance A ' and put C
Diff N point within 6 pixels and the point nearest with treating reconnaissance A ', be designated as a C ', and calculate and treat
Reconnaissance A ' is to the vector of a C 'If can not find this C ', giving up and treating reconnaissance A and treat reconnaissance A ' conduct
A pair match point, and terminate to treat reconnaissance A and the remaining steps treating reconnaissance A ' matching judgment；
3.5) formula (3) is used to calculate vector respectivelyWith vectorAngle α and vectorWith vector
Angle β, use formula (4) calculate vectorWith vectorDifference dis of modulus value1And vectorWith vectorDifference dis of modulus value2:
Wherein, vectorA, b are respectively vectorTransverse and longitudinal coordinate, vector
C, d are respectively vectorTransverse and longitudinal coordinate；
If it is vectorialWith vectorAngle α < 15 ° andWith vectorDifference dis of modulus value1< 8 and to
AmountWith vectorAngle β < 18 ° and vectorWith vectorDifference dis of modulus value2< 10, then will treat
Reconnaissance A with treat that reconnaissance A ', as a pair match point to be selected, and terminates to treat in reconnaissance A and right figure dot set surplus
The matching judgment of remaining point, otherwise gives up and treats reconnaissance A and treat reconnaissance A ' as a pair match point to be selected.
A kind of fruit image based on spot extraction with neighbor point vector method the most according to claim 1
Method of completing the square, it is characterised in that: described step 4) in carry out Mismatching point and reject and specifically use following steps:
4.1) marginal point is rejected:
Match point to be selected concentration is a certain respectively to be treated in left surface image and right hand view picture match point to be selected
Select match point P and match point P ' to be selected, and centered by match point P to be selected and match point P ' to be selected, respectively
Choose the rectangular area of 20 × 20, if exist in any one rectangular area in two rectangular areas red, green,
Blue three-component value is all higher than the pixel of 200, then give up match point P to be selected and match point P ' to be selected as one
To correct match point；
4.2) first round vector determination is carried out:
4.2.1) if the above-mentioned logarithm of match point to be selected concentration match point that obtains is less than 3, step is the most directly carried out
4.3)；
4.2.2) in left surface image, centered by match point P to be selected, the rectangular area of 70 × 70 is selected,
Match point to be selected closest with some P in searching for this rectangular area of 70 × 70, is designated as match point Q to be selected,
Calculate the match point P to be selected vector to match point Q to be selected
If can not find this match point Q to be selected, giving up match point P to be selected and match point P ' to be selected and aligning as one
True match point, and terminate the remaining of match point P to be selected and match point P ' to be selected is judged step；
4.2.3) in right hand view picture corresponding for match point Q to be selected with left surface image, match point to be selected is Q ',
Calculate the vector of match point P ' to be selected to match point Q ' to be selected
4.2.4) in left surface image, in the search 70 × 70 rectangular areas centered by match point P to be selected and
The match point to be selected that match point P to be selected distance time is near, is designated as match point R to be selected, calculates match point P to be selected
Vector to match point R to be selected
If can not find this match point R to be selected, give up match point P to be selected and match point P ' to be selected as a pair
Join a little, and terminate the remaining of match point P to be selected and match point P ' to be selected is judged step；
4.2.5) in right hand view picture corresponding for match point R to be selected with left surface image, match point to be selected is R ',
Calculate the vector of match point P ' to be selected to match point R ' to be selected
4.2.6) formula (3) is used to calculate vector respectivelyWith vectorAngle ρ and vectorWith vectorAngle μ, use formula (4) calculate vectorWith vectorDifference dis of modulus value3And vectorWith
VectorDifference dis of modulus value4；
4.2.7) if meeting vectorWith vectorAngle ρ < 15 ° and vectorWith vectorModulus value
Difference dis3< 8, or meet vectorWith vectorAngle μ < 18 ° and vectorWith vectorMould
Difference dis of value4< 10, then proceed below step, otherwise give up match point P to be selected and match point P ' to be selected
As a pair correct match point；
4.3) second vector determination is taken turns；
To step 4.2.7) the match point P to be selected that obtains and match point P ' repeat the above steps 4.2.1 to be selected)
～4.2.6), again obtain vectorWith vectorAngle ρ, vectorWith vectorThe difference of modulus value
dis3, vectorWith vectorAngle μ, vectorWith vectorDifference dis of modulus value4If, full
Foot vectorWith vectorAngle ρ < 15 ° and vectorWith vectorDifference dis of modulus value3< 8 and to
AmountWith vectorAngle μ < 18 ° and vectorWith vectorDifference dis of modulus value4< 10, this is treated
Selecting match point P and match point P ' to be selected is correct match point, otherwise gives up match point P to be selected and to be selected
Join a P ' as a pair correct match point.
A kind of fruit image based on spot extraction with neighbor point vector method the most according to claim 1
Method of completing the square, it is characterised in that: described fruit is snake fruit.
A kind of fruit image based on spot extraction with neighbor point vector method the most according to claim 1
Method of completing the square, it is characterised in that: described left surface image and the resolution ratio of right hand view picture are 0.146
mm/pixel。
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410215892.7A CN104036492B (en) | 2014-05-21 | 2014-05-21 | A kind of fruit image matching process based on spot extraction with neighbor point vector method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410215892.7A CN104036492B (en) | 2014-05-21 | 2014-05-21 | A kind of fruit image matching process based on spot extraction with neighbor point vector method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104036492A CN104036492A (en) | 2014-09-10 |
| CN104036492B true CN104036492B (en) | 2016-08-31 |
Family
ID=51467251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410215892.7A Active CN104036492B (en) | 2014-05-21 | 2014-05-21 | A kind of fruit image matching process based on spot extraction with neighbor point vector method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104036492B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107247953B (en) * | 2017-05-31 | 2020-05-19 | 大连理工大学 | Feature point type selection method based on edge rate |
| CN113109240B (en) * | 2021-04-08 | 2022-09-09 | 国家粮食和物资储备局标准质量中心 | Method and system for determining imperfect grains of grains implemented by computer |
| CN113177925B (en) * | 2021-05-11 | 2022-11-11 | 昆明理工大学 | Method for nondestructive detection of fruit surface defects |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103177435A (en) * | 2013-04-10 | 2013-06-26 | 浙江大学 | Apple surface non-redundancy information image processing method based on machine vision |
| CN103336946A (en) * | 2013-06-17 | 2013-10-02 | 浙江大学 | Binocular stereoscopic vision based clustered tomato identification method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH09218024A (en) * | 1996-02-13 | 1997-08-19 | Daido Denki Kogyo Kk | Method for inspecting surface unevenness of vegetable and fruit |
-
2014
- 2014-05-21 CN CN201410215892.7A patent/CN104036492B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103177435A (en) * | 2013-04-10 | 2013-06-26 | 浙江大学 | Apple surface non-redundancy information image processing method based on machine vision |
| CN103336946A (en) * | 2013-06-17 | 2013-10-02 | 浙江大学 | Binocular stereoscopic vision based clustered tomato identification method |
Non-Patent Citations (2)
| Title |
|---|
| Image fusion of visible and thermal images for fruit detection;D.M.Bulanon et al;《Biosystems Engineering》;20090531;第103卷(第1期);12-22 * |
| 基于立体视觉的遮挡柑橘识别与空间匹配研究;李玉良;《中国优秀硕士学位论文全文数据库 工程科技Ⅰ辑》;20071105(第05期);B024-144 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104036492A (en) | 2014-09-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104318548B (en) | Rapid image registration implementation method based on space sparsity and SIFT feature extraction | |
| CN103679636B (en) | Based on point, the fast image splicing method of line double characteristic | |
| CN106446750B (en) | A kind of bar code read method and device | |
| CN105389554B (en) | Living body determination method and equipment based on recognition of face | |
| CN108982508A (en) | A kind of plastic-sealed body IC chip defect inspection method based on feature templates matching and deep learning | |
| CN105335725B (en) | A kind of Gait Recognition identity identifying method based on Fusion Features | |
| CN102542275B (en) | Automatic identification method for identification photo background and system thereof | |
| CN110765992B (en) | Seal identification method, medium, equipment and device | |
| Wang et al. | Edge extraction by merging 3D point cloud and 2D image data | |
| CN106504262A (en) | A kind of small tiles intelligent locating method of multiple features fusion | |
| CN104835175A (en) | Visual attention mechanism-based method for detecting target in nuclear environment | |
| CN104123529A (en) | Human hand detection method and system thereof | |
| CN107784667A (en) | Based on parallel global ocean mesoscale eddy Fast Recognition Algorithm | |
| CN107392929A (en) | A kind of intelligent target detection and dimension measurement method based on human vision model | |
| CN108154147A (en) | The region of interest area detecting method of view-based access control model attention model | |
| CN107292869A (en) | Image Speckle detection method based on anisotropic Gaussian core and gradient search | |
| CN109389165A (en) | Oil level gauge for transformer recognition methods based on crusing robot | |
| CN104408711A (en) | Multi-scale region fusion-based salient region detection method | |
| CN104036492B (en) | A kind of fruit image matching process based on spot extraction with neighbor point vector method | |
| CN107992783A (en) | Face image processing process and device | |
| CN111861866A (en) | Panoramic reconstruction method for substation equipment inspection image | |
| CN102074017A (en) | Method and device for detecting and tracking barbell central point | |
| CN109766850A (en) | Fingerprint image matching method based on Fusion Features | |
| CN115861274A (en) | Crack detection method integrating three-dimensional point cloud and two-dimensional image | |
| CN108733749A (en) | A kind of image search method based on sketch |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210521 Address after: 310012 room 1102, block B, Lishui digital building, 153 Lianchuang street, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province Patentee after: Hangzhou nuotian Intelligent Technology Co.,Ltd. Address before: 310058 Yuhang Tang Road, Xihu District, Hangzhou, Zhejiang 866 Patentee before: ZHEJIANG University |
|
| TR01 | Transfer of patent right |