CN104036492B - A kind of fruit image matching process based on spot extraction with neighbor point vector method - Google Patents

Abstract

The invention discloses a fruit image matching method based on spot extraction and adjacent point vector method. Obtain the left and right side images of the fruit: perform spot extraction, and use the extreme point detection method, Harris corner point detection method, and Canny edge detection method to obtain the left image spot set and the right image spot set; match each point Judgment to obtain the matching point set to be selected; remove the wrong matching points from the matching point set to be selected, and filter to obtain the correct matching point set; complete the matching of the fruit image. The present invention achieves excellent stability, accuracy and real-time performance in fruit image matching through spot extraction and vector method judgment on the spots.

Description

A Fruit Image Matching Method Based on Spot Extraction and Neighboring Point Vector Method

technical field

The invention relates to a fruit image matching method, in particular to a fruit image matching method based on spot extraction and adjacent point vector method in the technical field of image processing.

Background technique

Image matching refers to identifying the same point between two or more images through a certain matching method. In recent years, image matching has become a key technology and research hotspot in the fields of image analysis and processing such as object recognition, robot map perception and navigation, image stitching, 3D model building, gesture recognition, image tracking, and motion comparison.

The acquisition of fruit surface information is the basis for the detection of fruit quality indicators such as size, shape, surface color and surface defects. The accuracy of surface color and surface defect detection depends on the acquisition of complete fruit surface images. Image mosaic technology is the key to achieve complete fruit surface image acquisition, and image matching technology is the basis of image mosaic technology.

The SIFT (Scale Invariant Feature Transform) method is a local feature descriptor proposed by David Lowe in 1999 (David.G.Lowe.Object recognition from local scale-invariant features.International Conference on Computer Vision, Corfu,Greece,1999:1150-1157 ), and carried out further development and improvement in 2004 (David.G.Lowe.Distinctiveimage features from scale-invariant keypoints[J].International Journal of ComputerVision,2004,60(2):91-110). The extracted SIFT feature vector remains invariant to rotation, scaling, and brightness changes, and also maintains a certain degree of stability to viewing angle changes, affine transformations, and noise.

However, the online detection and grading of fruits has high requirements on the method. On the one hand, the SIFT method has caused the complexity of the method to improve the matching adaptability, so the calculation amount is large and time-consuming, and the original SIFT method cannot meet the online requirements; on the other hand , the commonly used false matching point elimination method RANSAC method (Chum O, Matas J.Optimized randomized RANSAC[J].IEEE Trans.On Pattern Analysis and Machine Intelligence,2008,30(8):1472-1482) The basic assumption is that in the sample Contains correct data (inliers, data that can be described by the model) and abnormal data (outliers, data that deviates far from the normal range and cannot adapt to the mathematical model), that is, the data set contains noise. These abnormal data may be caused by wrong measurements, wrong assumptions, wrong calculations, etc., and the small difference in fruit coding color features makes it impossible to obtain matching points stably for fruit image matching based on the SIFT method. The number of matching points obtained by one match is 0 to more, so this type of method cannot be used to eliminate false matching points of fruit images.

Contents of the invention

In order to solve the problems existing in the background technology, the present invention proposes a fruit image matching method based on spot extraction and adjacent point vector method.

The technical scheme that the present invention solves its technical problem adopts is to comprise the following steps:

1) Obtain the side image of the fruit:

Place the fruit on the fruit tray so that the calyx and fruit stem of the fruit are basically perpendicular to the horizontal plane, collect a left side image from the side of the fruit, and then use the fruit calyx and fruit stem as the axis, rotate the fruit 60° and then Acquire a right side image;

2) Perform speckle extraction to obtain the speckle set in the left image and the speckle set in the right image;

3) carry out matching judgment to each point of the left and right image spot sets that step 2) obtains, obtain the matching point set to be selected;

4) Carry out false matching point elimination for each matching point to be selected in the set of matching points to be selected obtained in step 3), and filter to obtain the correct matching point;

5) Complete the matching of fruit images.

Carrying out speckle extraction in described step 2) comprises the following concrete steps:

2.1) Extreme point detection:

First, the left side image and the right side image are calculated according to the following formula (1) to obtain the initial Gaussian left image and the initial Gaussian right image:

L(x,y,σ)=G(x,y,σ)*I(x,y) (1)

Among them, L(x, y, σ) is the calculated initial Gaussian image, I(x, y) is the side image to be calculated, and the expression of G(x, y, σ) is the following formula (2):

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&pi;\&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Among them, σ——scale coordinates, x and y are the horizontal and vertical coordinates of the left side image or the right side image respectively;

Then the initial Gaussian left image and the initial Gaussian right image are doubled to obtain the first layer of Gaussian left image and the first layer of Gaussian right image; ) to calculate again to obtain the second layer of Gaussian left image and the second layer of Gaussian right image;

Finally, the Gaussian difference left image is obtained by subtracting the Gaussian left image of the second layer from the Gaussian left image of the first layer, and the Gaussian difference right image is obtained by subtracting the Gaussian right image of the second layer from the Gaussian right image of the first layer;

For each pixel p in the left image of Gaussian difference and the right image of Gaussian difference, if the gray value of pixel p is smaller or larger than the gray value of other pixels in the 3×3 neighborhood centered on pixel p value, the pixel point p is marked as an extreme point;

Traverse each pixel in the left image of Gaussian difference and the right image of Gaussian difference to obtain the extreme point set of Gaussian difference in the left image and the extreme point set of Gaussian difference in the right image respectively;

2.2) Harris corner detection:

Carry out Harris corner point detection respectively to the left side image and the right side side image that step 1) obtains, obtain the left figure Harris corner point set and the right figure Harris corner point set respectively;

2.3) Canny edge detection:

The left side image and the right side image obtained in step 1) are respectively subjected to Canny edge detection to obtain the left edge image and the right edge image, and a morphological expansion operation is performed on the left edge image and the right edge image, and then Extract the contours of the image obtained after the morphological expansion operation and fill all the contours, remove the contours with an area larger than 100 pixels, and then perform a morphological erosion operation, and then perform contour extraction on the image obtained after the morphological erosion operation And output the coordinates of all the contour center points to obtain the left contour center point set and the right contour center point set respectively;

Then merge the Gaussian difference extreme point set in the left image, the Harris corner point set in the left image, and the contour center point set in the left image. For the points with repeated coordinates, keep one of them and remove the remaining points with repeated coordinates to obtain the spot set in the left image; Merge the Gaussian difference extreme point set on the right image, the Harris corner point set on the right image, and the contour center point set on the right image. For the points with repeated coordinates, keep one of them and remove the remaining points with repeated coordinates to obtain the spot set on the right image.

In the described step 3), matching and judging each point of the speckle set of the left figure and each point of the speckle set of the right figure comprises the following specific steps:

3.1) Take the point A to be selected in the spot set in the left image as the center, select a rectangular area of 70×70, search for the point in the spot set in the left image that is within the rectangular area and the point closest to the point A to be selected, record it as point B, and calculate The vector from point A to point B to be selected is denoted as

3.2) Search for the candidate point A' whose ordinate difference is within 6 pixels between the spot concentration on the right image and the point A to be selected. With the point A' as the center, select a rectangular area of 70×70 and search for the spot concentration on the right image. The point within the rectangular area that is within 6 pixels of the ordinate difference from point B and the point closest to point A' to be selected is denoted as point B', and the vector from point A' to point B' to be selected is calculated If the point B' cannot be found, discard the point A to be selected and the point A' to be selected as a pair of matching points, and end the remaining steps of judging the matching between the point A to be selected and the point A' to be selected;

3.3) Search for the spot on the left image that is located in the 70×70 rectangular area centered on the point A to be selected and the second closest point to the point A to be selected, denoted as point C, and calculate the vector from point A to point C to be selected If the point C cannot be found, the point A to be selected and the point A' to be selected are discarded as a pair of matching points, and the remaining steps of matching judgment between the point A to be selected and the point A' to be selected are terminated;

3.4) Search for the points on the right image that are located in the 70×70 rectangular area centered on the point A' to be selected, and the points that are within 6 pixels of the vertical coordinate difference of point C and are closest to the point A' to be selected, denoted as Point C', and calculate the vector from point A' to point C' to be selected If the point C' cannot be found, discard the point A to be selected and the point A' to be selected as a pair of matching points, and end the remaining steps of judging the matching between the point A to be selected and the point A' to be selected;

3.5) Use formula (3) to calculate vectors respectively with vector The included angle α and the vector with vector The included angle β of , using the formula (4) to calculate the vector with vector The difference between the modulus values of dis ₁ and the vector with vector The modulus difference dis ₂ :

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{{\mathrm{<span\; class="notranslate">\; a</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

Among them, the vector a, b are vectors respectively The horizontal and vertical coordinates of the vector c, d are vectors respectively the horizontal and vertical coordinates;

if vector with vector The included angle α<15° and with vector The difference between the modulus dis ₁ <8 and the vector with vector The included angle β<18° and the vector with vector dis ₂ <10, then the candidate point A and the candidate point A' will be regarded as a pair of candidate matching points, and the matching judgment between the candidate point A and the remaining points in the spot set on the right will be ended, otherwise the candidate will be discarded Point A and candidate point A' are used as a pair of candidate matching points.

In the described step 4), the following steps are specifically adopted for removing mismatching points:

4.1) Edge point removal:

A matching point to be selected in the set of matching points to be selected is a matching point to be selected P and a matching point to be selected in the left side image and the right side image respectively, and the matching point P to be selected and the matching point P to be selected ' as the center, respectively select a 20×20 rectangular area, if any of the two rectangular areas has red, green, and blue three-component values greater than 200 pixels, discard the matching point P and the candidate matching point Match point P' as a pair of correct matching points;

4.2) Perform the first round of vector judgment:

4.2.1) If the logarithm of the matching points in the set of matching points to be selected obtained above is less than 3, then directly proceed to step 4.3);

4.2.2) In the image on the left side, with the matching point P as the center, select a 70×70 rectangular area, search for the matching point within the rectangular area that is closest to the point P, and record it as the matching point Q , calculate the vector from the candidate matching point P to the candidate matching point Q

If the candidate matching point Q cannot be found, discard the candidate matching point P and the candidate matching point P' as a pair of correct matching points, and end the remaining judgment steps of the candidate matching point P and the candidate matching point P' ;

4.2.3) The matching point to be selected in the right side image corresponding to the matching point Q to be selected in the left side image is Q', calculate the vector from the matching point P' to the matching point Q' to be selected

4.2.4) In the left side image, search for the candidate matching point with the next closest distance to the candidate matching point P in the 70×70 rectangular area centered on the candidate matching point P, and record it as the candidate matching point R, and calculate The vector from the candidate matching point P to the candidate matching point R

If can not find this matching point R to be selected then discard matching point P to be selected and matching point P' to be selected as a pair of matching points, and end the remaining judgment steps of matching point P to be selected and matching point P' to be selected;

4.2.5) The matching point to be selected in the right side image corresponding to the matching point R to be selected in the left side image is R', and the vector from the matching point P' to the matching point R' to be selected is calculated

4.2.6) Use the formula (3) to calculate the vector with vector The included angle ρ and the vector with vector The included angle μ, using the formula (4) to calculate the vector with vector The difference between the modulus values of dis ₃ and the vector with vector The difference between the modulus value dis ₄ ;

4.2.7) If satisfy the vector with vector The included angle ρ<15° and the vector with vector The difference between the modulus value dis ₃ <8, or satisfy the vector with vector The included angle μ<18° and the vector with vector dis ₄ <10, then proceed to the following steps, otherwise discard the candidate matching point P and the candidate matching point P' as a pair of correct matching points;

4.3) The second round of vector judgment;

Repeat the above steps 4.2.1) to 4.2.6) for the candidate matching point P and candidate matching point P' obtained in step 4.2.7), and obtain the vector again with vector The included angle ρ, vector with vector The modulus difference dis ₃ , the vector with vector The included angle μ, vector with vector The difference between the modulus value dis ₄ , if the vector with vector The included angle ρ<15° and the vector with vector The difference between the modulus dis ₃ <8 and the vector with vector The included angle μ<18° and the vector with vector If the difference between the modulus dis ₄ <10, the candidate matching point P and the candidate matching point P' are the correct matching points, otherwise the candidate matching point P and the candidate matching point P' are discarded as a pair of correct matching points.

Described fruit is snake fruit.

The resolutions of the left side image and the right side image are both 0.146mm/pixel.

The benefits of the present invention are:

The invention achieves excellent stability, accuracy and real-time performance in fruit image matching through spot extraction and vector method judgment on the spots.

Description of drawings

Fig. 1 is a main flowchart of the matching method of the present invention.

Fig. 2 is two pieces of fruit original images of the embodiment of the present invention.

Fig. 3 is a first-layer Gaussian left image and a second-layer Gaussian left image of an embodiment of the present invention.

Fig. 4 is the left diagram of Gaussian difference according to the embodiment of the present invention.

Fig. 5 is a speckle coordinate diagram obtained based on the detection of extreme points of a Gaussian difference image according to an embodiment of the present invention.

FIG. 6 is a spot coordinate diagram obtained based on Harris corner detection according to an embodiment of the present invention.

FIG. 7 is a step of acquiring spot coordinates based on Canny edge detection according to an embodiment of the present invention.

FIG. 8 is a spot coordinate diagram obtained based on Canny edge detection according to an embodiment of the present invention.

Fig. 9 is a schematic diagram of selection of the closest point and the next closest point in the blob matching process according to the embodiment of the present invention.

Fig. 10 is an implementation diagram of the speckle matching judging process according to the embodiment of the present invention.

Fig. 11 is a schematic diagram of edge point removal according to an embodiment of the present invention.

Fig. 12 is an implementation diagram of the elimination process of mismatching points according to the embodiment of the present invention.

Fig. 13 is a matching point map obtained by the embodiment of the present invention.

detailed description

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

As shown in Figure 1, the inventive method comprises the steps:

1) Obtain the side image of the fruit:

Place the fruit on the fruit tray so that the calyx and fruit stem of the fruit are basically perpendicular to the horizontal plane, collect a left side image from the side of the fruit, and then use the fruit calyx and fruit stem as the axis, rotate the fruit 60° and then Acquire a right profile image.

2) Carry out speckle extraction, speckle extraction comprises following three methods, obtains the speckle set of left figure and the speckle set of right figure:

2.1) Extreme point detection:

First, the left side image and the right side image are calculated according to the following formula (1) to obtain the initial Gaussian left image and the initial Gaussian right image:

L(x,y,σ)=G(x,y,σ)*I(x,y) (1)

Among them, L(x, y, σ) is the calculated initial Gaussian image, I(x, y) is the side image to be calculated, and the expression of G(x, y, σ) is the following formula (2):

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&pi;\&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Among them, σ——scale coordinates, x and y are the horizontal and vertical coordinates of the left side image or the right side image respectively;

Then the initial Gaussian left image and the initial Gaussian right image are doubled to obtain the first layer of Gaussian left image and the first layer of Gaussian right image; ) to calculate again to obtain the second layer of Gaussian left image and the second layer of Gaussian right image;

Finally, the Gaussian difference left image is obtained by subtracting the Gaussian left image of the second layer from the Gaussian left image of the first layer, and the Gaussian difference right image is obtained by subtracting the Gaussian right image of the second layer from the Gaussian right image of the first layer;

For each pixel p in the left image of Gaussian difference and the right image of Gaussian difference, if the gray value of pixel p is smaller or larger than the gray value of other pixels in the 3×3 neighborhood centered on pixel p value, the pixel point p is marked as an extreme point;

Traverse each pixel in the left image of Gaussian difference and the right image of Gaussian difference to obtain the extreme point set of Gaussian difference in the left image and the extreme point set of Gaussian difference in the right image respectively.

2.2) Harris corner detection:

Harris corner detection (ChrisHarris, Mike Stephens, A Combined Corner and Edge Detector, 4th Alvey Vision Conference, 1988, pp147-151) was performed on the left side image and the right side image obtained in step 1), respectively, and the left image Harris The corner set and the Harris corner set on the right.

2.3) Canny edge detection:

The left side image and the right side image obtained in step 1) are respectively subjected to Canny edge detection (Canny, J., A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.) , to obtain the left edge image and the right edge image, and then perform a morphological expansion operation on the left edge image and the right edge image (Rafael C.Gonzalez, Richard E. Woods, Digital Image Processing, Third Edition, 2010, pp402 -442), and then extract the outline of the image obtained after the morphological expansion operation and fill all the outlines, remove the outline with an area greater than 100 pixels, and then perform a morphological erosion operation (Rafael C.Gonzalez, Richard E.Woods ,Digital Image Processing,Third Edition,2010,pp402-442), and then extract the outline of the image obtained after the morphological erosion operation and output the coordinates of all the outline center points, and obtain the left image outline center point set and the right image outline respectively center point set;

Then merge the Gaussian difference extreme point set in the left image, the Harris corner point set in the left image, and the contour center point set in the left image. For the points with repeated coordinates, keep one of them and remove the remaining points with repeated coordinates to obtain the spot set in the left image; Merge the Gaussian difference extreme point set on the right image, the Harris corner point set on the right image, and the contour center point set on the right image. For the points with repeated coordinates, keep one of them and remove the remaining points with repeated coordinates to obtain the spot set on the right image.

3) Carry out speckle matching, step 2) each point of the speckle set in the left figure that obtains and each point of the speckle set in the right figure carry out matching judgment and include the following specific steps:

3.1) Take the point A to be selected in the spot set in the left image as the center, select a rectangular area of 70×70, search for the point in the spot set in the left image that is within the rectangular area and the point closest to the point A to be selected, record it as point B, and calculate The vector from point A to point B to be selected is denoted as

3.2) Search for the candidate point A' whose ordinate difference is within 6 pixels between the spot concentration on the right image and the point A to be selected. With the point A' as the center, select a rectangular area of 70×70 and search for the spot concentration on the right image. The point within the rectangular area that is within 6 pixels of the ordinate difference from point B and the point closest to point A' to be selected is denoted as point B', and the vector from point A' to point B' to be selected is calculated

If the point B' cannot be found, the point to be selected A and the point to be selected A' are discarded as a pair of matching points, and the remaining steps of matching judgment to the point to be selected A and the point to be selected to be selected are completed;

3.3) Search for the spot on the left image that is located in the 70×70 rectangular area centered on the point A to be selected and the second closest point to the point A to be selected, denoted as point C, and calculate the vector from point A to point C to be selected If the point C cannot be found, the candidate point A and the candidate point A' are discarded as a pair of matching points, and the remaining steps of judging the matching of the point A and the point A' are ended.

3.4) Search for the points on the right image that are located in the 70×70 rectangular area centered on the point A' to be selected, and the points that are within 6 pixels of the vertical coordinate difference of point C and are closest to the point A' to be selected, denoted as Point C', and calculate the vector from point A' to point C' to be selected If the point C' cannot be found, discard the candidate point A and the candidate point A' as a pair of matching points, and end the remaining steps of matching judgment between the candidate point A and the candidate point A'.

3.5) Use formula (3) to calculate vectors respectively with vector The included angle α and the vector with vector The included angle β of , using the formula (4) to calculate the vector with vector The difference between the modulus values of dis ₁ and the vector with vector The modulus difference dis ₂ :

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{{\mathrm{<span\; class="notranslate">\; a</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

where the vector a, b are vectors respectively The horizontal and vertical coordinates of the vector c, d are vectors respectively The horizontal and vertical coordinates of ; when substituting, the vector vector are vectors and vector

if vector with vector The included angle α<15° and with vector The difference between the modulus dis ₁ <8 and the vector with vector The included angle β<18° and the vector with vector dis ₂ <10, then the candidate point A and the candidate point A' will be regarded as a pair of candidate matching points, and the matching judgment between the candidate point A and the remaining points in the spot set on the right will be ended, otherwise the candidate will be discarded Point A and candidate point A' are used as a pair of candidate matching points.

4) false matching points are eliminated, and each matching point to be selected in the set of matching points to be selected obtained in step 3) is screened respectively according to the following steps to obtain the correct matching point:

4.1) Edge point removal:

A matching point to be selected in the set of matching points to be selected is a matching point to be selected P and a matching point to be selected in the left side image and the right side image respectively, and the matching point P to be selected and the matching point P to be selected ' as the center, respectively select a 20×20 rectangular area, if any of the two rectangular areas has red, green, and blue three-component values greater than 200 pixels, discard the matching point P and the candidate matching point The matching point P' is regarded as a pair of correct matching points.

4.2) Perform the first round of vector judgment:

4.2.1) If the logarithm of the matching points in the set of matching points to be selected obtained above is less than 3, then directly proceed to step 4.3);

4.2.2) In the left side image, take the matching point P to be selected as the center and select a 70×70 rectangular area;

Search for the candidate matching point closest to the point P in the rectangular area, record it as the candidate matching point Q, and calculate the vector from the candidate matching point P to the candidate matching point Q

If the candidate matching point Q cannot be found, discard the candidate matching point P and the candidate matching point P' as a pair of correct matching points, and end the remaining judgment steps of the candidate matching point P and the candidate matching point P' ;

4.2.3) The matching point to be selected in the right side image corresponding to the matching point Q to be selected in the left side image is Q', calculate the vector from the matching point P' to the matching point Q' to be selected

4.2.4) In the left side image, search for the candidate matching point with the next closest distance to the candidate matching point P in the 70×70 rectangular area centered on the candidate matching point P, and record it as the candidate matching point R, and calculate The vector from the candidate matching point P to the candidate matching point R

If the candidate matching point R cannot be found, discard the candidate matching point P and the candidate matching point P' as a pair of correct matching points, and end the remaining judgment steps of the candidate matching point P and the candidate matching point P' ;

4.2.5) The matching point to be selected in the right side image corresponding to the matching point R to be selected in the left side image is R', and the vector from the matching point P' to the matching point R' to be selected is calculated

4.2.6) Use the formula (3) to calculate the vector with vector The included angle ρ and the vector with vector The included angle μ, use the formula (4) to calculate the vector with vector The difference between the modulus values of dis ₃ and the vector with vector The difference between the modulus value dis ₄ ;

When substituting, the vectors in formula (3) and formula (4) vector are vectors and vector

4.2.7) If satisfy the vector with vector The included angle ρ<15° and the vector with vector The difference between the modulus value dis ₃ <8, or satisfy the vector with vector The included angle μ<18° and the vector with vector dis ₄ <10, then proceed to the following steps, otherwise discard the candidate matching point P and the candidate matching point P' as a pair of correct matching points;

4.3) The second round of vector judgment;

Repeat the above steps 4.2.1) to 4.2.6) for the candidate matching point P and candidate matching point P' obtained in step 4.2.7), and obtain the vector again with vector The included angle ρ, vector with vector The modulus difference dis ₃ , the vector with vector The included angle μ, vector with vector The difference between the modulus value dis ₄ , if the vector with vector The included angle ρ<15° and the vector with vector The difference between the modulus dis ₃ <8 and the vector with vector The included angle μ<18° and the vector with vector If the difference between the modulus values of dis ₄ <10, the matching point P to be selected and the matching point P' to be selected are correct matching points, otherwise they are discarded.

5) Complete the matching of fruit images.

Described fruit is snake fruit.

The resolutions of the left side image and the right side image are both 0.146mm/pixel.

When the left side image and the right side image are acquired, the adjustable object distance is 970mm, the zoom lens of the camera is adjusted so that the focal length is 25mm, and the CCD size of the camera is 1/3 inch, so that the captured image resolution is 0.146mm/pixel .

Embodiments of the present invention adopt red snake fruit as experimental object, specifically:

Step (1), the left side image and the right side image obtained from the original image are shown in Fig. 2 .

Step (2), spot extraction.

2.1) Extreme point detection.

Generate a Gaussian image with 4 layers. Firstly, calculate the left side image and the right side image according to formula (1), where σ=0.5 to get the initial Gaussian left image and the initial Gaussian right image, and the initial Gaussian left image and the initial Gaussian right image obtained by upsampling The image is doubled to obtain the first-layer Gaussian left image and the first-layer Gaussian right image; then the first-layer Gaussian left image and the first-layer Gaussian right image are calculated according to formula (1), and at this time σ=1 is obtained, The second layer Gaussian left image and the second layer Gaussian right image. The obtained first-layer Gaussian left image and the second-layer Gaussian left image are shown in the left and right images of Figure 3, respectively. The left image of the final Gaussian difference is shown in Figure 4.

Then mark the extreme points; traverse the left image of Gaussian difference and the right image of Gaussian difference to obtain the extreme point set of Gaussian difference in the left image and the extreme point set of Gaussian difference in the right image respectively. The coordinates of the extreme point set of Gaussian difference in the left image are shown in the figure 5.

2.2) Perform Harris corner detection on the left side image and the right side image respectively, and obtain the Harris corner set of the left picture and the Harris corner set of the right picture respectively, and the coordinates of the Harris corner set of the left picture are shown in Fig. 6 .

2.3) As shown in Figure 7, the left side image and the right side image are respectively subjected to Canny edge detection and related operations, and the left image contour center point set and the right image contour center point set are respectively obtained, and the left image contour center point set is The coordinates are shown in Figure 8.

Then the point sets are merged to obtain the left image spot set and the right image spot set.

Step (3), blob matching, as shown in Figure 9, performs matching judgment on each point in the blob set in the left image and the blob set in the right image obtained in step (2), and the effect of the judgment process is shown in Figure 10.

Step (4), elimination of mismatching points. Each matching point to be selected in the set of matching points to be selected obtained in step (3) is screened separately:

4.1) Edge point removal. As shown in Figure 11, remember that the matching points to be selected are concentrated in the left picture and the right picture. The matching points to be selected are P and P' respectively, and take P and P' as the center, respectively select a 20×20 rectangular area, and investigate Whether there are points with red, green, and blue three-component values greater than 200 in the two rectangular areas. In the left picture of Figure 11, the matching point to be selected in the lower left corner is close to the edge of the fruit, and there is a white background in the rectangular area. The red, green, and blue three-component values of the white background are all greater than 200, so discarding the pair of matching points is the correct matching point;

4.2) Carry out the first round of vector judgment and the second round of vector judgment in sequence.

Step (5), traversing the matching point set to be selected to obtain the correct matching point set.

The realization effect of the matching point judgment process in the above process is shown in FIG. 12 , and the matching point schematic diagram finally obtained by the method of the present invention is shown in FIG. 13 .

Through experimental verification, the results show that the matching success rate reaches 100%, and the false matching point rate is 4.4%. For a fruit image with an image resolution of 0.15mm/pixel, the total time for the entire matching process is 0.53 seconds.

Therefore, the present invention proposes to perform fruit image matching by spot extraction and vector method judgment on the spots, so that the fruit image matching can obtain excellent stability, accuracy and real-time performance. And the reliability of the matching point calculation method of the present invention is verified through experiments.

The specific embodiments above are used to explain the present invention, rather than to limit the present invention. Within the spirit of the present invention and the protection scope of the claims, any modification and change made to the present invention will fall into the protection scope of the present invention.

Claims (5)

1. a fruit image matching method based on speckle extraction and adjacent point vector method, is characterized in that comprising the steps: 1) Obtain the side image of the fruit: Place the fruit on the fruit tray so that the calyx and fruit stem of the fruit are basically perpendicular to the horizontal plane, collect a left side image from the side of the fruit, and then use the fruit calyx and fruit stem as the axis, rotate the fruit 60° and then Acquire a right side image; 2) Perform speckle extraction to obtain the speckle set in the left image and the speckle set in the right image; Carrying out speckle extraction in described step 2) comprises the following concrete steps: 2.1) Extreme point detection: First, the left side image and the right side image are calculated according to the following formula (1) to obtain the initial Gaussian left image and the initial Gaussian right image: L(x,y,σ)=G(x,y,σ)*I(x,y) (1) Among them, L(x,y,σ) is the calculated initial Gaussian image, I(x,y) is the side image to be calculated, and the expression of G(x,y,σ) is the following formula (2): $\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&sigma;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&pi;\&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$ Among them, σ——scale coordinates, x and y are the horizontal and vertical coordinates of the left side image or the right side image respectively; Then the initial Gaussian left image and the initial Gaussian right image are doubled to obtain the first layer of Gaussian left image and the first layer of Gaussian right image; ) to calculate again to obtain the second layer of Gaussian left image and the second layer of Gaussian right image; Finally, the Gaussian difference left image is obtained by subtracting the Gaussian left image of the second layer from the Gaussian left image of the first layer, and the Gaussian difference right image is obtained by subtracting the Gaussian right image of the second layer from the Gaussian right image of the first layer; For each pixel p in the left image of Gaussian difference and the right image of Gaussian difference, if the gray value of pixel p is smaller or larger than the gray value of other pixels in the 3×3 neighborhood centered on pixel p value, the pixel point p is marked as an extreme point; Traverse each pixel in the left image of Gaussian difference and the right image of Gaussian difference to obtain the extreme point set of Gaussian difference in the left image and the extreme point set of Gaussian difference in the right image respectively; 2.2) Harris corner detection: Carry out Harris corner point detection respectively to the left side image and the right side side image that step 1) obtains, obtain the left figure Harris corner point set and the right figure Harris corner point set respectively; 2.3) Canny edge detection: The left side image and the right side image obtained in step 1) are respectively subjected to Canny edge detection to obtain the left edge image and the right edge image, and a morphological expansion operation is performed on the left edge image and the right edge image, and then Extract the contours of the image obtained after the morphological expansion operation and fill all the contours, remove the contours with an area larger than 100 pixels, and then perform a morphological erosion operation, and then perform contour extraction on the image obtained after the morphological erosion operation And output the coordinates of all contour center points to obtain the left contour center point set and the right contour center point set respectively; Then merge the Gaussian difference extreme point set in the left image, the Harris corner point set in the left image, and the contour center point set in the left image. For the points with repeated coordinates, keep one of them and remove the remaining points with repeated coordinates to obtain the spot set in the left image; Merge the Gaussian difference extreme point set in the right image, the Harris corner point set in the right image, and the contour center point set in the right image. For the points with repeated coordinates, keep one of them and remove the remaining points with repeated coordinates to obtain the spot set in the right image; 3) carry out matching judgment to each point of the left and right image spot sets that step 2) obtains, obtain the matching point set to be selected; 4) Carry out false matching point elimination for each matching point to be selected in the set of matching points to be selected obtained in step 3), and filter to obtain the correct matching point; 5) Complete the matching of fruit images. 2. a kind of fruit image matching method based on speckle extraction and adjacent point vector method according to claim 1, is characterized in that: in described step 3) to each point of left figure speckle set and right figure speckle set The matching judgment for each point of the algorithm includes the following specific steps: 3.1) Take the point A to be selected in the spot set in the left image as the center, select a rectangular area of 70×70, search for the point in the spot set in the left image that is within the rectangular area and the point closest to the point A to be selected, record it as point B, and calculate The vector from point A to point B to be selected is denoted as 3.2) Search for the candidate point A' whose ordinate difference is within 6 pixels between the spot concentration on the right image and the point A to be selected. With the point A' as the center, select a rectangular area of 70×70 and search for the spot concentration on the right image. The point within the rectangular area that is within 6 pixels of the ordinate difference from point B and the point closest to point A' to be selected is denoted as point B', and the vector from point A' to point B' to be selected is calculated If the point B' cannot be found, discard the point A to be selected and the point A' to be selected as a pair of matching points, and end the remaining steps of judging the matching between the point A to be selected and the point A' to be selected; 3.3) Search for the spot on the left image that is located in the 70×70 rectangular area centered on the point A to be selected and the second closest point to the point A to be selected, denoted as point C, and calculate the vector from point A to point C to be selected If the point C cannot be found, the point A to be selected and the point A' to be selected are discarded as a pair of matching points, and the remaining steps of matching judgment between the point A to be selected and the point A' to be selected are terminated; 3.4) Search for the points on the right image that are located in the 70×70 rectangular area centered on the point A' to be selected, and the points that are within 6 pixels of the vertical coordinate difference of point C and are closest to the point A' to be selected, denoted as Point C', and calculate the vector from point A' to point C' to be selected If the point C' cannot be found, discard the point A to be selected and the point A' to be selected as a pair of matching points, and end the remaining steps of judging the matching between the point A to be selected and the point A' to be selected; 3.5) Use formula (3) to calculate vectors respectively with vector The included angle α and the vector with vector The included angle β of , using the formula (4) to calculate the vector with vector The difference between the modulus values of dis ₁ and the vector with vector The modulus difference dis ₂ : $\mathrm{<span\; class="notranslate">\; \&theta;</span>}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ,</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; c</span>}\mathrm{<span\; class="notranslate">\; o</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; b</span>}\mathrm{<span\; class="notranslate">\; d</span>}}{\sqrt{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; a</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; c</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; d</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$ $\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; \&Right\; Arrow;</span>}{\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{{\mathrm{<span\; class="notranslate">\; a</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; b</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$ Among them, the vector a, b are vectors respectively The horizontal and vertical coordinates of the vector c, d are vectors respectively the horizontal and vertical coordinates; if vector with vector The included angle α<15° and with vector The difference between the modulus dis ₁ <8 and the vector with vector The included angle β<18° and the vector with vector dis ₂ <10, then the candidate point A and the candidate point A' will be regarded as a pair of candidate matching points, and the matching judgment between the candidate point A and the remaining points in the spot set on the right will be ended, otherwise the candidate will be discarded Point A and candidate point A' are used as a pair of candidate matching points. 3. a kind of fruit image matching method based on speckle extraction and adjacent point vector method according to claim 1, is characterized in that: in described step 4), carry out false matching point and reject and specifically adopt the following steps: 4.1) Edge point removal: A matching point to be selected in the set of matching points to be selected is a matching point to be selected P and a matching point to be selected in the left side image and the right side image respectively, and the matching point P to be selected and the matching point P to be selected ' as the center, respectively select a 20×20 rectangular area, if any of the two rectangular areas has red, green, and blue three-component values greater than 200 pixels, discard the matching point P and the candidate matching point Match point P' as a pair of correct matching points; 4.2) Perform the first round of vector judgment: 4.2.1) If the logarithm of the matching points in the set of matching points to be selected obtained above is less than 3, then directly proceed to step 4.3); 4.2.2) In the image on the left side, with the matching point P as the center, select a 70×70 rectangular area, search for the matching point closest to the point P in the 70×70 rectangular area, and record it as the matching point to be selected Select the matching point Q, and calculate the vector from the matching point P to the matching point Q to be selected If the candidate matching point Q cannot be found, discard the candidate matching point P and the candidate matching point P' as a pair of correct matching points, and end the remaining judgment steps of the candidate matching point P and the candidate matching point P' ; 4.2.3) The matching point to be selected in the right side image corresponding to the matching point Q to be selected in the left side image is Q', calculate the vector from the matching point P' to the matching point Q' to be selected 4.2.4) In the left side image, search for the candidate matching point with the next closest distance to the candidate matching point P in the 70×70 rectangular area centered on the candidate matching point P, and record it as the candidate matching point R, and calculate The vector from the candidate matching point P to the candidate matching point R If can not find this matching point R to be selected then discard matching point P to be selected and matching point P' to be selected as a pair of matching points, and end the remaining judgment steps of matching point P to be selected and matching point P' to be selected; 4.2.5) The matching point to be selected in the right side image corresponding to the matching point R to be selected in the left side image is R', and the vector from the matching point P' to the matching point R' to be selected is calculated 4.2.6) Use the formula (3) to calculate the vector with vector The included angle ρ and the vector with vector The included angle μ, using the formula (4) to calculate the vector with vector The difference between the modulus values of dis ₃ and the vector with vector The difference between the modulus value dis ₄ ; 4.2.7) If satisfy the vector with vector The included angle ρ<15° and the vector with vector The difference between the modulus value dis ₃ <8, or satisfy the vector with vector The included angle μ<18° and the vector with vector dis ₄ <10, then proceed to the following steps, otherwise discard the candidate matching point P and the candidate matching point P' as a pair of correct matching points; 4.3) The second round of vector judgment; Repeat the above steps 4.2.1) to 4.2.6) for the candidate matching point P and candidate matching point P' obtained in step 4.2.7), and obtain the vector again with vector The included angle ρ, vector with vector The modulus difference dis ₃ , the vector with vector The included angle μ, vector with vector The difference between the modulus value dis ₄ , if the vector with vector The included angle ρ<15° and the vector with vector The difference between the modulus dis ₃ <8 and the vector with vector The included angle μ<18° and the vector with vector If the difference between the modulus dis ₄ <10, the candidate matching point P and the candidate matching point P' are the correct matching points, otherwise the candidate matching point P and the candidate matching point P' are discarded as a pair of correct matching points. 4. A fruit image matching method based on spot extraction and neighboring point vector method according to claim 1, characterized in that: said fruit is red snake fruit. 5. a kind of fruit image matching method based on speckle extraction and adjacent point vector method according to claim 1, is characterized in that: the resolution of described left side image and right side image is 0.146mm/pixel .