KR102624308B1 - Apparatus and method for image matching based on matching point - Google Patents

Abstract

The present invention relates to a corresponding point-based image matching device and method, which selects a plurality of neighboring points for each of the plurality of reference points set for a plurality of images including common areas, respectively, and relates to a plurality of neighboring points. Calculate the distribution and generate a histogram for each reference point, and search for corresponding points in each image using the histogram for each reference point, so that the plurality of images can be precisely searched based on the searched corresponding points. It relates to a matching device and method.

Description

Matching point-based image matching device and method {APPARATUS AND METHOD FOR IMAGE MATCHING BASED ON MATCHING POINT}

The present invention relates to a corresponding point-based image matching device and method. More specifically, the present invention relates to a corresponding point-based image matching device and method. More specifically, the present invention relates to a plurality of neighboring points for each of a plurality of reference points set for a plurality of images including common areas, respectively, to select a plurality of neighboring points, respectively. A device for generating a histogram for each reference point by calculating a distribution for neighboring points, and searching for corresponding points in each image through the histogram for each reference point to precisely match the plurality of images; It's about how.

Image matching is a technology that integrates the different images by matching the same area when different images include the same area and represents the different images in one coordinate system. In other words, image registration is performed when multiple images of a specific object that mutually contain the same area are acquired through a camera, etc., and a single integrated image of the specific object is created through a registration process for the same area. As such, it is used in various fields such as map production and creation of panoramic images of specific objects.

The main method used for image registration is matching multiple images based on corresponding points, which are multiple specific reference points included in another image that correspond to multiple specific reference points set in multiple images mutually containing the same area. A representative example is the iterative closest point (ICP) method.

However, the conventional ICP method has the problem that the coordinate system and scale of each image must be the same, and information about the corresponding points must be provided in advance.

That is, the conventional ICP method has the limitation of having to acquire multiple images containing the same area through the same image acquisition device (e.g., camera or scanner, etc.), and the corresponding point must be directly given by the user to obtain the image. Matching can be performed stably.

Accordingly, in the present invention, in order to solve the limitations and problems of the conventional ICP method, a source image that serves as a standard for image matching is provided for a plurality of images that contain common areas and each set a plurality of reference points, and the source image According to the distribution of reference points set in at least one target image to be matched, a histogram representing the distribution of each reference point and a plurality of neighboring points neighboring each reference point is generated, and the generated histogram is used to set the source image. We would like to propose a method for effectively matching each image regardless of scale or coordinate system by searching for corresponding points that are reference points set in the at least one target image that correspond to a plurality of reference points.

Next, we will briefly describe the prior art existing in the technical field of the present invention, and then describe the technical details that the present invention seeks to achieve differently compared to the prior art.

Korean Patent No. 0591608 (June 13, 2006) is a prior invention regarding a method of searching for corresponding points when registering images, extracting feature points from a reference image and a model image using a corner detector, and matching the feature points to the reference image from which the feature points were extracted. After constructing a Voronoi plane with distance information between the feature points, the model area with the largest variance value of the feature points within a certain region for the model image from which the feature points are extracted is selected as the matching region, and then the boro that matches the feature points within the region is selected. Candidate matching regions whose distance on the Noi plane is close to the minimum are extracted from the reference image, color matching degrees of the extracted candidate matching regions are calculated, and the candidate matching region with the minimum error from the model region is selected as the final target matching region. This relates to a method of searching for corresponding points when matching images.

The prior invention matches the reference image and the model image using the Voronoi plane, which includes distance information between feature points of the reference image, and the dispersion value of the feature points of the model image, and includes the same area proposed in the present invention. For a plurality of reference points set for each of a plurality of images, the distribution of each reference point and neighboring points adjacent to each reference point is not calculated to generate a histogram for each reference point, and the histogram is not generated based on the generated histogram. A method of searching for corresponding points for each image and matching the plurality of images based on the searched corresponding points is also not described at all. Therefore, there are significant differences in technical features between the prior invention and the present invention.

In addition, Korean Patent No. 2196479 (December 23, 2020) is a prior invention regarding an image registration method and system using weighted feature points, and is a feature point for the first matching area of the first image and the second matching area of the second image. Extract each to specify the entire feature point list, select reference feature points to be used for generating conversion information for matching the first image and the second image from the specific feature point list, and then match based on the selected reference feature points. It relates to an image matching method and system using weighted feature points that generate conversion information for and match the first image and the second image based on the generated conversion information.

That is, the prior invention simply describes matching the first image and the second image using weighted feature points set for the first image and the second image.

On the other hand, the present invention calculates the distribution of a plurality of neighboring points adjacent to the reference point for each of a plurality of reference points set for the source image that is the standard for matching and the target image to be matched to the source image, and assigns the distribution to each reference point. generating a histogram for each reference point, searching for corresponding points in the source image and the target image using the histogram for each reference point, and matching the source image and the target image based on the searched corresponding points, The above prior inventions do not describe, suggest, or have any indication of the technical features of the present invention.

The present invention was created to solve the above problems, and searches for a plurality of corresponding points in a source image that includes a common area and serves as a standard for matching and a target image to be matched to the source image. The purpose is to provide a corresponding point-based image matching device and method that enables precise matching of the plurality of images based on corresponding points.

In addition, the present invention calculates the distribution of a plurality of neighboring points centered on each of the plurality of reference points set in each image, generates a histogram for each reference point, and creates a histogram for each reference point. Another purpose is to provide a device and method for effectively searching for corresponding points for matching.

In addition, the present invention generates vectors to all the selected neighboring points around each reference point, and determines the range of the length ratio and angle calculated for all pairs of vectors that can be selected from the generated vectors. Another purpose is to provide an apparatus and method for generating a histogram for each reference point by calculating the coefficients of the bins of the histogram by generating a distribution for the same.

In addition, the present invention provides a reference point selected from the source image or Another object is to provide an apparatus and method for performing a matching process of searching for a corresponding point that is a selected reference point or a neighboring point in the target image corresponding to a neighboring point and matching the source image and the target image based on the searched corresponding point. The purpose.

In addition, in the present invention, in the entire registered area, if the matching ratio of the plurality of reference points set in the source image with the reference point of the target image exceeds a preset threshold, the matching is finally terminated. Another purpose is to provide an apparatus and method for performing the matching by repeatedly performing the matching process until the matching value exceeds the threshold value when the threshold value is below the threshold value.

A corresponding point-based image matching device according to an embodiment of the present invention uses a histogram generator to generate a histogram for each of a plurality of reference points set in a source image and a target image each including a common area, and the generated histogram. and an image matching unit that matches the source image and the target image by searching for corresponding points corresponding to the source image and the target image.

In addition, the image registration device further includes a neighboring point selection unit that selects a plurality of neighboring points neighboring each reference point, and a vector generation unit that generates vectors for each of the selected plurality of neighboring points.

In addition, the histogram generator calculates the angle of the interior angle for all selectable vector pairs, one length ratio for each pair of vectors, and the angle of the exterior angle and another length ratio for each pair of vectors generated, and calculates the angle of the exterior angle and another length ratio for each pair of vectors that can be selected. The distribution is calculated according to the range of and a histogram is generated for each reference point.

In addition, the rows of the histogram are generated as many as the number of reference points selected, and the columns of the histogram are generated as many as the number of sections for the preset angle range multiplied by the number of sections for the length ratio range. , the coefficients of each bin constituting each row and column of the generated histogram are set according to the calculated angle of the interior angle and the length ratio of one, and the angle of the exterior angle and the other length ratio. It is characterized by accumulating and filling the number corresponding to the range of angles and the range of length ratios.

In addition, the image registration unit generates a result histogram by multiplying the square root of the histogram for each reference point set in the source image and the square root of the result of transposing the rows and columns of the histogram for each reference point set in the target image. A bin having a coefficient exceeding a preset threshold is detected from the generator and the generated result histogram, and the reference point selected from the source image and the reference point selected from the target image matched through the detected bin are used as corresponding points. A corresponding point search unit that searches for the corresponding point by detecting, as corresponding points, at least two neighboring points with the same distribution of angles and length ratios calculated for a plurality of neighboring points, respectively, neighboring each reference point detected as the corresponding point; and It further includes a region selection unit that selects image regions including at least one searched corresponding point, and matches the source image and the target image with respect to each selected image region centered on the searched corresponding point. Do it as

In addition, the image registration unit is configured to operate the image registration unit when the matching ratio of the number of reference points matching the reference points set in the target image to all reference points set in the source image exceeds a preset threshold in the matched area. The matching is performed repeatedly while changing the selected image area.

Additionally, the image including the source image and target image is characterized as a pore image.

In addition, the corresponding point-based image matching method according to an embodiment of the present invention includes a histogram generating step of generating a histogram for each of a plurality of reference points set in the source image and the target image including a mutual common area, and the generated histogram. An image matching step of matching the source image and the target image by searching for corresponding points corresponding to the source image and the target image.

In addition, the image matching method further includes a neighboring point selection step of selecting a plurality of neighboring points neighboring each reference point, and a vector generation step of generating vectors for each of the selected plurality of neighboring points.

In addition, in the histogram generation step, in each of the generated vectors, the angle of the interior angle for all selectable vector pairs, one length ratio for each pair of vectors, and the angle of the exterior angle and another length ratio are calculated for each corresponding vector pair. The distribution is calculated according to the range of the bin, and a histogram is generated for each reference point.

In addition, the image registration step generates a resulting histogram by multiplying the square root of the histogram for each reference point set in the source image and the square root of the result of transposing the rows and columns of the histogram for each reference point set in the target image. In the histogram generation step and the generated result histogram, a bin having a coefficient exceeding a preset threshold is detected, and a reference point selected from the source image and a reference point selected from the target image that are matched through the detected bin are used as corresponding points. A corresponding point search step of searching for the corresponding point by detecting at least two neighboring points with the same angle and length ratio distribution as corresponding points for a plurality of neighboring points, respectively neighboring each reference point detected as the corresponding point. and a region selection step of selecting image regions including at least one searched corresponding point, wherein for each selected image region, matching the source image and the target image around the searched corresponding point. It is characterized by

In addition, in the image matching step, as a result of the matching, in the matched area, the matching ratio for the number of reference points matching the reference points set in the target image with respect to all reference points set in the source image is a preset threshold. The matching is performed repeatedly while changing the selected image area until the value is exceeded.

As described above, the corresponding point-based image matching device and method of the present invention are based on the distribution of a plurality of reference points each set on a source image that serves as a matching standard among a plurality of images including a common area and a target image for matching the source image. Accordingly, for each reference point, a distribution map for the plurality of neighboring points selected around each reference point is calculated to generate a histogram for each reference point, and the target corresponding to the reference point of the source image is used using the generated histogram. By searching for the reference point of the image, it is possible to precisely match the source image and the target image.

In addition, when generating a histogram for the reference point, the present invention generates vectors for the plurality of selected neighboring points centered on each reference point, and the angle of the interior angle for all vector pairs that can be selected from the generated vectors. and calculating the length ratio of the pair of vectors in the direction for calculating the angle of the interior angle and another length ratio of the pair of vectors in the direction for calculating the angle of the exterior angle and the angle of the exterior angle, and calculating the angle and the angle length. By increasing the coefficient for the bin of the histogram corresponding to the ratio, there is an effect of generating a histogram for the reference point so that the corresponding point can be easily searched.

1 is a conceptual diagram illustrating a corresponding point-based image matching device and method according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a method of calculating a distribution for neighboring points centered on a reference point according to an embodiment of the present invention.
Figure 3 is a diagram showing a histogram for a reference point according to an embodiment of the present invention.
Figure 4 is a diagram illustrating a process of searching for corresponding points according to an embodiment of the present invention.
Figure 5 is a diagram illustrating a process of matching a source image and a target image according to an embodiment of the present invention.
Figure 6 is a block diagram showing the configuration of a corresponding point-based image matching device according to an embodiment of the present invention.
Figure 7 is a block diagram showing the configuration of an image matching unit according to an embodiment of the present invention.
Figure 8 is a flowchart showing a procedure for matching images based on corresponding points according to an embodiment of the present invention.
Figure 9 is a flowchart showing a detailed procedure for the image matching step according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the corresponding point-based image matching device and method of the present invention will be described in detail with reference to the attached drawings. The same reference numerals in each drawing indicate the same member. In addition, specific structural and functional descriptions of the embodiments of the present invention are merely illustrative for the purpose of explaining the embodiments of the present invention, and unless otherwise defined, all terms used herein, including technical or scientific terms, are provided. The terms have the same meaning as generally understood by those skilled in the art to which the present invention pertains. Terms as defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings they have in the context of the related technology, and unless clearly defined in this specification, should not be interpreted in an idealized or overly formal sense. It is desirable not to. In the present invention, data can be interpreted as digital information.

1 is a conceptual diagram illustrating a corresponding point-based image matching device and method according to an embodiment of the present invention. In the corresponding point-based image matching device and method of the present invention, the image is preferably applied to the pore image, but can naturally be applied to other images as well.

As shown in FIG. 1, when a plurality of images including a common area are input, the corresponding point-based image matching device 100 according to an embodiment of the present invention uses a source image and a source image that serve as a standard for matching for the plurality of images. Set the target image to match the image.

A plurality of images may be acquired through the same image acquisition means and input into the same coordinate system, or may be acquired through different image acquisition means and input into different coordinate systems, and may have the same or different scales.

Additionally, the corresponding point-based image matching device 100 sets a plurality of reference points in the source image and the target image, respectively.

Here, the reference point refers to the feature point of the source image and target image. For example, if the plurality of images are pore images taken from the scalp, the pores are set as the reference point. However, it is not limited to this, and depending on the type of image or the type of object included in the image, the intersection point, end point, maximum and minimum point of the curve, corner point, etc. of the line included in each image may be set as the reference point.

Additionally, the reference point can be set automatically through various feature point extraction algorithms for extracting feature points, such as SIFT (scale invariant feature transform), or can be set directly by the user.

In addition, the corresponding point-based image matching device 100 selects each reference point set in the source image and the target image and a plurality of neighboring points neighboring each reference point, and generates a histogram for each selected reference point. At this time, an image area formed by each selected reference point and a plurality of neighboring points is selected.

That is, the corresponding point-based image matching device 100 calculates the distribution of a plurality of neighboring points neighboring each reference point for each reference point, and uses the calculated distribution of the plurality of neighboring points for each reference point to determine each reference point. The goal is to create a histogram. Meanwhile, generating a histogram for each reference point will be described in detail with reference to FIGS. 2 and 3.

Additionally, in order to match the source image and the target image, the corresponding point-based image matching device 100 first uses a histogram for each reference point to search for corresponding points that correspond to each other between reference points set in the source image and the target image.

In addition, the corresponding point-based image matching device 100 selects a plurality of neighboring points from the source image and the target image, respectively, neighboring each reference point searched for as a corresponding point, and selects at least two neighbors with the same calculated distribution among the plurality of neighboring points. By detecting a point as a corresponding point, at least two corresponding points among the plurality of neighboring points are searched for.

Additionally, the corresponding point-based image matching device 100 selects an image area including the searched corresponding point from the source image and the target image, respectively, and matches the source image and the target image centered on the searched corresponding point. At this time, it is preferable to use the iterative closest point (ICP) method, which performs image registration based on the feature points that correspond between the two images, but the present invention is not limited to this and image registration is performed using the corresponding points between the two images. There are various ways to do this.

At this time, the process of searching for corresponding points and performing the matching is repeatedly performed until the matching result exceeds a preset threshold. Meanwhile, the process of searching for corresponding points and performing matching will be described in detail with reference to FIGS. 4 and 5.

In addition, after matching the source image and the target image, if another target image exists, the matching result is set as the source image, and the process of searching and matching the corresponding points is repeatedly performed to match all the input images. is finally completed.

Hereinafter, the process of calculating the distribution of a plurality of neighboring points for each reference point and the process of generating a histogram for each reference point using the distribution of the calculated neighboring points will be described.

Figure 2 is a diagram illustrating a method of calculating a distribution for neighboring points centered on a reference point according to an embodiment of the present invention. Below, the process of calculating the distribution of a plurality of neighboring points for one reference point will be described. However, calculating the distribution for the plurality of neighboring points is performed for each reference point.

As shown in FIG. 2, the corresponding point-based image matching device 100 according to an embodiment of the present invention uses a plurality of reference points set in each image to generate a histogram for a plurality of reference points set in the source image and the target image. Each reference point is selected and a plurality of neighboring points adjacent to each selected reference point are selected.

At this time, when selecting each reference point and a plurality of neighboring points neighboring each reference point, an image area for each image formed by each reference point and a plurality of neighboring points neighboring each reference point is selected, as described above. same.

Additionally, the corresponding point-based image matching device 100 first calculates the distribution of each selected reference point and a plurality of neighboring points neighboring the reference point for each reference point.

At this time, the corresponding point-based image matching device 100 selects a plurality of neighboring points as many as a preset number centered on each selected reference point. In addition, a plurality of neighboring points are selected in the order of the closest distance around the reference point, up to a preset number. It is desirable to select 4 neighboring points, but 3 to 5 or more may be selected.

Additionally, the corresponding point-based image matching device 100 generates vectors for a plurality of neighboring points selected around each reference point.

In addition, the corresponding point-based image registration device 100 provides information on the angle of the interior angle and each pair of vectors according to the number of sections for the range of the preset angle and the number of sections for the range of the length ratio in each generated vector. By calculating one length ratio, the angle of the exterior angle, and another length ratio for each pair of vectors, the distribution for a plurality of neighboring points neighboring each selected reference point is calculated.

At this time, it is calculated using all selectable vector pairs, and practically means the distribution of all selectable pairs of neighboring points from a plurality of neighboring points selected around the selected reference point. Meanwhile, a neighboring point may be another reference point, and the reference point may be a neighboring point of another reference point.

Additionally, the interior angle refers to the angle formed inside each pair of vectors and is calculated counterclockwise (or clockwise). Additionally, the length ratio of one pair of vectors is calculated as the direction in which the interior angles are calculated.

For example, when vectors for neighboring point #3 and neighboring point #4 are respectively created centering on the reference point (reference point #1) shown in Figure 2, the length ratio of one for the generated vector pair is the interior angle. The ratio of the length of the vector for neighboring point #4 to the length of the vector for neighboring point #3 is calculated in the counterclockwise direction at the time of calculation.

Additionally, the exterior angle refers to the angle formed outside each vector pair and is calculated counterclockwise (or clockwise). Additionally, another length ratio for a pair of vectors is calculated as the direction when calculating exterior angles. That is, when vectors for neighboring point #3 and neighboring point #4 are respectively created centering on the reference point shown in FIG. 2, the length ratio of the generated vector pair is the neighboring point # in the counterclockwise direction when calculating the exterior angle. The ratio of the length of the vector for 3 to the length of the vector for neighboring point #4 is calculated.

In addition, as shown in Figure 2, it can be seen that two distributions for a plurality of neighboring points are generated for each pair of vectors, including the angle of the interior angle and one length ratio, and the angle of the exterior angle and another length ratio.

In other words, when a total of 4 neighboring points are selected around a specific reference point, a total of 6 vector pairs are generated, and the distribution of neighboring points neighboring the reference point is calculated for a total of 12, 2 for each pair of neighboring points.

Figure 3 is a diagram showing a histogram for a reference point according to an embodiment of the present invention.

As shown in Figure 3, the histogram for each reference point set in the source image and target image according to an embodiment of the present invention is for the reference point and a plurality of neighboring points neighboring the reference point selected around the reference point. It is created using the calculated distribution.

The histogram for each reference point is created in a matrix-like format. The rows of the histogram are created as many as the number of reference points selected from the source image and the target image, and the columns of the histogram are the number of sections for the preset angle range and It is generated as many times as the number of sections for the length ratio range.

According to one embodiment of the present invention, if the preset range for each bin is set in units of 120 degrees for the angle, the angle range is divided into three sections and the length ratio is 0 to 0.5. and 0.5 to infinity, it is divided into two sections, so the total number of rows in the bin is 3x2 = 6. However, a smaller value can be used for the angle range or length ratio range, so it is natural that the number of bins increases. According to another embodiment of the present invention, if the angle is set in units of 30 degrees, the angle range is divided into 12 sections, and the length ratio is set to 0 to 0.5, 0.5 to 1.0, 1.0 to 1.5, and 1.5 to infinity. Since it is divided into 4 sections, the total number of columns is 12x4 = 48.

Ultimately, in one embodiment of the present invention, the case where the number of columns is 6 is described, but the actual number of columns can be configured in various ways depending on settings.

Additionally, for each bin of a histogram consisting of rows and columns, the angle range and vector length ratio range are set in advance.

In addition, the corresponding point-based image matching device 100 provides a distribution of the reference point and the neighboring point (i.e., a vector according to the direction in which the angle of the interior angle and the angle of the interior angle for the pair of vectors generated centered on the reference point and the neighboring point are calculated). When the length ratio and the angle of the exterior angle and another length ratio of the vector according to the direction in which the angle of the exterior angle is calculated are calculated, a bin is constructed for each reference point according to the calculated distribution and corresponds to a preset range. A histogram for each reference point is finally created by accumulating and filling the coefficients.

For example, the angle and length ratio of the first bin of the row constructed for each reference point is set to 0 to 120 degrees and 0 to 0.5, and the angle (internal or external angle) and length of the calculated distribution above are 0 to 120 degrees. If there are two numbers with a ratio between 0 and 0.5 (one length ratio or another length ratio), the coefficients of the corresponding bins are accumulated (i.e., 2) to fill the number. That is, the corresponding point-based image matching device 100 generates a histogram for each reference point using the distribution of a plurality of neighboring points neighboring the calculated reference point.

According to the distribution of neighboring points for reference point #1 calculated in FIG. 2, it can be seen that the coefficients of each bin formed in the row of the histogram constructed for the reference point #1 are 1, 1, 1, 3, 4, and 2.

Below, we will explain in detail how to match the source image and the target image by searching for corresponding points.

Figure 4 is a diagram illustrating a process of searching for corresponding points according to an embodiment of the present invention.

As shown in FIG. 4, in order to match a source image and a target image according to an embodiment of the present invention, the corresponding point-based image matching device 100 uses an angle generated for each of a plurality of reference points set in the source image. Calculate the square root of the histogram (Hist _source in Figure 4) for the reference point. That is, the corresponding point-based image registration device 100 calculates the square root of the histogram for each reference point by calculating the square root of the coefficient for each bin of the generated histogram for each reference point set in the source image.

In addition, the corresponding point-based image registration device 100 transposes the rows and columns of the histogram (Hist _target in FIG. 4) for each reference point generated for each reference point set in the target image, and then takes the square root of the transposed result. Calculate. That is, the corresponding point-based image registration device 100 converts the histogram for each reference point set in the target image into a transpose matrix, and calculates the square root of the corresponding histogram converted into the transpose matrix.

In addition, the corresponding point-based image registration device 100 multiplies the square root of the histogram for each reference point set in the calculated source image by the square root of the result of converting the histogram for each reference point set in the calculated target image into a transpose matrix to obtain a result. Create a histogram.

At this time, each reference point set in the source image and each reference point set in the target image are matched by intersecting through the bins of the resulting histogram.

In addition, the corresponding point-based image matching device 100 checks the coefficient for each bin of the resulting histogram, detects a bin with a coefficient exceeding a preset threshold, and matches the reference point selected from the source image through the bin. The reference point selected from the target image is detected as a corresponding point.

That is, the corresponding point-based image matching device 100 uses the reference point set in the source image and the reference point set in the target image as corresponding points through bins with coefficients exceeding a preset threshold in the result histogram. By making judgments, we search for corresponding points that correspond to each other between reference points.

For example, as shown in Figure 4, if the coefficients of the bins of row 1, column 1, row 2, column 4, and row 3, column 5 of the resulting histogram exceed the preset threshold, the reference point # set in the source image 1, reference point #2 and reference point #3, and reference point #1, reference point #4, and reference point #5 set in the target image correspond to each other, and are each searched for as corresponding points between reference points.

Meanwhile, in Figure 4, the number of reference points set in the source image and the target image is shown as 5. However, when each image is a pore image, the reference point is set for each pore. Not only some of the hair pore points shown in Figure 4 can be used as reference points. In other words, all pore points in one image can be reference points and can also be neighboring points. Depending on the user's settings, only some neighboring points can be set as reference points and neighboring points. In addition, the histogram can be updated by selecting the k closest neighboring reference points for each reference point according to the preset number k of neighboring points (e.g., k=4), expressing them as neighbors, and calculating their distribution. Therefore, because any reference point can be a neighboring point of another reference point, there are no restrictions or restrictions on the reference point or neighboring points.

Here, the threshold value is set by the following [Equation 1].

[Equation 1]

Threshold = ( _K C ₂ ) x (1.0 - (2 x outlier_ratio)) x 2

Here, K is the number of the selected neighboring points (e.g. 4) for each reference point, and the outlier_ratio is the number of reference points of the source image and the It refers to the ratio of reference points without corresponding points that correspond to each other in the target image.

At this time, if the ratio of reference points without corresponding points as a result of matching is high, it means that proper matching was not performed. Therefore, the outlier_ratio represents the approximate ratio of reference points that do not have corresponding points as a result of the matching and is set in advance. In the present invention, the outlier_ratio was set to 0.35.

Meanwhile, by checking the coefficients for each bin of the resulting histogram, the coefficients of bins with coefficients exceeding the preset threshold are set to 1 (True), and the coefficients of other bins are set to 0 (False). Thus, the resulting histogram can be binarized, and when searching for corresponding points between the reference points, the corresponding points between the reference points can be searched by referring to the coefficient of the bin of the binarized result histogram and detecting a bin set to 1 (True). there is.

In addition, after searching for corresponding points between reference points, the corresponding point-based image matching device 100 selects each reference point found as a corresponding point and selects a plurality of neighboring points adjacent to each selected reference point.

In addition, the corresponding point-based image matching device 100 compares the distributions calculated for a plurality of neighboring points selected from the source image and a plurality of neighboring points selected from the target image, and as a result of the comparison, at least two neighbors having the same distribution are found. By detecting a point as a corresponding point, the corresponding point between the neighboring points is searched.

At this time, the distribution for the plurality of neighboring points is calculated for each pair of neighboring points that can be selected for the plurality of neighboring points, and when mutually corresponding points between the neighboring points are searched, at least two (i.e., one pair of neighboring points) are calculated for each pair of neighboring points that can be selected for the plurality of neighboring points. Point) or more neighboring points are searched for as corresponding points.

That is, the corresponding point-based image matching device 100 detects at least two neighboring points with the same distribution of angles and length ratios calculated for a plurality of neighboring points neighboring each reference point selected from the source image and the target image as corresponding points. By doing so, the corresponding points between the neighboring points are searched.

Additionally, the corresponding point-based image matching device 100 selects an image area including at least one searched corresponding point for the source image and the target image, respectively.

Additionally, the corresponding point-based image matching device 100 matches the source image and the target image centered on the searched corresponding point for each selected image area. The process of performing matching will be explained with reference to FIG. 5.

Figure 5 is a diagram illustrating a process of matching a source image and a target image according to an embodiment of the present invention.

As shown in FIG. 5, the corresponding point-based image matching device 100 according to an embodiment of the present invention searches for corresponding points for each reference point through the result histogram, and as a result, corresponding points for each reference point and a plurality of selected points for each reference point are found. When at least two corresponding points are searched for between neighboring points, an image area containing the specific reference point and the neighboring points searched for by the corresponding points is selected.

That is, the number of reference points searched for as corresponding points is more than a preset number (threshold number of corresponding points), and a specific reference point among the reference points searched for as corresponding points is formed by at least two or more neighboring points neighboring the specific reference point searched as corresponding points. Image areas are selected from the source image and the target image, respectively, and the source image and the target image are matched around the searched corresponding points.

The number of critical corresponding points is set according to the following [Equation 2].

[Equation 2]

Number of critical corresponding points = (K) x (1 - 2 x outlier_ratio)

Here, K and the outlier_ratio are the same as K and outlier_ratio in [Equation 1].

For example, as shown in Figure 5, a specific reference point (reference point #1) selected from the source image by searching for corresponding points, neighboring point #2 and neighbor point #3 neighboring the specific reference point are selected from the target image (reference point #1). By selecting image areas including reference point #1) and neighboring point #1 and neighboring point #2, respectively, reference point #1, neighboring point #2, and neighboring point #4 of the source image searched for the corresponding point and the target image The source image and the target image are matched around the reference point #1, neighboring point #1, and neighboring point #4.

At this time, the corresponding point-based image matching device 100 performs the matching by first selecting and matching one of the corresponding points found in the selected image area and sequentially matching the remaining corresponding points. For example, when the corresponding point-based image matching device 100 performs the matching using the ICP method, a specific corresponding point (e.g., neighboring point #2 of the source image and neighboring point # of the target image searched as corresponding points in FIG. 4) 1) is selected and used as initial guess information to first match, and then the matching is performed on the remaining corresponding points.

Additionally, the corresponding point-based image registration device 100 verifies the registration result. Verification is the result of registration. In the registered area, the registration ratio for the number of reference points matching the reference points set in the target image for all reference points set in the source image exceeds the preset threshold (threshold matching ratio). This is done by determining whether or not

As a result of verification, if the matching ratio exceeds the threshold, the corresponding point-based image matching device 100 ends the matching and outputs the matching result.

Meanwhile, as a result of verification, if the matching ratio is below the threshold, the corresponding point-based image matching device 100 detects a new reference point as a corresponding point in the source image and the target image and at least two neighbors of the reference point until the matching point exceeds the threshold. Registration is performed repeatedly by selecting the above neighboring points and selecting a new image area.

In other words, the corresponding point-based image registration device 100 repeats registration while changing the image area by newly selecting a reference point searched for as a corresponding point and a neighboring point adjacent to the reference point until the matching ratio exceeds a preset threshold. It is performed by.

Figure 6 is a block diagram showing the configuration of a corresponding point-based image matching device according to an embodiment of the present invention.

As shown in FIG. 6, the corresponding point-based image matching device 100 according to an embodiment of the present invention includes an image input unit 110 that receives a plurality of images including mutually common areas, and a plurality of images received. An image setting unit 120 that classifies and sets a source image and a target image, a reference point setting unit 130 that sets a plurality of reference points for the set source image and target image, and a reference point setting unit 130 that sets a plurality of reference points for each image. A neighboring point selection unit 140 that selects a plurality of neighboring points, a vector generating unit 150 that generates a vector for each selected neighboring point with each reference point as the center, and each reference point using the generated vector. A histogram generator 160 that generates a histogram for each reference point, searches for corresponding points corresponding to the source image and the target image using the histogram for each reference point, and matches the target image to the source image. It is configured to include an image registration unit 170 and an output unit 180 that outputs the registration result.

The image input unit 110 performs a function of receiving a plurality of images including a common area through various image acquisition means such as a camera, scanner, ultrasonic imaging device, etc.

In addition, the image setting unit 120 performs the function of setting a source image that serves as a standard for matching among the plurality of images input and a target image for matching the source image.

In addition, the reference point setting unit 130 sets a plurality of reference points for the source image and the target image, and the reference points refer to feature points of the source image and the target image, as described above.

In addition, the neighboring point selection unit 140 selects a plurality of neighboring points neighboring each of the plurality of reference points set for the source image and the target image as a basis for generating a histogram for each reference point.

At this time, as described above, the image areas formed by each reference point and a plurality of neighboring points adjacent to the reference point are selected.

In addition, a plurality of neighboring points neighboring each reference point are selected as the nearest neighboring points from each reference point, and a preset number of neighboring points are selected.

Additionally, the vector generator 150 generates vectors for a plurality of neighboring points adjacent to each reference point, with each reference point as the center.

Additionally, the histogram generator 160 calculates the distribution of each reference point and a plurality of neighboring points adjacent to each reference point, thereby generating a histogram for each reference point.

At this time, the histogram generator 160 generates a range of the angle of the interior angle and one length ratio and the angle of the exterior angle and another length ratio according to a preset range for each vector generated by the vector generator 150. By calculating the distribution of the number corresponding to , the distribution of a plurality of neighboring points neighboring each reference point is calculated for each reference point, thereby generating a histogram for all reference points.

Meanwhile, since generating a histogram for each reference point was explained with reference to FIGS. 2 and 3, further detailed description will be omitted.

In addition, the image matching unit 170 searches for corresponding points between the source image and the target image using the histogram for each generated reference point, and performs a function of matching the source image and the target image based on the searched corresponding points. Therefore, the detailed configuration of the image matching unit 170 will be described with reference to FIG. 7.

Additionally, the output unit 180 performs the function of outputting the final registration result from the image registration unit 170 to a display (not shown), a user terminal (not shown), etc.

Figure 7 is a block diagram showing the configuration of an image matching unit according to an embodiment of the present invention.

As shown in FIG. 7, the image registration unit 170 according to an embodiment of the present invention combines a histogram for each reference point set in the source image generated through the histogram generator 160 and an angle set in the target image. A result histogram generator 171 that generates a result histogram using a histogram for a reference point, a corresponding point search unit 172 that searches for corresponding corresponding points between the source image and the target image using the generated result histogram, An area selection unit 173 that selects an image area including the searched at least one corresponding point from the source image and the target image, respectively, and matches the source image and the target image around the corresponding point included in the selected image area. It is configured to include a matching unit 174 that verifies the matched result and a matching result verification unit 175 that verifies the matched result.

The result histogram generator 171 generates the square root of the histogram for each reference point set in the source image generated by the histogram generator 160 and the square root of the result of transposing the rows and columns of the histogram for each reference point set in the target image. is multiplied to generate a resulting histogram in which each reference point set in the source image and target image matches each other through bins. Meanwhile, since generating the resulting histogram has been described with reference to FIG. 4, further detailed description will be omitted.

In addition, the corresponding point search unit 172 detects a bin having a coefficient exceeding a preset threshold in the result histogram generated by the result histogram generator 171, and sets the bin in the source image matched through the detected bin. One reference point and a reference point set in the target image are detected as corresponding points, and at least two neighboring points with the same distribution of angles and length ratios calculated for a plurality of neighboring points neighboring each reference point detected as the corresponding points are corresponding points. By detecting, corresponding points that correspond to each other for the source image and the target image are searched.

Additionally, the area selection unit 173 selects an image area formed by a specific reference point searched for as a corresponding point and at least two neighboring points neighboring the specific reference point from the source image and the target image, respectively.

Additionally, the matching unit 174 matches the source image and the target image based on the searched corresponding points included in each selected image area. Meanwhile, if the number of corresponding points searched for the reference point is less than a preset number, the matching unit 174 determines that there is no need to match the source image and the target image (i.e., it determines that there is little or no common area). Therefore, registration is not performed.

In addition, according to the matching result, the matching result verification unit 175 determines whether the matching ratio of all reference points set in the source image in the matched area with the reference points set in the target image exceeds a preset threshold. In this case, the matching of the source image and the target image is finally completed, and the matching result is output through the output unit 180.

Meanwhile, as a result of matching, if the matching ratio is below the pre-set threshold, the image matching unit 170 selects each of the other reference points found as corresponding points until it exceeds the pre-set threshold and selects the source image. The registration is performed repeatedly while changing each image area selected for the target image.

Figure 8 is a flowchart showing a procedure for matching images based on corresponding points according to an embodiment of the present invention.

As shown in FIG. 8, the procedure for matching images based on corresponding points according to an embodiment of the present invention is first, the corresponding point-based image matching device 100 receives a plurality of images including a common area. Perform the steps (S110).

A plurality of images may be input from a user or through an image acquisition means such as a camera or scanner.

Next, the corresponding point-based image matching device 100 performs an image setting step of setting a source image that serves as a standard for matching among the plurality of input images and a target image to be matched with the source image (S120).

Next, the corresponding point-based image matching device 100 performs a reference point setting step of setting a plurality of reference points for the source image and the target image (S130).

As described above, a plurality of reference points can be set automatically or by the user using various algorithms (e.g., SIFT algorithm) for setting feature points from images. However, in the present invention, there is a limit to setting reference points. do not leave

Next, the corresponding point-based image matching device 100 selects each reference point set in the source image and the target image and a plurality of neighboring points neighboring each reference point, and performs a histogram generation step to generate a histogram for each reference point. Do it (S130).

As described above, the histogram for each reference point is generated using the distribution calculated for a plurality of neighboring points adjacent to each reference point.

At this time, the corresponding point-based image matching device 100 includes a neighboring point selection step of selecting a plurality of neighboring points as many as a preset number for each reference point to generate a histogram for each reference point, and at least one or more of the selected points. A vector generation step is further performed to generate vectors for each of the selected plurality of neighboring points centered on each reference point.

In addition, the corresponding point-based image matching device 100 searches for corresponding corresponding points in the source image and the target image using a histogram for each reference point, and matches the source image and the target image according to the results of searching for the corresponding points. Perform the image registration step. The image matching step will be described in detail with reference to FIG. 9.

Meanwhile, if another target image remains after matching, the matching result is set as the source image, and step S150 is repeatedly performed until all of the other target images are matched.

Figure 9 is a flowchart showing a detailed procedure for the image matching step according to an embodiment of the present invention.

As shown in FIG. 9, the detailed procedure for matching the source image and the target image according to an embodiment of the present invention is first, the image matching step is set in the histogram and target image for each reference point set in the source image. A result histogram generation step is performed to generate a result histogram using the histogram for each reference point (S210).

Each reference point set in the source image and each reference point set in the target image are matched with each other by each bin constituting the result histogram. Since the creation of the result histogram was explained with reference to FIG. 4, further detailed description will be omitted. .

Next, the image matching step detects bins with coefficients exceeding a preset threshold in the resulting histogram, and detects the reference point set in the source image and the reference point set in the target image as corresponding points through the detected bin. Step (S220) and a corresponding point detection step of detecting as corresponding points at least two neighboring points with the same distribution of angles and length ratios calculated for a plurality of neighboring points, respectively, neighboring each reference point detected as the corresponding point. (S230).

At this time, as described above, in the image matching step, if the result of detecting the reference point as a corresponding point does not exceed a preset number, the source image and the target image are not matched.

Next, in the image matching step, a region selection step is performed to select an image region containing the searched at least one corresponding point (S240).

That is, the image matching step is to select an image area formed by a specific reference point among the reference points searched for by the corresponding points and at least two neighboring points neighboring the specific reference point from the source image and the target image, respectively.

Next, the image matching step is performed to match the source image and the target image for each selected image area centered on the searched corresponding points (S250).

That is, the matching step is to match the source image and the target image centered on at least one corresponding point included in each selected image area.

Next, in the image matching step, a matching result verification step is performed to verify the result of matching the source image and the target image. As a result, in the matched area, the matching ratio for all reference points set in the source image is If it exceeds the preset threshold (S260), matching of the source image and target image is terminated and an output step is performed to output the matching result. Meanwhile, since the matching ratio has been described with reference to FIGS. 4 and 5, further detailed description will be omitted.

In addition, in the image matching step, if the matching ratio is below the preset threshold (S260) as a result of performing the matching result verification step, the image matching step involves performing a step of changing each selected image area (S261). Steps S250 and S261 are repeatedly performed until the matching ratio exceeds the preset threshold.

As described above, the present invention is a corresponding point-based image matching device and method that calculates the distribution of a plurality of neighboring points for each reference point set in a plurality of images including a mutually common area, There is an effect in that the plurality of images can be precisely determined by generating a histogram for and searching for corresponding corresponding points in the plurality of images using the histogram for each reference point.

In the above, the preferred embodiments according to the present invention have been described in detail, but the technical idea of the present invention is not limited thereto, and each component of the present invention is changed or modified within the technical scope of the present invention to achieve the same purpose and effect. It could be.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

100: Corresponding point-based image matching device 110: Image input unit
120: image setting unit 130: reference point setting unit
140: Neighboring point selection unit 150: Vector generation unit
160: histogram generation unit 170: image matching unit
171: Result histogram generation unit 172: Corresponding point search unit
173: Area selection unit 174: Matching unit
175: Matching result verification unit 180: Output unit

Claims (14)

In the corresponding point-based image matching device,
a histogram generator that generates a histogram for each of a plurality of reference points set in the source image and target image including mutual common areas; and
An image matching unit that matches the source image and the target image by searching for corresponding points between the source image and the target image using the generated histogram,
The image matching device,
a neighboring point selection unit that selects a plurality of neighboring points neighboring each reference point; and
It further includes a vector generator that generates vectors for each of the selected plurality of neighboring points,
The histogram generator,
In each of the generated vectors, the angle of the interior angle and one length ratio for each pair of vectors that can be selected, and the angle of the exterior angle and another length ratio are calculated and distributed according to the range of each corresponding bin. A corresponding point-based image registration device characterized by calculating and generating a histogram for each reference point. delete delete In claim 1,
The rows of the histogram are generated as many as the number of reference points selected,
The rows of the histogram are generated as many as the number of sections for the preset angle range multiplied by the number of sections for the length ratio range,
The coefficient of each bin constituting each row and column of the generated histogram is the angle set according to the calculated angle of the interior angle and the length ratio of one, and the angle of the exterior angle and the another length ratio. A corresponding point-based image matching device characterized by accumulating and filling the number corresponding to the range and length ratio of . In claim 1,
The image matching unit,
a result histogram generator that generates a result histogram by multiplying the square root of the histogram for each reference point set in the source image and the square root of a result of transposing the rows and columns of the histogram for each reference point set in the target image;
A bin having a coefficient exceeding a preset threshold is detected in the resulting histogram, and a reference point set in the source image and a reference point set in the target image that are matched through the detected bin are detected as corresponding points. a corresponding point search unit that searches for the corresponding points by detecting as corresponding points at least two neighboring points with the same distribution of angles and length ratios calculated for a plurality of neighboring points, respectively, adjacent to each reference point detected as the corresponding point; and
It further includes a region selection unit that selects each image region including the searched at least one corresponding point,
A corresponding point-based image matching device, characterized in that for each selected image area, the source image and the target image are matched based on the searched corresponding point. In claim 5,
The image matching unit,
As a result of the matching, in the matched area, until the matching ratio for the number of reference points matching the reference points set in the target image with respect to all reference points set in the source image exceeds a preset threshold, Corresponding point-based image registration device characterized in that the registration is repeatedly performed while changing the selected image area. The method of any one of claims 1 and 4 to 6,
A corresponding point-based image matching device, wherein the image including the source image and the target image is a pore image. In the corresponding point-based image matching method,
In the corresponding point-based image registration device, a histogram generating step of generating a histogram for each of a plurality of reference points set in the source image and the target image including mutual common areas; and
In the corresponding point-based image matching device, an image matching step of matching the source image and the target image by searching for corresponding points corresponding to the source image and the target image using the generated histogram,
The corresponding point-based image matching method is,
In the corresponding point-based image matching device, a neighboring point selection step of selecting a plurality of neighboring points neighboring each reference point; and
In the corresponding point-based image matching device, it further includes a vector generation step of generating vectors for each of the selected plurality of neighboring points,
The histogram generation step is,
In each of the generated vectors, the angle of the interior angle and one length ratio for each pair of vectors that can be selected, and the angle of the exterior angle and another length ratio are calculated and distributed according to the range of each corresponding bin. A corresponding point-based image registration method characterized by calculating and generating a histogram for each reference point. delete delete In claim 8,
The rows of the histogram are generated as many as the number of reference points selected,
The rows of the histogram are generated as many as the number of sections for the preset angle range multiplied by the number of sections for the length ratio range,
The coefficient of each bin constituting each row and column of the generated histogram is the angle set according to the calculated angle of the interior angle and the length ratio of one, and the angle of the exterior angle and the another length ratio. A corresponding point-based image registration method characterized by accumulating and filling the number corresponding to the range and length ratio of . In claim 8,
The image matching step is,
A result histogram generation step of generating a result histogram by multiplying the square root of the histogram for each reference point set in the source image and the square root of the result of transposing the rows and columns of the histogram for each reference point set in the target image;
A bin having a coefficient exceeding a preset threshold is detected in the resulting histogram, and a reference point set in the source image and a reference point set in the target image that are matched through the detected bin are detected as corresponding points. A corresponding point search step of searching for the corresponding points by detecting at least two neighboring points with the same distribution of angles and length ratios as calculated for a plurality of neighboring points to each reference point detected as the corresponding point. and
It further includes a region selection step of selecting each image region including the searched at least one corresponding point,
A corresponding point-based image matching method, characterized in that for each selected image area, the source image and the target image are matched based on the searched corresponding point. In claim 12,
The image matching step is,
As a result of the matching, in the matched area, until the matching ratio for the number of reference points matching the reference points set in the target image with respect to all reference points set in the source image exceeds a preset threshold, A corresponding point-based image registration method characterized in that the registration is repeatedly performed while changing the selected image area. The method of any one of claims 8 and 11 to 13,
A corresponding point-based image matching method, wherein the image including the source image and the target image is a pore image.