JP5856888B2 - Corresponding point search method, corresponding point search device, and corresponding point search program - Google Patents

Description

  The present invention relates to a corresponding point search method, a corresponding point search device, and a corresponding point search program for searching for a corresponding point that associates an input image with a reference image to be compared with the input image, and particularly has a periodic pattern. The present invention relates to a corresponding point search method, a corresponding point search apparatus, and a corresponding point search program capable of acquiring a correct corresponding point at high speed and efficiency when performing association between included images.

  2. Description of the Related Art Conventionally, a technique is known in which when an input image input from an image input device such as a scanner is compared with a reference image registered in advance, a corresponding point between both images is searched and the images are associated with each other.

  For example, in Patent Document 1, a reference partial image and an input partial image are created by dividing a reference image and an input image, respectively, and a similarity image obtained by calculating the similarity between the reference partial image and the input partial image is created. A technique is disclosed in which a corresponding point is determined by sequentially performing a process of adding the maximum pixel value in the vicinity of the target pixel of the adjacent similarity image to the pixel value of the target pixel of the target similarity image while changing the target pixel. According to this patent document 1, it is possible to acquire a stable association result that does not easily fall into a local solution at high speed and efficiently.

JP 2004-54682 A

  However, when a pattern with periodicity is included in the input image and the reference image, due to factors such as the quantization error of the input image and the reference image, the position of the position where the periodic shift has occurred rather than the correct position similarity. There is a problem that the degree of similarity becomes large and a correct corresponding point cannot be obtained.

  For this reason, when a pattern with periodicity is included in an image, how to obtain a correct corresponding point is an important issue.

  The present invention has been made to solve the above-described problems caused by the prior art, and when matching between images including a periodic pattern is performed, correct corresponding points can be acquired quickly and efficiently. An object is to provide a corresponding point searching method, a corresponding point searching apparatus, and a corresponding point searching program.

  In order to solve the above-described problems and achieve the object, the present invention relates to corresponding points for associating a first image including a periodic pattern with a second image to be compared with the first image. A corresponding point search method in a corresponding point search device for searching for a plurality of blocks having a pixel value of a similarity between a divided image composed of a plurality of blocks obtained by dividing the first image and the second image. A similarity image generation step for generating a similarity image, and a similarity exceeding a predetermined threshold among the similarities included in each block forming the similarity image generated by the similarity image generation step A search range calculation step for calculating a search range where a corresponding point can exist, and a constraint range obtained by adding an allowable amount of a corresponding point difference between blocks to the search range calculated by the search range calculation step. Self Propagation processing is performed on adjacent blocks adjacent to the lock, and the search range is updated using the constraint range propagated from the adjacent block, and the difference between corresponding points between the blocks is allowed for the updated search range. Based on the search range narrowing step that narrows down the search range and the search range narrowed down by the search range narrowing step by repeating the process of propagating the constraint range to which the capacity is added to adjacent blocks. And a corresponding point specifying step of specifying the position of the corresponding point on the second image corresponding to the position of the block center of each block obtained by dividing the first image.

  Further, according to the present invention, in the above invention, based on the similarity exceeding a predetermined threshold among the similarities included in each block forming the similarity image generated by the similarity image generating step, A priority calculation step for calculating a priority for processing the block; and a storage step for storing the priority calculated in the priority calculation step in a priority storage unit, wherein the search range narrowing step includes: Based on the priority stored in the priority storage unit, the processing order of the blocks that propagate the restriction range to the adjacent blocks is determined.

  Further, in the present invention according to the above invention, the similarity image generation step includes a division processing step of dividing the first image into a plurality of blocks, each block divided by the division processing step, and the block And a similarity calculation step of calculating the similarity with the second image.

  Further, in the present invention according to the above invention, in the division processing step, the first image is divided into a plurality of blocks having a size in which a distortion within each block fits in one pixel, and the search range is narrowed down. The step is characterized in that a range obtained by adding one peripheral pixel to the search range calculated in the search range calculation step is set as a restriction range.

  Further, in the present invention according to the above invention, the similarity calculation step uses each block as a template, and the template is shifted to a first position shifted by a predetermined number of pixels from the center position of the second image. Is used as a template, and the degree of similarity is set as the pixel value of the pixel corresponding to the first position of the similarity image, and then this template is sequentially shifted and matched within a predetermined range of the second image. The similarity is calculated sequentially.

  Further, in the present invention according to the present invention, in the search range calculation step, the restriction range is calculated by adding an allowable amount of a corresponding point difference between blocks to the search range calculated by the search range calculation step. A restriction range calculation step, a first propagation processing step of propagation processing the restriction range calculated by the restriction range calculation step to an adjacent block adjacent to the own block, and a restriction range propagated from the adjacent block A search range update processing step using the search range update processing step, and an adjacent block adjacent to the search range updated by the search range update processing step, to which a permissible difference of corresponding points between blocks is added And a second propagation processing step for carrying out the propagation processing.

  The present invention also provides a corresponding point search in a corresponding point search device that searches for a corresponding point that associates a first waveform including a periodic pattern with a second waveform to be compared with the first waveform. A similarity waveform generation method for generating a similarity waveform composed of a plurality of blocks having pixel values of similarity between a divided waveform composed of a plurality of blocks obtained by dividing the first waveform and the second waveform And a search range in which corresponding points can exist based on a similarity exceeding a predetermined threshold among the similarities included in each block forming the similarity waveform generated by the step and the similarity waveform generating step. A search range calculation step for each calculation, and a restriction range obtained by adding a tolerance for the difference of corresponding points between blocks to the search range calculated by the search range calculation step for adjacent blocks adjacent to the own block While performing the propagation process, while updating the search range using the constraint range propagated from the adjacent block, the updated search range is adjacent to the constraint range to which an allowable amount of the corresponding point difference between the blocks is added. Each of the divided first waveforms based on the search range narrowing step for narrowing down the search range and the search range narrowed down in the search range narrowing step by repeating the process of performing propagation processing on adjacent blocks And a corresponding point specifying step of specifying the position of the corresponding point on the second waveform corresponding to the position of the block center of the block.

  Further, the present invention is a corresponding point search device that searches for corresponding points that associate a first image including a periodic pattern with a second image to be compared with the first image, Storage means for storing a divided image composed of a plurality of blocks obtained by dividing the first image, and a plurality of blocks having the similarity between the divided image stored in the storage means and the second image as pixel values Similarity image generation means for generating a similarity image, and a similarity degree exceeding a predetermined threshold value among similarities included in each block forming the similarity image generated by the similarity image generation means A search range calculation means for calculating a search range in which corresponding points can exist, and a restriction range obtained by adding an allowable amount of difference of corresponding points between blocks to the search range calculated by the search range calculation means. Bro Propagation processing is performed on adjacent blocks adjacent to the block, and the search range is updated using the constraint range that has been propagated from the adjacent block, and the allowable amount of the corresponding point difference between the blocks with respect to the updated search range Based on the search range narrowed down by the search range narrowing means and the search range narrowed down by the search range narrowing means, by repeating the process of propagating the constraint range to which adjacent blocks are added, Corresponding point specifying means for specifying the position of the corresponding point on the second image corresponding to the position of the block center of each block obtained by dividing the first image is provided.

  The present invention also provides a correspondence point search apparatus that searches for a correspondence point that associates a first image including a periodic pattern with a second image to be compared with the first image. A degree of similarity, which is a point search program, and generates a similarity image composed of a plurality of blocks having pixel values of similarity between a divided image composed of a plurality of blocks obtained by dividing the first image and the second image Search that may have a corresponding point based on a similarity that exceeds a predetermined threshold among the similarities included in each block that forms the similarity image generated by the image generation procedure and the similarity image generation procedure A search range calculation procedure for calculating each range, and a constraint range obtained by adding a tolerance for the difference of corresponding points between blocks to the search range calculated by the search range calculation procedure A constraint range in which a tolerance of difference in corresponding points between blocks is added to the updated search range while the search range is updated using a constraint range propagated from an adjacent block. The first image is extracted based on the search range narrowing procedure for narrowing down the search range and the search range narrowed down by the search range narrowing procedure by repeating the process of performing propagation processing on adjacent blocks. The computer is caused to execute a corresponding point specifying procedure for specifying the position of the corresponding point on the second image corresponding to the position of the block center of each divided block.

  According to the present invention, a similarity image composed of a plurality of blocks having pixel values representing the similarity between a divided image composed of a plurality of blocks obtained by dividing the first image and the second image is generated, and the generated similarity Based on the similarity exceeding a predetermined threshold among the similarities included in each block forming the degree image, the search range where the corresponding point can exist is calculated, and between the blocks with respect to the calculated search range Propagation processing is performed on adjacent blocks adjacent to the current block, with the tolerance of the difference between corresponding points added, and the updated search range is updated using the constraint range propagated from the adjacent block. Narrow the search by narrowing the search range by repeating the process of propagating the constrained range to the adjacent block, adding the tolerance of the corresponding point difference between the blocks to the range. Since the position of the corresponding point on the second image corresponding to the position of the block center of each block obtained by dividing the first image is specified based on the surrounding area, it is stable because a periodic pattern is included. And corresponding points can be obtained.

FIG. 1 is a functional block diagram illustrating the configuration of the corresponding point search apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of a reference image and an input image used in the first embodiment. FIG. 3 is a diagram illustrating an example of the reference partial image. FIG. 4 is an explanatory diagram for explaining the concept of creating a similarity image. FIG. 5 is a functional block diagram illustrating a configuration of the search range narrowing processing unit illustrated in FIG. 1. FIG. 6 is a diagram illustrating an example of priority data. FIG. 7 is an explanatory diagram for explaining the concept of search range narrowing processing by the search range narrowing processing unit illustrated in FIG. 1. FIG. 8 is a flowchart illustrating a processing procedure from division of a reference image to storage of a reference partial image. FIG. 9 is a flowchart showing a processing procedure up to the propagation process of the constraint range. FIG. 10 is a flowchart showing a processing procedure for narrowing the search range. FIG. 11 is a diagram illustrating another example of the reference image, the input image, and the similarity image. FIG. 12 is a functional block diagram illustrating the configuration of the corresponding point search apparatus according to the second embodiment. FIG. 13 is a diagram illustrating an example of a reference waveform, an input waveform, and a similarity graph. FIG. 14 is an explanatory diagram for explaining the concept of search range narrowing processing by the search range narrowing processing unit.

  Exemplary embodiments of a corresponding point search method, a corresponding point search apparatus, and a corresponding point search program according to the present invention will be described below in detail with reference to the accompanying drawings. In the following first embodiment, the case where an image is a processing target is shown, and in the second embodiment, a case where a waveform is a processing target is shown.

  FIG. 1 is a functional block diagram illustrating the configuration of the corresponding point search apparatus 1 according to the first embodiment. The corresponding point search apparatus 1 shown in FIG. 1 divides a reference image input in advance to generate a plurality of reference partial images, and corresponds to the center position of each reference partial image when the input image is input. It is an apparatus for searching for corresponding points on the input image to be performed.

  If processing that considers a periodic pattern is not performed as in the prior art, the similarity of the position where the period deviation occurred is larger than the similarity of the original correct position, and the correct corresponding point is It cannot be determined accurately. For this reason, the corresponding point search apparatus 1 is configured so that correct corresponding points can be acquired quickly and efficiently even when a periodic pattern is included in the input image and the reference image.

  Specifically, in the corresponding point search device 1, when a plurality of vertical lines are included as a periodic pattern in the input image and the reference image, the correspondence near the center of each image has no period deviation. Although it is difficult to find a point, considering the fact that it is easy to find a corresponding point with no period shift in the end block of each image, the corresponding point is accurately and efficiently processed by propagating the search range to the adjacent block. Seeking.

  Specifically, as illustrated in FIG. 1, the corresponding point search device 1 includes an image input unit 10, a division processing unit 11, a storage unit 12, a similarity image generation processing unit 13, and a search range narrowing down. It has a processing unit 14 and a corresponding point determination processing unit 15. Although the detailed description is omitted here for convenience of explanation, when registering a reference image, a registration mode for performing reference image registration processing is selected with a dip switch or the like (not shown), and corresponding points of the input image are obtained. In this case, a processing mode for searching for corresponding points is selected by a dip switch or the like (not shown).

  The corresponding point search apparatus 1 can be realized by a CPU, a memory, and the like that are components of a computer. Specifically, the image input unit 10, the division processing unit 11, and the similarity image generation processing unit 13. By storing a program corresponding to the search range narrowing processing unit 14 and the corresponding point determination processing unit 15 in a nonvolatile memory or the like, and reading and executing the program when the power is turned on on the CPU, each of the above functions Can be realized.

  The image input unit 10 is an input unit that inputs a reference image and an input image. Specifically, a scanner that optically reads a document to acquire an image, an interface unit that acquires an image from a network, and a reading unit that reads an image from a secondary storage device are applicable. When the reference image is input in the registration mode, the reference image is output to the division processing unit 11. When the input image is input in the processing mode, the input image is output to the similarity image generation processing unit 13. .

  FIG. 2A shows an example of a reference image 21 made up of 100 × 100 pixels, and FIG. 2B shows an example of an input image 22 made up of 100 × 100 pixels. Here, a case where a vertical line is included in both images as a periodic pattern is shown. The input image 22 is an image to be searched for corresponding points, and may be a distortion or deformation. On the other hand, the reference image 21 is an image to be compared with the input image 22, and it is desirable that the image is not accompanied by distortion.

  The division processing unit 11 is a processing unit that divides the reference image input by the image input unit 10 into reference partial images. The reference partial image divided by the division processing unit 11 is stored in the storage unit 12. FIG. 3 is a diagram showing an example of the division result of the reference image 21 shown in FIG. 2A. As shown in FIG. 3, the reference image 21 of 100 × 100 pixels is composed of 25 × 25 pixels. A case is shown in which division processing is performed on 16 reference partial images 21a.

  Here, the division processing unit 11 performs division processing so that the distortion in each reference partial image is at most contained in one pixel. In this way, in order to keep the distortion within one pixel, one side of the reference partial image may be set to the inverse of the assumed maximum distortion. For example, in order to cope with 4% local distortion, the image is divided into reference partial images having 1 / 0.04 = 25 pixels as one side. As a result, the processing can be performed on the assumption that the allowable amount of the difference between corresponding points is ± 1 pixel between the adjacent reference partial images 21a.

  The storage unit 12 is a storage device such as a hard disk device or a nonvolatile memory, and stores therein a reference partial image 21a, search range data 12b, constraint range data 12c, and priority data 12d. The reference partial image 21a is a partial image obtained by dividing the reference image 21 as described above.

  The search range data 12b is data indicating a search range that is a range in which corresponding points in each block of the similarity image can exist, and the constraint range data 12c is a constraint range that is propagated to adjacent blocks of the similarity image. Is calculated based on the search range. The search range and the constraint range both specify the range of the position of the corresponding point, and the search range is data indicating a range narrowed down in the processing process, whereas the constraint range is based on the similarity image. It means a range (hypothesis) in which a corresponding point to be calculated may exist. The search range is given an initial value in advance, and is narrowed down by propagation and application of the constraint range.

  In the first embodiment, since the division processing is performed so that the distortion in each reference image is at most one pixel, the range obtained by adding one pixel to the left and right of the search range is the constraint range data 12c. The priority data 12d is data that holds a block ID as a priority that is the processing order of each block of the similarity image. Detailed description of the search range, the constraint range, and the priority will be described later.

  The similarity image generation processing unit 13 is a processing unit that calculates the similarity between the input image and the reference partial image and generates a similarity image having the similarity as a pixel value. The similarity image is an image whose one side is the search range (initial value), and the similarity corresponding to the movement amount of the corresponding point is the pixel value. For example, as shown in FIG. 4, when the initial value of the search range is ± 20 pixels for both the vertical (y-axis) and horizontal (x-axis), the similarity image having values of −20 to 20 on each axis. Will be created. At this time, the similarity when the movement amounts are x and y is plotted on the coordinates (x, y) of the similarity image 23. For this similarity, for example, generalized Hough transform using an edge image, normalized correlation, pixel value square error, or the like can be used.

  For example, in FIG. 4, the similarity is calculated by overlaying the center position of the reference partial image 21a as a template at a position shifted by 20 pixels to the left and 20 pixels upward from the initial corresponding point of the input image 22, and the calculated similarity is The pixel value of the upper left pixel of the similarity image 23 is set as the pixel value. After that, the similarity is sequentially obtained while shifting the template, and when the similarity at the position shifted by 20 pixels to the right and 20 pixels below from the initial corresponding point of the input image 22 is obtained, it corresponds to the reference partial image 21a. A block of similarity images is completed. In the first embodiment, since there are 16 reference partial images 21a, as shown in FIG. 4, a similarity image 23 composed of 16 blocks corresponding to each reference partial image 21a is generated. . In this case, the size of each block of the similarity image 23 is 41 × 41 (= 20 × 2 + 1) pixels.

  Returning to the description of FIG. 1, the search range narrowing processing unit 14 is a processing unit that narrows down the search range in each block of the similarity image. As shown in FIG. 5, the search range narrowing processing unit 14 includes a constraint range calculation unit 14a, a priority calculation unit 14b, a constraint range application unit 14c, a constraint range propagation unit 14d, and a narrowing processing unit 14e. And have.

  The constraint range calculation unit 14a sets a range of pixel positions having a pixel value equal to or higher than a predetermined threshold among pixel values indicating the similarity in each block of the similarity image as a constraint range. For example, when the generalized Hough transform is used as the similarity, the threshold value can be a value equal to or greater than N% of the peak value that is the largest pixel value. When normalized correlation is used as the similarity, the value of the similarity is a value in the range of −1 to +1, and the threshold is determined in consideration of this point. Further, when the square error of the pixel value is used as the similarity, it is necessary to set a threshold value in consideration of the peak with the smallest pixel value.

  The priority calculation unit 14b is a processing unit that calculates the priority for determining the order in which the constraint range is applied. For example, when the generalized Hough transform is used as the similarity, the priority can be calculated in descending order of the peak value that is the largest pixel value. When normalized correlation is used as the similarity or when a square error of the pixel value is used, the priority can be calculated using the variance of the pixel values of each reference partial image. Thereby, as shown in FIG. 6, the block ID of the block to which the restriction range is applied is stored in the priority data 12 d in the storage unit 12. In the case of the figure, the block ID is stored in the priority data 12d in the order of the block 23a, the block 23c, the block 23d, and the block 23b shown in FIG. 4, and processing is performed in this order.

  The restriction range application unit 14 c is a processing unit that applies a restriction range to a block of interest in the similarity image 23. Specifically, the restriction range is set as a new search range. In this way, the search range is narrowed down.

  The constraint range propagation unit 14d is a processing unit that propagates the constraint range to adjacent blocks of the similarity image. The reason why the restriction range is propagated is that the search range may change, so that the assumption that “the difference between corresponding points is ± 1 pixel between adjacent blocks of the similarity image” may not be satisfied. Because. Specifically, the restriction range, that is, the search range is expanded by one pixel around it to generate a new restriction range, and the generated restriction range is propagated to adjacent blocks.

  The narrowing-down processing unit 14e is a processing unit that narrows down the search range based on the constraint range generated by the constraint range calculation unit 14a and the constraint range propagated from the adjacent block. Specifically, a range where the constraint range that has been propagated from a block adjacent to the search range already applied by the constraint range application unit 14c is calculated, and this is set as a new search range. If the similarity that is equal to or higher than the predetermined threshold, that is, the peak value is not within the new search range, the constraint range is recalculated in the new search range, and the priority is also set as necessary. Recalculate. Further, a new search range is expanded by one pixel to obtain a restriction range, and the obtained restriction range is propagated to adjacent blocks. The direction in which the propagation process is performed need only be the direction in which this block moves away from the first block. For example, when propagation of the restriction range is received from the right block, the restriction range may be propagated only to the left block. However, if the new search range is not changed compared to the original search range, the process ends without performing the propagation process. In this way, by repeating the constraint range propagation processing, the search range of surrounding blocks is also narrowed down.

  The corresponding point determination processing unit 15 is a processing unit that determines a corresponding point using the narrowed search range after narrowing down the search range. The final corresponding point is determined by changing the calculation method of the constraint range from the above method and processing again. Changing the calculation method of the constraint range means that if there is only one peak value, that point is the constraint range, and if there are multiple peaks, the point closest to the average of those coordinate values is the constraint range. . By making the restriction range indicate only one point, one corresponding point can be determined.

  Next, a processing concept of the search range narrowing processing unit 14 illustrated in FIG. 1 will be described. FIG. 7 is an explanatory diagram for explaining the concept of search range narrowing processing by the search range narrowing processing unit 14 shown in FIG. 1. Here, for convenience of explanation, it is assumed that only the four blocks 23a to 23d of the similarity image 23 are targeted and the search range is only in the horizontal direction.

  As shown in FIG. 5A, in the initial state, the search range of the four blocks 23a to 23d of the similarity image 23 is the entire range in the horizontal direction.

  Here, as shown in FIG. 7B, the restriction range is calculated with the block 23a as the first processing object in accordance with the priority data 12d shown in FIG. Considering the similarity of the block 23a that is equal to or higher than the threshold value, the restriction range 31 shown in the figure is obtained. For this reason, the restriction range 31 is applied as a new search range, and the restriction range 32 obtained by extending the search range by one pixel on the left and right is propagated to the adjacent block 23b. As a result, the illustrated constraint range 32 of the block 23b is specified, and the constraint range 32 is applied as a new search range.

  Thereafter, the restriction range 33 obtained by extending the search range by one pixel on the left and right is propagated to the adjacent block 23c, and the restriction range 33 is applied as a new search range of the block 23c. Similarly, the restriction range 34 obtained by extending the search range by one pixel on the left and right is propagated to the adjacent block 23d, and the restriction range 34 is applied as a new search range of the block 23d. Through the series of processes described above, the search ranges of the blocks 23a to 23d are narrowed down.

  Next, the restriction range is calculated for the block 23c as a processing target according to the priority data 12d shown in FIG. 6, and the restriction range 35 shown in the figure is obtained. Therefore, the restriction range 35 is applied as a new search range, and the restriction range 36 obtained by extending the search range by one pixel on the left and right is propagated to the adjacent blocks 23b and 23d. As a result, the illustrated restriction range 36 of the blocks 23b and 23d is specified, and the restriction range 36 is applied as a new search range. At this point, the search range of the block 23d is fixed as shown in the figure. This is because there is only one similarity satisfying the constraint ranges 34 and 36 among the similarities exceeding the predetermined threshold of the block 23d.

  Similarly, in accordance with the priority data 12d shown in FIG. 6, the restriction range is propagated while calculating the restriction range for the block 23d as the processing target, and the restriction range is propagated while calculating the restriction range for the block 23b as the processing target. By processing, the search range of each block can be narrowed down.

  Next, a processing procedure from the division of the reference image to the storage of the reference partial image by the corresponding point search device 1 shown in FIG. 1 will be described. FIG. 8 is a flowchart showing a processing procedure from division of a reference image to storage of a reference partial image by the corresponding point search apparatus 1 shown in FIG. Here, it is assumed that a registration mode for performing a reference image registration process is selected by a dip switch or the like (not shown).

  As shown in the figure, when a reference image is input from the image input unit 10 (step S101), the division processing unit 11 divides the reference image (step S102), and the result is stored in the storage unit 12 as a reference part. Stored as an image 21a (step S103). Specifically, the division processing unit 11 performs division processing on the reference image so that the distortion in each divided reference image can be accommodated in one pixel.

  Next, the processing procedure up to the restriction range propagation processing by the corresponding point search apparatus 1 shown in FIG. 1 will be described. FIG. 9 is a flowchart showing a processing procedure up to the restriction range propagation processing by the corresponding point search apparatus 1 shown in FIG. Here, it is assumed that a processing mode for searching for corresponding points is selected by a dip switch (not shown) or the like.

  As shown in the figure, when the image input unit 10 inputs an input image (step S201), the similarity image between the input image and the reference partial image 21a stored in the storage unit 12 by the similarity image generation processing unit 13 is displayed. Is generated (step S202).

  Thereafter, the search range narrowing processing unit 14 calculates the constraint range for each block of the similarity image (step S203), calculates the priority of each block (step S204), and prioritizes the block ID as a priority. It is registered in the degree data 12d (step S205).

  Then, the first block to be processed is specified from the priority data 12d in the storage unit 12 (step S206), and the search range is specified by applying the constraint range (step S207). Thereafter, one pixel is added around the search range to calculate the restriction range (step S208), and the calculated restriction range is propagated to adjacent blocks (step S209). The processes in steps S206 to S209 are repeated while there are processing target blocks in the priority data 12d (step S210; Yes). When there are no processing target blocks in the priority data 12d (step S210; No), The process ends.

  Next, the processing procedure for narrowing down the search range by the corresponding point search apparatus 1 shown in FIG. 1 will be described. FIG. 10 is a flowchart showing a processing procedure for narrowing down the search range by the corresponding point search apparatus 1 shown in FIG.

  As shown in the figure, the search range narrowing processing unit 14 performs a narrowing process on a certain block, and if a restriction range is propagated from another block adjacent to this block (step S301; Yes). ) The search range is updated by applying the constraint range propagated from other blocks (step S302).

  Thereafter, the restricted range obtained by extending one pixel around the search range is propagated to another adjacent block (step S303), and the restricted range is calculated again (step S304). Note that if the constraint range is not propagated from another block (step S301; No), the process proceeds to step S304 as it is.

  As a result, when the calculated constraint range has changed from the previous constraint range (step S305; Yes), the priority is calculated again (step S306), and the block ID is stored in the priority data 12d of the storage unit 12. If registered (step S307; Yes), the corresponding block ID is deleted from the priority data 12d (step S308), and the recalculated priority is registered in the priority data 12d (step S309). If the calculated constraint range has not changed from the previous constraint range (step S305; No), the process is terminated as it is. If the block ID is not registered in the priority data 12d of the storage unit 12 (step S307; No), the process proceeds to step S309.

  As described above, in the first embodiment, the reference partial image 21a obtained by dividing the reference image input by the image input unit 10 by the division processing unit 11 is stored in the storage unit 12, and the input image is input by the image input unit 10. , The similarity image generation processing unit 13 generates a similarity image between the input image and the reference partial image 21a, the search range narrowing processing unit 14 sequentially narrows down the search range, and the corresponding point determination processing unit 15 Since the corresponding points are determined using the search range narrowed down by the above, when matching between images including a periodic pattern, correct corresponding points can be acquired quickly and efficiently. .

  In the first embodiment, the reference image is divided into reference partial images to generate a similarity image with the input image. However, the present invention is not limited to this, and the input image is not limited thereto. Can be divided into input partial images to generate a similarity with the reference image.

  Further, in the first embodiment, the case where the vertical line pattern with periodicity is included in the image and the corresponding point in the horizontal axis direction is obtained has been described, but the horizontal line pattern with periodicity is similarly included in the image. Thus, the corresponding points in the vertical axis direction can also be obtained. For example, when the reference image 41 shown in FIG. 11A and the input image 42 shown in FIG. 11B exist, a similarity image 43 shown in FIG. By performing the same search range narrowing process using, the corresponding points in the vertical axis direction can be obtained. Note that corresponding points in both directions of the vertical axis and the horizontal axis can be obtained, and corresponding points including diagonal lines can also be obtained. In addition, by narrowing down the search range a plurality of times while stepwise tightening the threshold value at the time of calculation of the constraint range, it is also possible to perform association that is less likely to cause a period shift.

  Furthermore, in the first embodiment, the corresponding point is determined using the search range. However, the present invention is not limited to this, and when determining the corresponding point, Japanese Patent Laid-Open No. 2004-54682 is disclosed. Using the technique disclosed in the above, it is possible to extend so as to perform stable association by accumulating the similarity information of adjacent blocks. Specifically, (1) a similarity image indicating the similarity between the input partial image obtained by dividing the input image into blocks and the reference partial image obtained by dividing the reference image is created. (2) A threshold value based on the peak value is provided in the order of blocks having the highest peak value for blocks having a peak value equal to or greater than a certain value in the similarity image. The search range of the image is propagated to adjacent blocks, and the search range of each block is narrowed down. (3) In the corresponding point determination stage, each similarity image is updated while repetitively accumulating the pixel values within the narrowed search range in each similarity image in the vertical direction and the horizontal direction. (4) In the order of blocks having the highest peak value in the similarity image, only one point having the peak value is set as a search range and propagated to adjacent blocks. Finally, the search range becomes one point, and this position is determined as a corresponding point on the input partial image. In this way, by performing global optimization more than before, it becomes difficult to fall into the local minimum, and the precision of matching is improved by fine adjustment of optimization that repeats cumulative addition, and parallel distributed processing is possible Therefore, hardware implementation becomes easy.

  In the first embodiment, the case where an image is a processing target has been described. However, the present invention is not limited to this, and the present invention can also be applied to a case where a “waveform” is a processing target. Also in the image processing field, “waveform” association such as histogram association is useful. For this reason, in the present Example 2, the case where the corresponding point of an input waveform and a reference waveform is searched is shown.

  FIG. 12 is a functional block diagram illustrating the configuration of the corresponding point search apparatus 50 according to the second embodiment. The corresponding point search device 50 shown in the figure divides a reference waveform inputted in advance to generate a plurality of reference partial waveforms, and corresponds to the center position of each reference partial waveform when the input waveform is inputted. This is a device for searching for corresponding points on the input waveform. In addition, about the function part which performs the process similar to the function part shown in FIG. 1, the detailed description is abbreviate | omitted as attaching | subjecting the same code | symbol.

  A waveform input unit 51 illustrated in FIG. 12 is an input unit that inputs the reference waveform and the input waveform illustrated in FIG. The reference waveform and the input waveform include a periodic signal, and the horizontal axis indicates time and the vertical axis indicates a signal value. The reference partial waveform 52 is a waveform obtained by dividing the reference waveform by the division processing unit 11. The similarity graph generation processing unit 53 is a processing unit that calculates the similarity between the input waveform and the reference partial waveform, and generates a similarity graph with the similarity as the vertical axis, and is divided as illustrated. The corresponding points corresponding to each reference partial waveform are narrowed down from the search range of the input waveform.

  Next, calculation of the constraint range performed by the search range narrowing processing unit 14 will be described with reference to FIG. As shown in FIG. 13B, for each block 61 to 64 of the similarity graph, a range of coordinate positions where the similarity exceeds a predetermined threshold is set as a constraint range. For example, in the case of the block 61, since the two left peaks of the three similarity peaks exceed the threshold value, the range of these two peaks becomes the restriction range. Similarly, in the case of the block 62, since all three similarity peaks exceed the threshold value, the range of the three peaks is the restriction range.

  Next, the search range narrowing process performed by the search range narrowing processing unit 14 will be described with reference to FIG. Here, it is assumed that block IDs are stored in order of block 61, block 64, block 62, and block 63 in the priority data 12d.

  First, since the block 61 is the first processing target, the restriction range of the block 61 is applied to set the restriction range as a new search range. Then, the search range is expanded to calculate a new constraint range, and the calculated constraint range is propagated to the block 62. When the block 62 is processed, the search range is specified by applying the restriction range, the search range is expanded to calculate a new restriction range, and the calculated restriction range is propagated to the block 63. Similarly, when processing the block 63, the search range is specified by applying the restriction range, the search range is expanded to calculate a new restriction range, and the calculated restriction range is propagated to the block 64. .

  As a result, in block 64, since only the peak located in the middle of the three peaks is located in the new search range, the position of this peak is determined as the search range. Thereafter, similarly, the restriction range is sequentially propagated from the block 64 to the adjacent blocks. Thereby, in the block 62, since only the peak located in the middle of the three peaks is located in all the search ranges, the position of this peak is determined as the search range.

  Thereafter, when processing the block 62, the restriction range is propagated to the adjacent blocks 61 and 63, and thereby the restriction range of the blocks 61 and 63 is determined. In block 63, the propagation process is stopped because there is no change in the restriction range.

  As described above, according to the second embodiment, the reference partial waveform 52 obtained by dividing the reference waveform input by the waveform input unit 51 by the division processing unit 11 is stored in the storage unit 12. If an input waveform is input, the similarity graph generation processing unit 53 generates a similarity graph between the input waveform and the reference partial waveform 52, and the search range narrowing processing unit 14 sequentially narrows down the search range, and corresponding point determination processing Since the corresponding points are determined using the search range narrowed down by the unit 15, the correct corresponding points can be acquired quickly and efficiently when the correlation between waveforms including a periodic signal is performed. Can do.

  As described above, the corresponding point search method, the corresponding point search device, and the corresponding point search program according to the present invention are suitable for performing association between images including a periodic pattern.

DESCRIPTION OF SYMBOLS 1 Corresponding point search apparatus 10 Image input part 11 Division | segmentation process part 12 Memory | storage part 12b Search range data 12c Restriction range data 12d Priority data 13 Similarity image generation process part 14 Search range narrowing-down process part 14a Restriction range calculation part 14b Priority Calculation unit 14c Restriction range application unit 14d Restriction range propagation unit 14e Narrowing processing unit 15 Corresponding point determination processing unit 21 Reference image 21a Reference partial image 22 Input image 23 Similarity image

Claims (9)

A corresponding point search method in a corresponding point search device for searching for a corresponding point that associates a first image including a periodic pattern with a second image to be compared with the first image,
A similarity image generation step of generating a similarity image consisting of a plurality of blocks having pixel values as the similarity between the divided image consisting of a plurality of blocks obtained by dividing the first image and the second image;
Based on the similarity exceeding a predetermined threshold value among the similarities included in each block forming the similarity image generated by the similarity image generating step, search ranges where corresponding points can exist are calculated. Search range calculation process,
Propagation processing is performed for the adjacent range adjacent to the own block, and the propagation processing is performed from the adjacent block by adding the allowable amount of the corresponding point difference between the blocks to the search range calculated by the search range calculation step. Processing to propagate the restriction range obtained by adding the allowable amount of the difference between corresponding points between the blocks to the updated search range while updating the search range using the set restriction range. Repeatedly, a search range narrowing step for narrowing down the search range;
Corresponding points that specify the position of the corresponding point on the second image corresponding to the position of the block center of each block obtained by dividing the first image based on the search range narrowed down by the search range narrowing step A corresponding point search method characterized by including a specific step. Priority for calculating the priority for processing each block based on the similarity exceeding a predetermined threshold among the similarities included in each block forming the similarity image generated by the similarity image generation step A calculation step; and a storage step of storing the priority calculated in the priority calculation step in a priority storage unit,
The search range narrowing step includes
The corresponding point search method according to claim 1, further comprising: determining a processing order of a block that propagates the restriction range with respect to an adjacent block based on the priority stored in the priority storage unit. The similarity image generation step includes:
A division processing step of dividing the first image into a plurality of blocks;
The corresponding point search method according to claim 1, further comprising: a similarity calculation step of calculating a similarity between each block divided by the division processing step and the second image. . The division processing step divides the first image into a plurality of blocks having a size in which distortion in each block fits in one pixel,
The corresponding point search method according to claim 3, wherein the search range narrowing step uses a range obtained by adding one peripheral pixel to the search range calculated by the search range calculation step.   The similarity calculation step uses each block as a template, and the similarity obtained when the template is applied as a template to a first position shifted by a predetermined number of pixels from the center position of the second image. Is a pixel value of a pixel corresponding to the first position of the similarity image, and then the template is sequentially shifted and matched within a predetermined range of the second image to sequentially calculate the similarity. The corresponding point search method according to claim 3. The search range calculation step includes
A constraint range calculation step of calculating a constraint range by adding an allowable amount of corresponding points between blocks to the search range calculated by the search range calculation step;
A first propagation processing step for carrying out propagation processing on the restriction range calculated by the restriction range calculation step with respect to an adjacent block adjacent to the own block;
A search range update processing step of updating the search range using a constraint range that has been propagated from the adjacent block;
A second propagation processing step of performing propagation processing on a neighboring range of a restriction range obtained by adding an allowable amount of a difference between corresponding points between blocks to the search range updated by the search range update processing step. The corresponding point search method according to claim 1, wherein the corresponding point is searched for. A corresponding point search method in a corresponding point search device for searching for a corresponding point that associates a first waveform including a periodic pattern with a second waveform to be compared with the first waveform,
A similarity waveform generation step of generating a similarity waveform consisting of a plurality of blocks having pixel values of the similarity between the divided waveform consisting of a plurality of blocks obtained by dividing the first waveform and the second waveform;
Based on the similarity exceeding a predetermined threshold value among the similarities included in each block forming the similarity waveform generated by the similarity waveform generating step, search ranges where corresponding points can exist are calculated. Search range calculation process,
Propagation processing is performed for the adjacent range adjacent to the own block, and the propagation processing is performed from the adjacent block by adding the allowable amount of the corresponding point difference between the blocks to the search range calculated by the search range calculation step. Processing to propagate the restriction range obtained by adding the allowable amount of the difference between corresponding points between the blocks to the updated search range while updating the search range using the set restriction range. Repeatedly, a search range narrowing step for narrowing down the search range;
Corresponding point that specifies the position of the corresponding point on the second waveform corresponding to the position of the block center of each block obtained by dividing the first waveform based on the search range narrowed down by the search range narrowing step A corresponding point search method characterized by including a specific step. A corresponding point search device that searches for a corresponding point that associates a first image including a periodic pattern with a second image to be compared with the first image,
Storage means for storing a divided image composed of a plurality of blocks obtained by dividing the first image;
Similarity image generation means for generating a similarity image composed of a plurality of blocks having the similarity between the divided image stored in the storage means and the second image as pixel values;
Based on the similarity exceeding a predetermined threshold value among the similarities included in each block forming the similarity image generated by the similarity image generating means, a search range where a corresponding point can exist is calculated. Search range calculation means,
Propagation processing is performed on the adjacent block adjacent to the own block, and the propagation processing is performed from the adjacent block by adding the allowable amount of the difference between the corresponding points between the blocks to the search range calculated by the search range calculation means Processing to propagate the restriction range obtained by adding the allowable amount of the difference between corresponding points between the blocks to the updated search range while updating the search range using the set restriction range. Repetitively, a search range narrowing means for narrowing down the search range;
Corresponding points that specify the position of the corresponding point on the second image corresponding to the position of the block center of each block obtained by dividing the first image based on the search range narrowed down by the search range narrowing means A corresponding point search device comprising: a specifying unit. A corresponding point search program that is executed by a corresponding point search device that searches for corresponding points that associate a first image including a periodic pattern with a second image to be compared with the first image. ,
A similarity image generation procedure for generating a similarity image composed of a plurality of blocks having pixel values as similarities between a divided image composed of a plurality of blocks obtained by dividing the first image and the second image;
Based on the similarity exceeding a predetermined threshold among the similarities included in each block forming the similarity image generated by the similarity image generation procedure, search ranges where corresponding points can exist are calculated. Search range calculation procedure;
Propagation processing is performed for the adjacent block adjacent to the own block, and the propagation processing is performed from the adjacent block by adding a tolerance of the difference between corresponding points between blocks to the search range calculated by the search range calculation procedure. Processing to propagate the restriction range obtained by adding the allowable amount of the difference between corresponding points between the blocks to the updated search range while updating the search range using the set restriction range. Repeat the search range narrowing procedure for narrowing down the search range,
Corresponding point that specifies the position of the corresponding point on the second image corresponding to the position of the block center of each block obtained by dividing the first image based on the search range narrowed down by the search range narrowing procedure Corresponding point search program that causes a computer to execute a specific procedure.

Images