JP2009044612A - Image processing apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately join a plurality of image data blocks forming original image data. <P>SOLUTION: An overlap area generated when an original image concerned with an acquired image data block and a reference image are overlapped to each other by using a geometric transformation formula T is extracted (S502). Feature points in the overlap area between the original image and the reference image are extracted (S503, S505). Correspondence points corresponding to respective feature points on the overlap area between the original image and the reference image are extracted (S504, S506) by using the geometric transformation formula T. The geometric transformation formula T is generated so that a degree of coincidence between the plurality of feature points and the plurality of correspondence points corresponding to the feature points becomes highest (S507). The geometric transformation formula T is repeatedly corrected while gradually increasing resolution R until the resolution R becomes maximum processing resolution (S508, S509). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and a program for generating original image data by joining a plurality of image data blocks.

  In order to convert a large original into digital data, a moving scanner is used to read the original in a plurality of times, and each image data block generated when the original is read is joined in a later process to generate the original of the original. Image data is generated. In this post-process, the operator can operate the general-purpose image editing software to join the image data blocks. However, the work is complicated and time-consuming, and sufficient image quality can be obtained. There are things that cannot be done. Thus, a technique for automatically and accurately joining image data blocks is known. For example, a manuscript is placed on a transparent plate on which a mark indicating position information is formed, and the arrangement position of each image data block is determined based on the mark written on the transparent plate, and read based on the calculated arrangement position. An image reading apparatus that automatically joins image data blocks is known (see Patent Document 1). In the image reading apparatus disclosed in Patent Document 1, it is necessary to install a transmission plate every time a new document is scanned. When a document to be scanned is large, the transmission plate is also large, and it is troublesome to install the transmission plate. It is complicated. Furthermore, it is difficult to disassemble and carry the transmission plate. Therefore, an image processing apparatus is known that performs pattern matching on image data blocks to be joined to extract matching points, and joins the image data blocks using the extracted matching points (see, for example, Patent Document 2). ).

JP-A-5-252349 (FIG. 4) Japanese Patent Laying-Open No. 2003-189111 (FIG. 1)

  In the image processing apparatus disclosed in Patent Document 2, it is possible to join image data blocks without installing a transparent plate. However, when the original is read in a plurality of times while moving the scanner, the posture of the scanner with respect to the original may slightly change. Since such a change becomes a disturbance when extracting matching points in pattern matching, it is difficult to accurately join image data blocks.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus and a program capable of accurately joining a plurality of image data blocks forming original image data.

  The image processing apparatus according to the present invention includes an image storage unit that stores a plurality of image data blocks that form one original image data by being arranged so as to partially overlap each other in one direction, and the image storage unit stores For each of the two image data blocks overlapping each other, a rough alignment means for aligning an original image that is one of the image data blocks and a reference image that is the other image data block; Region extracting means for extracting regions overlapping each other in the image and the reference image, feature point extracting means for extracting one or a plurality of feature points for each region of the original image and the reference image, and the feature point extracting means For each area from which feature points have been extracted, corresponding to each feature point of the area, Corresponding point extracting means for extracting the corresponding points by performing a normalized correlation with reference to the coordinate position of the feature point, and a plurality of the feature points in each region and a plurality of corresponding points corresponding thereto Calculating a geometric transformation formula for moving the original image so that the degree is the highest, and moving the original image based on the calculated geometric transformation formula, thereby aligning the original image with the reference image Detailed positioning means for performing the above and joining means for joining the original image and the reference image aligned by the detailed positioning means.

  The program of the present invention stores image storage means for storing a plurality of image data blocks forming one original image data by being arranged so as to partially overlap each other in one direction, and the image storage means stores the image storage means. For each of the two image data blocks that overlap each other, a rough alignment means for aligning an original image that is one of the image data blocks and a reference image that is the other image data block, the original image and the reference Area extraction means for extracting overlapping areas in each of the images, feature point extraction means for extracting one or a plurality of feature points for each area of the original image and the reference image, and feature point extraction means for extracting feature points For each region, corresponding points in the other region corresponding to each feature point of the region and overlapping the region Corresponding point extracting means for extracting by performing normalized correlation with reference to the coordinate position of the feature point, the plurality of feature points in each region, and the matching degree of the corresponding points corresponding thereto are the highest. As described above, a detailed alignment for calculating the geometric transformation equation for moving the original image and performing alignment between the original image and the reference image by moving the original image based on the calculated geometric transformation equation And a computer functioning as a joining unit that joins the original image and the reference image aligned by the detailed positioning unit.

  According to the present invention, for each of the original image area and the reference image area, feature points and corresponding points corresponding thereto are extracted, and the extracted feature points and corresponding points corresponding thereto are matched. Since the geometric transformation formula is calculated so that the degree becomes the highest, a robust geometric transformation formula can be obtained, and a plurality of image data blocks can be accurately joined.

  In the present invention, the approximate alignment means can obtain a phase correlation between the vicinity of one end of the original image in the one direction and the vicinity of the other end of the reference image in the one direction. In addition, it is preferable to perform alignment of the original image and the reference image by calculating a geometric transformation formula for moving the original image and moving the original image based on the calculated geometric transformation formula. According to this, since the geometric transformation formula is calculated by using the phase correlation method in the rough alignment, the geometric transformation formula is calculated more quickly than when the geometric transformation formula is calculated only by the degree of coincidence of the normalized correlation. be able to.

  At this time, when the approximate alignment means cannot obtain a phase correlation between the vicinity of the one end of the original image and the vicinity of the other end of the reference image, the original image and More preferably, a geometric transformation equation for moving the original image is calculated so that a phase correlation can be obtained between the entire reference images. According to this, even when the majority of the image data blocks overlap, the geometric transformation formula can be calculated quickly.

  According to the present invention, there is further provided a multiresolution means for generating the image data block obtained by reducing the resolution of each image data block stored in the image storage means at a plurality of resolutions and storing the generated image data block in the image storage means. The region extracting unit extracts each region relating to the original image and the reference image aligned by the detailed alignment unit in the order of increasing resolution from the lowest resolution stored in the image storage unit; Preferably, the joining unit joins the original image and the reference image, which are aligned by the detailed positioning unit with respect to the highest resolution stored in the image storage unit. According to this, since the geometric transformation formula is corrected in the order from the lowest resolution to the highest resolution, a robust geometric transformation formula can be calculated.

  Furthermore, the present invention further includes a dividing unit that divides each image data block stored in the image storage unit into processed images having a predetermined size, and the joining unit is divided by the dividing unit. It is preferable that the original image and the reference image are joined in units of the processed image. According to this, it is possible to deal with a high-resolution image data block that cannot be developed in a memory.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram of a control apparatus which is an image processing apparatus according to a preferred embodiment of the present invention. As shown in FIG. 1, the control device 1 joins a plurality of image data blocks generated when the scanner 2 divides and reads a large document and generates joined image data as original image data. This is realized by a control program activated on a PC (personal computer). The PC includes a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and the like, and the control device is executed by the CPU executing a control program. 1 will be described later. The control device 1 is connected to the scanner 2 so as to be communicable. Further, the control device 1 is connected to a display 20 for confirming the contents of the image data block and the joined image data, and an input device such as a mouse and a keyboard (not shown).

  The control device 1 includes an image storage unit (image storage unit) 11, a pre-processing unit 12, a schematic registration unit (general registration unit) 13, an overlapping region extraction unit (region extraction unit) 14, and feature point extraction. Part (feature point extracting means) 15, corresponding point extracting part (corresponding point extracting means) 16, detailed positioning part (detailed positioning means) 17, global geometric transformation expression generating part 18, and joining part (joining means) 19). The image storage unit 11 stores image data blocks related to a plurality of input images related to the original read by the scanner 2. A document reading method of the scanner 2 will be described with reference to FIG. FIG. 2 is a diagram showing an operation when the scanner 2 reads a document. The scanner 2 can move in one direction. As shown in FIG. 2, the scanner 2 reads a part of the document at each of a plurality of positions in the longitudinal direction of the document, generates an image data block, and transmits the generated image data block to the control device 1. In this way, the scanner 2 generates image data blocks related to a plurality of input images by reading a document divided into a plurality of parts. At this time, the plurality of image data blocks are generated such that a part (5 to 10 cm in the present embodiment) overlaps with each other in the moving direction (one direction). Accordingly, the input images indicated by the plurality of image data blocks are arranged so as to partially overlap each other with respect to the moving direction, thereby forming the original image indicated by the original image data. In the present embodiment, the resolution of the image data block is 600 dpi.

  The pre-processing unit 12 performs pre-processing of each image data block stored in the image storage unit 11 when generating the original image data, and includes a multi-resolution unit (multi-resolution unit) 12a and a dividing unit. (Dividing means) 12b. The multi-resolution conversion unit 12a reduces the resolution of each image data block stored in the image storage unit 11 at a plurality of resolutions. Specifically, the multi-resolution converting unit 12a cumulatively multiplies the image data block by 1 / (1/2 power) until the image data block is equal to or smaller than the size of the divided image data block described later. For example, in the present embodiment, since the resolution of the image data block is 600 dpi, the image data blocks with reduced resolution of 600 dpi → 424 dpi → 300 dpi → 212 dpi →.

  The dividing unit 12b is configured to divide the image data block whose resolution has been reduced by the multi-resolution converting unit 12a into tiles using divided image data blocks (processed images) of a predetermined size and store them in the image storage unit 11. The divided image data block is composed of 256 × 256 dots. In the divided image data block, the value of all dots is 0 for the area exceeding the size of the image data block whose resolution has been reduced. Further, the divided image data block is output in the JPEG format and is encrypted by a common key encryption method in order to improve the security function. The divided image data block may be output in another format such as a bitmap, may be encrypted by another encryption method, or may not be encrypted.

  The rough alignment unit 13 uses, as an original image, an image indicated by one of two image data blocks that overlap each other in the original image data among the plurality of image data blocks stored in the image storage unit 11, and the other image data block When the image indicated by is used as a reference image, an approximate geometric transformation formula for moving the original image with respect to the reference image is generated so that the original image and the reference image are aligned.

  The approximate alignment unit 13 includes a phase correlation geometric transformation expression generation unit 13a. The phase correlation geometric transformation expression generation unit 13a first includes an edge portion on the reference image side in the original image (near one end portion in one direction) and an edge portion on the original image side in the reference image (near the other end portion in one direction). Are resampled at a coarse resolution (20 dpi in this embodiment). Here, resampling is a resolution conversion from a reduced-resolution image data block having a resolution closest to the coarse resolution among the image data blocks stored in the image storage unit 11, and performs the resolution conversion. Acquisition of a desired area from the obtained image data block. Then, by using the phase correlation method, the edge portion of the reference image related to the edge portion of the original image that can obtain the phase correlation between the edge portion of the original image and the edge portion of the reference image Is calculated, and a rough geometric transformation formula is generated from the calculated parallel movement amount. Thereafter, a normalized correlation value indicating the degree of coincidence between the edge portion of the original image and the reference image when the edge portion of the original image is translated based on the generated approximate geometric transformation formula is calculated. Here, when the normalized correlation value is not equal to or greater than a predetermined value (for example, 0.8 to 1), it is determined that the original image and the reference image overlap deeper than the edge portion, and the original image Resample the entire image and the entire reference image with coarse resolution, and again calculate the amount of translation relative to the reference image related to the original image using the phase correlation method, and generate an approximate geometric transformation formula from the calculated amount of translation To do.

  The overlapping area extraction unit 14 acquires an image data block indicating the original image and the reference image from the image storage unit 11, and the approximate geometric conversion formula or detailed registration generated by the approximate registration unit 13 in the acquired image data block When the original image and the reference image are overlapped using the geometric conversion expression T, which is a detailed geometric conversion expression (described later) generated by the unit 17, an overlapping area that overlaps the original image and the reference image is extracted. is there. This overlapping area is rectangular.

  The feature point extraction unit 15 uses a Moravec interest operator to extract feature points in each overlap region of the original image and the reference image extracted by the overlap region extraction unit 14. In the present embodiment, the feature points are dots that have a large change in luminance with respect to other dots arranged in the vicinity. Specifically, the feature point extraction unit 15 converts the original image and the reference image to gray scale. Then, Moravec's interest operator is applied to the image converted to grayscale, and the average value and standard deviation of the interest values are obtained. Then, when each interest value is three times or more the standard deviation larger than the average value and there is no other feature point within a certain range, a dot related to the interest value is extracted as a feature point. Note that feature points may be extracted by other methods.

  The corresponding point extraction unit 16 extracts, for each overlapping region from which the feature point extracting unit 15 has extracted the feature points, corresponding points in other overlapping regions that overlap the overlapping region corresponding to the feature points of the overlapping region. It is. Specifically, first, a feature region that is a peripheral region of ± 3 pixels (7 × 7 pixels) around each feature point in the overlapping region of the original image is extracted. Further, by applying the geometric transformation equation T to each feature point, a search reference point that is a point on the overlapping region of the reference image is calculated, and ± 10 pixels centering on each search reference point in the overlapping region of the reference image A search area that is a peripheral area of (21 × 21 pixels) is extracted. Then, using the normalized correlation method, an area having a high degree of coincidence with the feature area in the search area is searched. As a result of the search, the center point of the discovered area is extracted as a corresponding point corresponding to the feature point. Next, similar processing is performed for each feature point in the overlapping region of the reference image, and corresponding points on the overlapping region of the original image corresponding to each feature point are extracted. Note that a search reference point, which is a point on the overlapping area of the source image, is calculated by applying inverse transformation of the geometric transformation equation T to each feature point related to the overlapping area of the reference image. A set of extracted feature points and corresponding points corresponding to the feature points is stored as a feature point list.

  The detailed alignment unit 17 selects the original image with respect to the reference image so that the degree of coincidence of the plurality of feature points extracted by the feature point extraction unit 15 in each overlapping region and the corresponding points corresponding thereto is the highest. A detailed geometric conversion formula to be moved is generated. Here, the detailed geometric transformation equation is robustly estimated by using a method in which the RANSAC (Random Sample Consensus) method, the least median method, and the least square method are mixed with the feature point list generated by the corresponding point extraction unit 16. Generate. Note that Helmat transform is used for geometric transformation to be applied. Specifically, two sets of feature points and corresponding points corresponding to the feature points are selected at random from the feature point list, and simultaneous equations are solved for these sets to generate a Helmat transform equation. Then, a square error of a set of feature points and corresponding points corresponding thereto is obtained for this Helmart transform equation. Further, a median (median value) relating to the square error is obtained, a set of feature points having a square error larger than that of the median and corresponding points corresponding thereto is excluded, and the Helmart transform formula is generated again by the least square method. The above process is repeated a predetermined number of times (100 times in the present embodiment), and the median is minimized. A conversion formula.

  The global geometric transformation expression generation unit 18 generates a global geometric transformation expression for moving each image data block stored in the image storage unit 11 so that original image data is generated. For example, the input image a to the input image d indicated by the image data block stored in the image storage unit 11 are arranged in order with respect to the moving direction of the scanner 2, and a plurality of detailed geometries generated by the detailed alignment unit 17. When the transformation formula (geometric transformation formula for moving the original image) is Ta to Td corresponding to the input image a to the input image d, the global geometric transformation formula Tgb related to the input image b is Tgb = Ta · Tb. The global geometric transformation equation Tgc related to the input image c is Tgb = Ta · Tb · Tc, and the global geometric transformation equation Tgd related to the input image d is Tgd = Ta · Tb · Tc · Td. Since the input image a is not an original image, Ta = Tga = identity conversion formula.

  The joining unit 19 joins each image data block and generates joined image data as original image data by a global geometric transformation formula for each image data block generated by the global geometric transformation formula generation unit 18. In addition, the joining unit 19 smoothes the joining part by performing multi-resolution spline composition when joining the image data blocks. The multi-resolution spline composition will be described later.

  Next, the operation of the control device 1, particularly the image processing for generating the original image data by joining the image data blocks read by the scanner 2 will be described with reference to FIG. 3. FIG. 3 is a flowchart showing the operation of the control device 1. As shown in FIG. 3, when image processing for generating original image data is started, the process proceeds to step S <b> 101 (hereinafter referred to as S <b> 101. The same applies to other steps), and the preprocessing unit 12 is moved to the image storage unit 11. Performs a multi-resolution image generation process which is a pre-process of each image data block stored in the memory. Then, the process proceeds to S <b> 102, and an image alignment process that is an alignment process of a plurality of image data blocks stored in the image storage unit 11 is performed. Further, the process proceeds to S103, and among the image data blocks that have been aligned by the image alignment process, image data blocks that are adjacent to each other are subjected to image joining processing to generate original image data, and the flowchart of FIG. finish. Hereinafter, detailed contents of the multi-resolution image generation process, the image registration process, and the image joining process will be described in order.

  The multi-resolution image generation process will be described with reference to FIG. FIG. 4 is a flowchart showing the operation of multi-resolution image generation processing. As shown in FIG. 4, when the multi-resolution image generation process is started, the process proceeds to S201, and a scale indicating the resolution to be generated is initialized to 1.0. Thereafter, the process proceeds to S202, and each image data block stored in the image storage unit 11 is sequentially read into the work area on the RAM. Then, the process proceeds to S203, and the multi-resolution converting unit 12a scales each read image data block based on the scale value. As will be described later, each time scaling is performed, the scale value is multiplied by 1 / (1/2) from the initial value 1.0. That is, the resolution is reduced by scaling. In step S204, the dividing unit 12b divides the image data block whose resolution has been reduced by the multi-resolution converting unit 12a into tiles using divided image data blocks of a predetermined size and stores them in the image storage unit 11. As described above, the divided image data block is composed of 256 × 256 dots. And in the area | region exceeding the size of an image data block in a divided image data block, the value of all the dots is 0. The divided image data block is output in the JPEG format encrypted by the common key encryption method.

  Then, the process proceeds to S205, where the dividing unit 12b determines whether the number of divided divided image data blocks is 1 or less. If the number of divided divided image data blocks is not 1 or less (S205: NO), the process proceeds to S206, the scale is multiplied by 1 / (1/2 power), the process proceeds to S202 again, and the above-described processing is repeated. . As a result, image data blocks of 600 dpi → 424 dpi → 300 dpi → 212 dpi →... Are generated in order, and divided image data blocks related to the respective image data blocks are generated. On the other hand, when the number of divided divided image data blocks is 1 or less (S205: YES), the flowchart of FIG. 4 ends.

  The image alignment process will be described with reference to FIG. FIG. 5 is a flowchart showing the operation of the image alignment process. As shown in FIG. 5, when the image alignment process is started, the process proceeds to S <b> 301, and the approximate alignment unit 13 converts the original image data among the plurality of image data blocks stored in the image storage unit 11. In forming, two image data blocks that overlap each other are selected as an original image and a reference image. Then, the process proceeds to S <b> 302, and the approximate alignment unit 13 performs approximate alignment processing on the selected original image and reference image. Thereafter, the process proceeds to S303, and based on the result of the alignment by the approximate alignment process, the detailed alignment process for the original image and the reference image is performed. In step S304, it is determined whether there is another combination of two image data blocks that overlap each other. If there is another combination (S304: YES), the process proceeds to S301 again, selects another combination, and repeats the above-described processing.

  On the other hand, if there is no other combination (S304: NO), the process proceeds to S305, where the joining unit 19 joins adjacent image data blocks based on the result of the alignment by the detailed alignment process. Image joining processing for generating original image data is performed. Then, the flowchart of FIG. 5 ends. Hereinafter, the detailed contents of the approximate alignment process, the detailed alignment process, and the image joining process will be described in order.

  First, the schematic positioning process will be described in detail with reference to FIG. FIG. 6 is a flowchart showing the operation of the rough alignment process. As shown in FIG. 6, when the rough alignment process is started, the process proceeds to S401, where the phase correlation geometric transformation expression generation unit 13a of the rough alignment unit 13 performs the reference image side of the previously selected original image. The edge portion and the edge portion on the original image side of the reference image are resampled at 20 dpi. As described above, at this time, the phase correlation geometric transformation expression generation unit 13a performs resolution conversion from the image data block having the resolution closest to 20 dpi among the image data blocks stored in the image storage unit 11, and the resolution Resampling is performed by obtaining a desired region from the converted image data block. Then, the process proceeds to S402, where the phase correlation geometric transformation expression generation unit 13a calculates the parallel movement amount with respect to the edge portion of the reference image related to the edge portion of the original image using the phase correlation method, Generate approximate geometric transformation. Thereafter, the process proceeds to S403, where the edge portion of the original image is translated based on the generated geometric transformation formula, and the normalized correlation value of the edge portion of the original image and the reference image is calculated. Then, the process proceeds to S404, and it is determined whether or not the calculated normalized correlation value is a predetermined value (for example, 0.8 to 1) or more. If it is determined that the normalized correlation value is equal to or greater than the predetermined value (S404: YES), the flowchart of FIG. 6 is terminated.

  On the other hand, when it is determined that the normalized correlation value is not equal to or greater than the predetermined value (S404: NO), it is determined that the original image and the reference image overlap deeper than the edge portion, and the process proceeds to S405, where the entire original image is processed. And the entire reference image are resampled at a resolution of 20 dpi or less. Then, the process proceeds to S406, and again using the phase correlation method, the amount of translation with respect to the reference image related to the original image is calculated, and an approximate geometric conversion formula is generated from the calculated amount of translation.

  Next, the detailed alignment process will be described in detail with reference to FIG. FIG. 7 is a flowchart showing the operation of the detailed alignment process. As shown in FIG. 7, when the detailed alignment process is started, the process proceeds to S501, and the value of the resolution R indicating the resolution of the image data block to be processed is obtained by the multi-resolution converting unit 12a of the preprocessing unit 12. The generated image data block is initialized to the lowest resolution, and the geometric transformation formula T is initialized to the approximate geometric transformation formula generated in the rough alignment process.

  In step S502, the overlapping area extraction unit 14 acquires an image data block having a resolution R indicating the original image and the reference image from the image storage unit 11, and geometrically converts the original image and the reference image related to the acquired image data block. When overlapping is performed using Expression T, overlapping regions that overlap each other in the original image and the reference image are extracted. In step S503, the feature point extraction unit 15 extracts a feature point in the overlapping region of the original image using a Moravec interest operator. In step S504, the corresponding point extraction unit 16 extracts a corresponding point in the overlapping area of the reference image corresponding to each feature point in the overlapping area of the original image using the geometric transformation formula T. In step S505, the feature point extraction unit 15 extracts a feature point in the overlapping region of the reference image using a Moravec interest operator. In step S506, the corresponding point extraction unit 16 extracts the corresponding points in the overlapping area of the original image corresponding to the feature points in the overlapping area of the reference image by using the inverse transformation of the geometric transformation formula T. As described above, at this time, a set of extracted feature points and corresponding points corresponding to the feature points is stored as a feature point list.

  Thereafter, the process proceeds to S507, where the detailed alignment unit 17 uses a method in which the RANSAC method, the least median method, and the least square method are mixed with the feature point list generated by the corresponding point extraction unit 16, and the detailed geometry is used. A conversion equation T is generated by robust estimation. In other words, the detailed geometric transformation formula for moving the original image with respect to the reference image so that the degree of coincidence of the plurality of feature points in each overlapping region of the original image and the reference image and the corresponding points corresponding thereto is the highest. Is generated. Then, the generated detailed geometric transformation formula is defined as the following geometric transformation formula T.

  Then, the process proceeds to S508, where it is determined whether or not the value of the resolution R is a predetermined maximum processing resolution. When the value of the resolution R is the maximum processing resolution (S508: YES), the current geometric transformation formula T is set as the detailed geometric transformation formula, and the flowchart of FIG. On the other hand, when the value of resolution R is not the maximum processing resolution (S508: NO), the next largest resolution is registered as the value of resolution R, the process proceeds to S502, and the above-described processing is repeated. As described above, in the detailed alignment process, while the resolution R is increased stepwise, the overlapping region extraction unit 14 extracts the overlapping region related to the original image and the reference image moved by the detailed geometric transformation formula, and the overlapping region is extracted. The geometric transformation formula T is corrected from the feature points of the region and the corresponding points corresponding to the feature points.

Further, the image joining process will be described in detail with reference to FIGS. FIG. 8 is a flowchart showing the operation of the image joining process. FIG. 9 is a block diagram showing the procedure for constructing the input image pyramid. FIG. 10 is a block diagram showing the procedure for constructing the blend image pyramid. FIG. 11 is a block diagram showing the procedure for constructing the output image pyramid. FIG. 12 is a block diagram showing a procedure for generating original image data. 9 to 12, it is assumed that image data blocks I _i (i = 0 to M) related to M input images are stored in the image storage unit 11. As shown in FIG. 8, when the image joining process is started, the process proceeds to S <b> 601, and image data blocks I _i related to M input images are sequentially read out to the image storage unit 11. The global geometric transformation formula T relating to the image data block I _i is applied.

Thereafter, the joint portion between the image data blocks I _i is smoothed by multi-resolution spline synthesis. First, the process proceeds to S602, and an input image pyramid by Burt & Adelson is constructed for each image data block I _i . The input image pyramid represents the image data block I _i with N + 1 levels (0 to N) having different resolutions. It is assumed that the resolution decreases in order from level 0 to level N. Specifically, as shown in FIG. 9, the image data blocks I _i according to the detailed geometric transformation T, copied as Gaussian image IG _{i, 0} level 0 of the input image pyramid. Next, the REDUCE operation (reduction in resolution) is repeatedly executed to create Gaussian images IG _{i, 1 to} IG _{i, N} of each level 0 to _N of the input image pyramid. Next, an EXPAND operation (higher resolution) is applied to the Gaussian image IG _{i, 1 to} IG _{i, N} of one level above, and a Gaussian image IG _{i, 0 to} IG _{i, N-1 of} that level. Are taken to create Laplacian images IG _{i, 0 to} IG _{i, N-1} at each level. However, for the highest level N, the Gaussian image IG _{i, N} is used as the Laplacian image IL _{i, N} as it is.

  In constructing the input image pyramid, it is necessary to perform various operations on each image data block. In this embodiment, since the image data block has a large capacity, the entire image data block is stored in the RAM. Unable to expand into the work area. Therefore, the divided image data block generated in the dividing unit 12b is managed by the memory cache. Specifically, only the divided image data blocks necessary for the processing are executed, and the divided image data blocks that have been calculated are stored in a memory cache managed by the LRU (Last Recently Used) method, and the divided image data blocks are stored. When the image data block is needed again later, the divided image data block is searched in the memory cache, and when found, the divided image data block is taken out from the memory cache and reused.

In step S603, a blend mask B _i is created for each image data block I _i . First, the following weighting function is set.
w _i = w (ξ, η) = − max (abs (ξ), abs (η))
w _i : Weighting value for the i-th input image.
ξ, η: Normalized coordinates of the i-th input image.
max: Maximum value function.
abs: Absolute value function.
The normalized coordinates ξ and η use the following equations.
ξ = 2 (x / W−1)
η = 2 (y / H-1)
x, y: coordinates after applying the global geometric transformation formula of the i-th input image.
W, H: width and height of the input image.
The above w _i is calculated for each pixel of each input image (1 ≦ i ≦ M). Then, to produce a blend of the pixel mask B _i a ₍₁ ≦ i ≦ M) as follows. If w _i ≧ w _j holds for all 1 ≦ j ≦ M, the pixel value of the blend mask B _i is set to 1, and if 1 ≦ j ≦ M satisfying w _i <w _j exists, blending The pixel value of the mask B _i is set to 0. However, since this does not satisfy the Burt & Adelson blend mask standard (it is necessary to have a value of 50% at the joint), expansion processing for one pixel is performed, and at the joint, the blend mask B _{i of} two adjacent images is used. Both pixel values are set to 1. Since w _i takes a larger value as it is closer to the center of the input image, it can be used as a comparison function for extracting a neighboring region of the input image, and is suitable for blend mask creation.

In step S604, a blend image pyramid is constructed for each image data block I _i using the blend mask B _i . Specifically, as shown in FIG. 10, the blend mask B _i to which the detailed geometric transformation T is applied is copied as a Gaussian image BG _{i, 0} of level 0 of the blend image pyramid. Then, by performing an iterative manner REDUCE operation, creating Gaussian image _BG i for each level 0~N blend image _pyramid, 0 ~BG _i, the _N.

Then, the process proceeds to S605, and an output image pyramid is constructed. Specifically, as shown in FIG. 11, for image data blocks I _{1 to} I _M , the Laplacian image of the input image pyramid with the pixel values of the Gaussian images BG _{i, 0 to} BG _{i, N} of the blend image pyramid as a ratio. The pixel values of LG _{i, 0 to} LG _{i, N} are multiplied and the result of multiplication is added to create a Laplacian image OL _i . By performing this operation for all levels _{0 to N} , Laplacian images OL _{0 to} OL _N of the output image pyramid are constructed. Note that FIG. 11 shows the processing relating to the image data block _I 1 (i = 1) only.

Thereafter, the process proceeds to S606, where the joint image data O is output by developing the output image pyramid. Specifically, as shown in FIG. 12, for each level 0 to _N of the output image pyramid, the EXPAND operation is performed on the Gaussian images OG _{1 to} OG _{N one} level higher, and the Laplacian image of the level By taking the sum of OL _{0 to} OL _N−1 , Gaussian images OG _{0 to} OG _N of levels _{0 to N} are created. However, for the highest level N, the Laplacian image OL _N is directly used as the Gaussian image OG _N. Finally, the output image pyramid level 0 Gaussian image OG ₀ is output as the joined image data O. This joint image data O becomes original image data. Then, the flowchart of FIG. 8 ends.

  In the calculation of each image pyramid in the above-described multiple spline synthesis, a single precision floating point is used as a data type representing a pixel value. This is because the pixel value of a Laplacian image generally has both positive and negative signs, and in order to express the results of the REDUCE operation and EXPAND operation with sufficient accuracy, a real value is generally required.

  As described above, according to the present embodiment described above, feature points and corresponding points corresponding thereto are extracted for each of the original image region and the reference image region, and a plurality of extracted feature points and a plurality of corresponding points are extracted. Since the detailed geometric conversion formula is calculated so that the matching degree of the corresponding points is the highest, it is possible to obtain a robust detailed geometric conversion formula, and the image data blocks can be accurately joined. This improves the accuracy of the original image data.

  Since the rough alignment unit 13 calculates a parallel movement amount with respect to the edge portion of the reference image related to the edge portion of the original image using the phase correlation method, and generates an approximate geometric transformation expression from the calculated parallel movement amount, In the alignment process, the approximate geometric transformation expression can be quickly generated as compared with the case where the approximate geometric transformation expression is generated only by the matching by the normalized correlation.

  At this time, when the approximate alignment unit 13 translates the edges of the original image based on the generated approximate geometric transformation formula, the normalized correlation values of the edge portions of the original image and the reference image become a predetermined value or more. If not, the phase correlation method is used again to calculate the amount of translation of the entire original image relative to the entire reference image, and an approximate geometric transformation equation is generated from the calculated amount of translation. For this reason, even when the original image and the reference image are deeply overlapped, an approximate geometric conversion formula can be quickly generated.

  In addition, the multi-resolution converting unit 12a generates an image data block obtained by reducing the resolution of each image data block at a plurality of resolutions. In the detailed alignment process, the overlapping area extraction unit 14 extracts the overlapping area related to the original image and the reference image moved by the detailed geometric transformation formula while increasing the resolution step by step, and features of the overlapping area Since the geometric transformation equation T is corrected from the points and the corresponding points corresponding to the feature points, a more robust detailed geometric transformation equation can be generated.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments, and various modifications can be made as long as they are described in the claims. For example, in the above-described embodiment, the approximate alignment unit 13 calculates a translation amount with respect to the edge portion of the reference image related to the edge portion of the original image using the phase correlation method, and approximates from the calculated translation amount. Although it is the structure which produces | generates a geometric transformation type | formula, the structure which calculates the parallel displacement amount with respect to the edge part of the reference image which concerns on the edge part of an original image by the normalization correlation method without using a phase correlation method may be sufficient.

  In the above-described embodiment, the normalized correlation value of the edge portions of the original image and the reference image when the approximate alignment unit 13 translates the edges of the original image based on the generated approximate geometric transformation formula. In the case where is not equal to or greater than the predetermined value, again using the phase correlation method, the parallel movement amount relative to the entire reference image is calculated, and an approximate geometric transformation formula is generated from the calculated parallel movement amount However, when the normalized correlation value of the edge portion of the original image and the reference image is not equal to or greater than the predetermined value, the amount of translation of the entire original image with respect to the entire reference image is calculated using the normalized correlation method. It may be configured to.

  Further, in the above-described embodiment, the overlapping area extraction unit 14 extracts the overlapping area related to the original image and the reference image moved by the detailed geometric transformation formula while increasing the resolution stepwise, and the characteristics of the overlapping area The geometric transformation equation T is corrected from the point and the corresponding point corresponding to the feature point, but the original image and the reference image with relatively high resolution are used without increasing the resolution stepwise. The detailed geometric transformation formula may be generated only once.

It is a functional block diagram of a control device concerning a suitable embodiment of the present invention. FIG. 2 is a diagram illustrating an operation when the scanner illustrated in FIG. 1 reads a document. It is a flowchart which shows operation | movement of the control apparatus shown in FIG. 4 is a flowchart showing an operation of multi-resolution image generation processing shown in FIG. 3. FIG. 4 is a flowchart showing an operation of image alignment processing shown in FIG. 3. FIG. FIG. 6 is a flowchart showing an operation of a schematic alignment process shown in FIG. 5. It is a flowchart which shows the operation | movement of the detailed alignment process shown in FIG. It is a flowchart which shows the operation | movement of the image joining process shown in FIG. It is the block diagram which showed the construction procedure of the input image pyramid shown in FIG. It is the block diagram which showed the construction procedure of the blend image pyramid shown in FIG. It is the block diagram which showed the construction procedure of the output image pyramid shown in FIG. It is the block diagram which showed the production | generation procedure of the original image data shown in FIG.

Explanation of symbols

1 Control device (image processing device)
2 Scanner 11 Image storage unit (image storage means)
12 Pre-processing unit 12a Multi-resolution unit (multi-resolution unit)
12b Dividing part (dividing means)
13 Approximate alignment part (Approximate alignment means)
13a Phase correlation geometric transformation expression generation unit 14 Region extraction unit (region extraction means)
15 Feature point extraction unit (feature point extraction means)
16 Corresponding point extraction unit (corresponding point extraction means)
17 Detailed alignment part (Detailed alignment means)
18 Global geometric transformation expression generator 19 Joint (joining means)
20 display

Claims (6)

Image storage means for storing a plurality of image data blocks forming one original image data by being arranged so as to partially overlap each other in one direction;
For each of the two overlapping image data blocks stored in the image storage means, a rough position for aligning the original image that is one of the image data blocks and the reference image that is the other image data block Matching means;
Area extracting means for extracting areas overlapping each other in each of the original image and the reference image;
Feature point extraction means for extracting one or more feature points for each region of the original image and the reference image;
For each region from which the feature point extraction unit has extracted feature points, normalization is performed on the basis of the coordinate position of the feature point corresponding to the feature point of the region and the corresponding point in the other region that overlaps the region. Corresponding point extraction means for extracting by performing correlation;
A geometric transformation equation for moving the original image is calculated so that the degree of coincidence between the plurality of feature points in each region and the plurality of corresponding points corresponding thereto is the highest, and based on the calculated geometric transformation equation Detailed alignment means for aligning the original image and the reference image by moving the original image;
An image processing apparatus comprising: a joining unit that joins the original image and the reference image aligned by the detailed alignment unit.   The original image so that the rough alignment means can obtain a phase correlation between the vicinity of one end of the original image in the one direction and the vicinity of the other end of the reference image in the one direction. The position of the original image and the reference image is adjusted by moving the original image based on the calculated geometric transformation formula. The image processing apparatus described.   When the rough alignment means cannot obtain a phase correlation between the vicinity of the one end of the original image and the vicinity of the other end of the reference image, the original image and the reference image The image processing apparatus according to claim 2, wherein a geometric transformation expression for moving the original image is calculated so that a phase correlation can be obtained between the entirety. The image storage means further comprises a multi-resolution means for generating the image data block obtained by reducing the resolution of each image data block at a plurality of resolutions and storing the image data block in the image storage means,
The region extracting unit extracts each region related to the original image and the reference image aligned by the detailed alignment unit in the order of increasing resolution from the lowest resolution stored in the image storage unit,
The said joining means joins the said original image and the said reference image which the said detailed position alignment means aligned regarding the highest resolution which the said image memory | storage means has memorize | stored. An image processing apparatus according to claim 1. Further comprising a dividing means for dividing each image data block stored in the image storage means into processed images having a predetermined size;
The image processing apparatus according to claim 1, wherein the joining unit joins the original image and the reference image in units of the processed image divided by the dividing unit. Image storage means for storing a plurality of image data blocks forming one original image data by being arranged so as to partially overlap each other in one direction;
For each of the two overlapping image data blocks stored in the image storage means, a rough position for aligning the original image that is one of the image data blocks and the reference image that is the other image data block Matching means,
Area extracting means for extracting areas overlapping each other in each of the original image and the reference image;
Feature point extraction means for extracting one or more feature points for each region of the original image and the reference image;
For each region from which the feature point extraction unit has extracted feature points, normalization is performed on the basis of the coordinate position of the feature point corresponding to the feature point of the region and the corresponding point in the other region that overlaps the region. Corresponding point extraction means for extracting by performing correlation,
A geometric transformation equation for moving the original image is calculated so that the degree of coincidence between the plurality of feature points in each region and the plurality of corresponding points corresponding thereto is the highest, and based on the calculated geometric transformation equation Detailed alignment means for aligning the original image and the reference image by moving the original image; and
A program that causes a computer to function as a joining unit that joins the original image and the reference image registered by the detailed positioning unit.

Images