JP5322885B2 - Image combination apparatus and image combination position calculation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image combining device for combining a plurality of images at a higher speed with higher accuracy. <P>SOLUTION: The image combining device 106 includes a unit 165 for setting a matching area for correlation detection, a unit 166 for setting an area to be matched for correlation detection, a correlation calculation unit 167, a combining position specifying unit 168, and a combining position estimating unit 169. The setting unit 165 sets up a matching area composed of a plurality of first images in a first image group. The setting unit 166 sets up a plurality of areas to be matched of different sizes in a second image group, the areas to be matched including an area to be matched in which the size of an image is the same as that of an image in the matching area. The correlation calculation unit 167 calculates a value which specifies correlation between the matching area and each of the plurality of areas to be matched. The combining position specifying unit 168 selects an area to be matched having the highest correlation with the matching area, and specifies a position in the sub-scanning direction of an image included in the area to be matched for combining the first image included in the matching area, as a combining position. The combining position estimating unit 169 estimates the image combining position where specification of the combining position is suspended. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to a technique for combining a plurality of images acquired in a main scanning direction.

  Generally, in an image input / output system applied to a digital copying machine, an image scanner, a facsimile machine, etc., its input means, for example, a one-dimensional image sensor (line sensor) is moved in the reading direction of the document, or It is well known to read an image line by line by moving a document to be read with respect to a line sensor. A direction parallel to the line sensor is called a main scanning direction, and a direction perpendicular to the line sensor (moving direction of the document or the line sensor) is called a sub-scanning direction.

  A CCD sensor or a CMOS sensor used as a line sensor in such an image input / output system is usually produced on a 6-inch or 8-inch wafer due to manufacturing problems in the semiconductor integrated technology, and therefore reads an A3 size or larger document. It is difficult to manufacture a long line sensor that can

  For this reason, conventionally, a plurality of short line sensors shorter than the required length are arranged on the line as one long line sensor, and one line of the original is divided by each short line sensor. In addition to reading, a pixel interpolation process is performed on a missing portion between images read by a plurality of line sensors to compensate for the missing portion image.

  However, with such pixel interpolation processing, an image with sufficient accuracy cannot be obtained as an image of a missing portion. For this reason, in Patent Document 1, a plurality of short line sensors are arranged in a staggered manner so that image data in the document is not lost during reading, and the image information of one line area of the document is divided at different times. In addition to reading as a region, the image data near the boundary of each divided region is read by overlapping several pixels, and the edge of the image read by two adjacent short line sensors is overlapped by several pixels in the main scanning direction Techniques relating to input means are disclosed. In the technique described in Patent Document 1, since a short line sensor arranged in a staggered pattern has a gap in the sub-scanning direction, an image read by the short line sensor due to such a gap is spatially shifted. Will occur. For this reason, in the technique described in Patent Document 1, when images read by the respective short line sensors are combined, a fixed shift amount (gap) Δx is determined in advance, and this is caused by a delay using a memory. The correction is performed by correcting the shift amount Δx.

  Patent Document 2 discloses a technique for generating a single image by combining a plurality of images. For example, in the technique described in Patent Document 2, the mutual distortion / extension detection unit detects mutual distortion and misalignment of the images, and the image correction unit corrects the image based on the detection result, and the image synthesis unit Combine multiple images.

Japanese Patent No. 2982032 (FIGS. 8, 12, paragraph 0038) International Publication No. 2004/077356 Pamphlet (Figure 17, Summary)

  In the technique described in Patent Document 1, the amount of deviation increases or decreases during document reading due to structural problems of the apparatus, or the amount of deviation of an image read by a short line sensor is determined to be an initial amount of deviation Δx. If the value is changed to a different value, the image quality after image combination is affected by the change in the amount of deviation.

  In the technique described in Patent Document 2, images can be corrected and appropriately combined even when the image misalignment varies depending on the reading position of the document. However, for example, it is a measure of misalignment or distortion detection position such as thin gradient or white plain color. In areas where there is no characteristic image such as an edge, the interpolation method is determined based on the amount of displacement and distortion detected in the characteristic image parts before and after that. If the detection of distortion and misalignment between images at a position where a characteristic image exists is not completed in advance, the image joining process cannot be started. In addition, an enormous amount of processing time is required in an apparatus that simultaneously scans a number of line sensors corresponding to the width of an A4 size or A3 size document in a line.

  Therefore, an object of the present invention is to combine a plurality of images with high speed and high accuracy by image signal processing.

  In order to solve the above-described problems, the present invention provides a first imaging unit that acquires a first image and a second image that acquires a second image adjacent to the first image in the main scanning direction. The first image group having a plurality of the first images picked up by the first image pickup means and the second image pickup means in a plurality of scanning lines arranged in the sub-scanning direction Storage means for storing the second image group having a plurality of the second images, the first image included in the first image group, and the image included in the second image group , And a control unit for combining, wherein the control unit includes the first position of the head position on the scanning line in which the combination position is not calculated in the first image group. A plurality of images acquired from a plurality of continuous scanning lines from one image. Correlation detection area setting means for setting a matching area consisting of the first image, and a size of the matching area from the first line in the scanning line in which the combined position is not calculated in the second image group Correlated detection area setting means for setting a plurality of matching areas having different sizes including a matching area that results in an image of the same size, the matching area, and each of the plurality of matching areas, Correlation calculating means for calculating a value for specifying the correlation degree, and when the correlation degree specified by the value for specifying the correlation degree indicates a correlation higher than a predetermined correlation degree, The matching region is selected by selecting the matching region having the highest degree of correlation and combining the first images included in the matching region. When the position in the sub-scanning direction of the included image is specified as the combination position, and the degree of correlation specified by the value specifying the degree of correlation does not indicate a correlation higher than a predetermined degree of correlation, A combination position specifying unit for deferring the process of specifying the combination position of the first image included in the matching area; and the first image for which the combination position specifying unit has deferred the process of specifying the combination position. The position of the image included in the second image group to be combined in the sub-scanning direction is different from the first image in which the combination position specifying unit has suspended the process of specifying the combination position. And a combined position estimating means for calculating the combined position from the combined position of the image.

  As described above, according to the present invention, it is possible to provide an image composition device that combines a plurality of images with high speed and high accuracy.

1 is a schematic diagram of an image reading apparatus according to Embodiment 1. FIG. FIG. 3 is a schematic diagram illustrating an example of combining a plurality of divided images into one image in the first embodiment. FIG. 3 is a schematic diagram illustrating an example of combining a plurality of divided images into one image when an image in which the image position to be combined is shifted in the sub-scanning direction is obtained in the first embodiment. FIG. 3 is a schematic diagram for explaining the principle of image misalignment in the sub-scanning direction in the first embodiment. FIG. 3 is a functional block diagram illustrating a functional configuration of the image combining device according to the first embodiment. FIG. 3 is a schematic diagram of a combined position information table according to the first embodiment. FIG. 3 is a schematic diagram illustrating an example of a matching region set for an N image group in the first embodiment. FIG. 3 is a schematic diagram illustrating an example of a matching area set for an image group in the first embodiment. (A) And (b) is the schematic explaining the process at the time of specifying the image position which combines an attention line with the to-be-matched area | region of the same magnitude | size as a matching area | region in Embodiment 1. FIG. (A)-(c) is the schematic explaining the process at the time of specifying the image position which couple | bonds an attention line to the to-be-matched area | region smaller than a matching area | region in Embodiment 1. FIG. (A)-(c) is the schematic explaining the process at the time of specifying the image position which couple | bonds an attention line to the to-be-matched area | region larger than a matching area | region in Embodiment 1. FIG. (A) And (b) demonstrates the process at the time of specifying the image position which couple | bonds a reservation line and an attention line with the matching area | region of the same magnitude | size as the matching area | region which has a reservation line in Embodiment 1. FIG. Schematic. (A)-(c) demonstrates the process at the time of specifying the image position which couple | bonds a reservation line and an attention line to the to-be-matched area | region smaller in size than the matching area | region containing a reservation line in Embodiment 1. FIG. Schematic to do. (A)-(c) is a case where, in the first embodiment, an image position where each reserved line and target line are combined with a matching area larger than the matching area Er including the reserved line S1 is specified. Schematic explaining the process. 6 is a flowchart (part 1) illustrating image processing according to the first embodiment. 7 is a flowchart (part 2) illustrating image processing according to the first embodiment. FIG. 3 is a schematic diagram illustrating an example of a combined position information table when calculating a combined position in the first embodiment. FIG. 3 is a schematic diagram of a deviation amount table in the first embodiment. FIG. 3 is a schematic diagram of a deviation amount change table in the first embodiment. FIG. 3 is a schematic diagram of a deviation amount change table in the first embodiment. FIG. 3 is a schematic diagram of a deviation amount table in the first embodiment. FIG. 6 is a schematic perspective view showing imaging optical system elements and main optical paths in a modification of the first embodiment. FIG. 6 is a schematic cross-sectional view showing imaging optical system elements and main optical paths of two cells arranged in a modification of the first embodiment.

Embodiment 1 FIG.
FIG. 1 is a schematic diagram of an image reading apparatus 100 according to Embodiment 1 of the present invention. As illustrated, the image reading apparatus 100 includes an imaging optical system 101, an illumination light source 102, a top plate 103, and a plurality of image sensor elements 141, 142, 143, 144, 145, which are arranged on a substrate 104. 146, 147, 148, a line memory 105, and an image combining device 106.

  FIG. 1 shows an example in which the image combining device 106 of the present embodiment is applied to a digital copying machine. For example, a device that needs to combine a plurality of images, for example, a hand scanner, a facsimile, a panoramic image composition device. It can also be applied.

  In the following description, an XYZ orthogonal coordinate system is used. In the image reading, the main scanning direction Dx is the X axis, the sub scanning direction Dy orthogonal to the main scanning direction Dx is the Y axis, and the depth direction Dz orthogonal to both the main scanning direction Dx and the sub scanning direction Dy is the Z axis. To do. Accordingly, the main scanning direction Dx is also referred to as “X-axis direction”, the sub-scanning direction Dy is also referred to as “Y-axis direction”, and the depth direction Dz is also referred to as “Z-axis direction”. In the following description, the Z-axis direction is the thickness direction of the image reading apparatus 100.

  The imaging optical system 101 includes a plurality of imaging optical units 111, 112,... Corresponding to a plurality of short line sensors 141, 142,. .., 118 included.

  Each of the imaging optical units 111, 112,..., 118 is a functionally independent optical means. The plurality of imaging optical units 111, 112,..., 118 are staggered on a plane parallel to the XY plane (adjacent imaging optical units are spaced apart from each other at a predetermined position in the X-axis direction). , Composed of cylindrical optical cells arranged in a second position spaced apart by a predetermined distance in the X-axis direction and the Y-axis direction (usually referred to as a Selfoc lens array). ).

  Here, in the first embodiment, among the plurality of imaging optical units 111, 112,..., 118, one row of imaging optical units 111, 113, 115, 117 arranged in the main scanning direction Dx is provided. The first imaging optical unit group, and one row of the imaging optical units 112, 114, 116, and 118 arranged in the main scanning direction Dx are referred to as a second imaging optical unit group. Thus, since the imaging optical system is composed of a plurality of cells, the apparatus can be miniaturized.

  The first imaging optical unit groups 111, 113, 115, 117 are the first imaging optical unit groups 111, 113, 115, 117 from the imaged areas 31, 33, 35, 37 of the document toward the first imaging optical unit groups 111, 113, 115, 117. The principal rays of light are configured to be parallel to each other. That is, the first imaging optical unit groups 111, 113, 115, and 117 are configured such that the optical axes of the imaging optical units are parallel to each other. The second imaging optical unit groups 112, 114, 116, and 118 are the first imaging optical unit groups 112, 114, 116, and 118 that are directed from the imaged regions 32, 34, 36, and 38 of the document toward the second imaging optical unit group 112, 114, 116, and 118. The principal rays of the two lights are configured to be parallel to each other. That is, the second imaging optical unit group 112, 114, 116, 118 is configured such that the optical axes of the imaging optical unit are parallel to each other.

  FIG. 1 illustrates a case where the imaging optical system 101 includes a first imaging optical unit group 111, 113, 115, 117 and a second imaging optical unit group 112, 114, 116, 118. However, the present invention is not limited to such an embodiment, and three or more coupling optical unit groups may be arranged in the sub-scanning direction Dy.

  On the substrate 104, short line sensors 141, 142,..., 148 are formed in a staggered pattern so as to correspond to the imaging optical units 111, 112,. A row of short line sensors 141, 143, 145, and 147 arranged in the main scanning direction Dx are arranged so as to correspond to the first imaging optical unit groups 111, 113, 115, and 117, and these one row of the line sensors 141, 143, 145, and 147 are arranged. The short line sensors 141, 143, 145, and 147 are referred to as a first image sensor unit group. In addition, one row of short line sensors 142, 144, 146, 148 arranged in the main scanning direction Dx is arranged so as to correspond to the second imaging optical unit group 112, 114, 116, 118. The short line sensors 142, 144, 146, and 148 in the row are referred to as a second image sensor unit group.

  FIG. 1 illustrates a case where the first image sensor unit groups 141, 143, 145, and 147 and the second image sensor unit groups 142, 144, 146, and 148 are formed on the substrate 104. However, the present invention is not limited to such an embodiment, and three or more rows of imaging element units can be arranged in the sub-scanning direction Dy of the substrate 104.

  The illumination light source 102 is composed of, for example, a fluorescent lamp or an LED. For example, the illumination light source 102 is disposed below the top plate 103 and at a position where there is no trouble in reading the imaged areas 31, 32, 33, 34, 35, 36, 37, and 38 of the document. The illumination light source 102 irradiates the imaged areas 31, 32,. The shape of the illumination light source 102 is not limited to the long shape as shown in FIG. 1, and may be another shape. FIG. 1 shows the case where the illumination light source 102 is disposed only on one side of the imaging optical system 101 in the sub-scanning direction Dy. You may arrange | position on both sides of Dy.

  The top plate 103 is a document placing member on which a document (not shown in FIG. 1), which is an example of an object to be imaged (subject), is placed, and can be composed of a transparent glass plate, for example. Further, the top plate 103 functions as a positioning unit that can determine the position of the document in the depth direction Dz (that is, the distance from the imaging optical system 101 to the imaged region of the document). The document is placed on the top plate 103 at a position on the top surface 103a of the top plate 103 or a position shifted in the depth direction Dz from the top surface 103a. In the first embodiment, the upper surface 103a of the top plate 103 is the in-focus position, and when the imaged areas 31, 32,..., 38 existing at the original reading position are in the in-focus position, Images are formed on the line sensors 141, 142,..., 148 from the imaging regions 31, 32,.

  The top plate 103 is not limited to a glass plate as long as it functions as a positioning unit for an object to be imaged and can image the imaged areas 31, 32,. Other means capable of positioning may be used. In addition to the manuscript on which text, a document, a photograph, and the like are displayed, the object to be imaged includes everything that can be an object to read an image, such as a banknote or a human finger.

  In FIG. 1, imaged areas 31, 32,..., 38 are surfaces on the top plate 103 side of the document, and are areas (viewing field ranges) read by the line sensors 141, 142,. There are a plurality of regions arranged in the main scanning direction Dx. Here, the imaged areas 31, 32,..., 38 are configured such that a part of each area overlaps in the main scanning direction Dx.

  The image reading apparatus 100 reads the images of the imaged areas 31, 32,..., 38 of the document on the top plate 103 along the main scanning direction Dx, and every time reading in the main scanning direction Dx is completed. The reading position is relatively moved in the sub-scanning direction Dy. The reading position in the sub-scanning direction Dy can be moved by either moving the document or moving the image reading apparatus 200 including the imaging optical system 101.

  The line memory 105 is a storage unit that temporarily stores image information acquired by the first image sensor unit groups 141, 143, 145, and 147 and the second image sensor unit groups 142, 144, 146, and 148. A plurality of lines are stored and held with the imaging areas 31, 32,..., 38 read in the main scanning direction as one line.

  The image combining device 106 generates a combined image by combining the image information stored in the line memory 105. In FIG. 1, the line memory 105 and the image combining device 106 are shown as separate configurations, but these may be an integrated configuration on the same circuit board, for example.

  Here, the short line sensors 141, 142,..., 148 divide the imaged areas 31, 32,. , 142,... 148 output a plurality of images obtained by dividing the read original. For example, as shown in FIG. 2, an image obtained by dividing a document is output, and in order to copy the document, the images must be combined into a single image.

  Here, as in the present embodiment, the first image sensor unit groups 141, 143, 145, and 147 and the second image sensor unit groups 142, 144, 146, and 148 are arranged in the sub-scanning direction Dy. The optical axes of the first image sensor unit groups 141, 143, 145, and 147 and the optical axes of the second image sensor unit groups 142, 144, 146, and 148 are projected on the YZ plane. In this case, when intersecting at the in-focus position (in this embodiment, the upper surface 103a of the top plate 103), the combined position of the adjacent images in the main scanning direction Dx is constant, but the sub-scanning is performed. The coupling position in the direction Dy changes according to the position of the document (the distance between the document and the top plate 103).

  Therefore, as shown in FIG. 3, an image in which the image position to be combined is shifted in the sub-scanning direction Dy is obtained. Therefore, when two adjacent images are combined, it is necessary to adjust the overlapping position and combine them. On the other hand, there is an effect obtained by shifting the image in the sub-scanning direction Dy in this way. When the amount of deviation in the sub-scanning direction Dy is known, the distance Δz between the original and the top plate 103 can be obtained. By using the distance Δz from the top plate 103 to the document surface as a parameter for image blur correction processing, appropriate correction according to the degree of blur can be performed, so that improvement in image quality can be expected.

  With reference to FIG. 4, the principle of image misalignment in the sub-scanning direction Dy will be described in more detail. In FIG. 4, as an example, the image reading apparatus 100 reads a document 70 partially floating Δz from the top plate 103 while moving from left to right.

  When the image reading apparatus 100 is at the positions P1 and P3, the upper surface 103a of the top plate 103 and the document 70 are in contact with each other, so that the first imaging element unit groups 141, 143, 145, 147, and the second imaging There is no deviation in the images read by the element group 142, 144, 146, 148. On the other hand, when the image reading apparatus 100 is at the position P2, since the upper surface 103a of the top plate 103 and the original 70 are separated by a distance of Δz, the first image sensor unit groups 141, 143, 145, 147, and the first There is a deviation of Δy in the images read by the two image sensor unit groups 142, 144, 146, and 148.

  As shown in FIG. 4, when a document 70 that is spaced apart from the in-focus position (focusing surface) is read, the amount of deviation between adjacent images changes according to the distance Δz between the document 70 and the in-focus position. To do. As in the technique described in Patent Document 1, when a fixed shift amount is determined in advance and images are combined while correcting the shift by a delay using a memory, the shift amount (here, the original 70 and the in-focus position) is obtained. If the distance between them changes, the images will not be smoothly connected. Therefore, in order to combine images with high accuracy, it is necessary to detect a position where two adjacent images are connected and to enlarge or reduce the image.

  FIG. 5 is a functional block diagram showing a functional configuration of the image combining device 106. As illustrated, the image combining device 106 includes a storage unit 161 and a control unit 163.

  The storage unit 161 includes a coupling position information storage unit 162. The combined position information storage unit 162 stores combined position information for specifying a position where an image in each line imaged by each line sensor is combined with an image in each line imaged by an adjacent line sensor. For example, in the present embodiment, a combined position information table 162a as shown in FIG. 6 is stored.

  As illustrated, the combined position information table 162a includes an image group column 162b, a line position row 162c, and a combined position storage area 162d.

  The image group sequence 162b specifies an image read by each of the line sensors 141, 142,. For example, one image group is an image captured by the line sensor 141, two image groups are images captured by the line sensor 142, three image groups are images captured by the line sensor 143, and four image groups. Is an image captured by the line sensor 144, a 5 image group is an image captured by the line sensor 145, a 6 image group is an image captured by the line sensor 146, and a 7 image group is the line sensor 147. The image imaged in is shown.

  The line position row 162c specifies the position of the scanning line in the sub-scanning direction Dy for reading an image. For example, the first line is the first (first) scan line in the sub-scanning direction Dy, the second line is the next scan line in the sub-scanning direction Dy, and the third line is the sub-scanning direction Dy. The second scanning line in FIG. 4 shows the next scanning line, and the fourth line shows the third scanning line in the sub-scanning direction Dy. It should be noted that the line position row 162c is provided with the number of scanning lines determined according to a target such as a document from which an image is read. Further, a minimum number of lines (hMAX (described later) + 1 line in the case of this embodiment) may be provided and used.

  In the combined position storage area 162d, the image included in the image group read by the line sensors 141, 142,..., 148 specified by the image group column 162b in the scanning line specified by the line column row 162c. Is stored in the main scanning direction Dx of the same scanning line as the image. The information for specifying the position to be combined with the image included in the next image group is stored. The value stored in the combined position storage area 162d is calculated by a combined position specifying unit 168, which will be described later. If the combined position cannot be specified, it is left blank as a reserved line flag.

  For example, in the present embodiment, the combined position storage area 162d includes the line sensors 141, 142,..., 148 specified by the image group column 162b in the scanning line specified by the line column row 162c. When an image belonging to the read image group is combined with an image included in the next image group in the main scanning direction Dx of the same scanning line as the image, the image included in the next image group of the image The information for specifying the combined arrival position in the sub-scanning direction Dy is stored as information for specifying the position to be combined. In the example of FIG. 6, the numerical value “1.9” stored in the column of the second line of the first image group column is the second line image included in the first image group and the image included in the second image group. Are combined at a ratio of 0.9 from the beginning in the sub-scanning direction Dy to the end in the sub-scanning direction Dy of the image of the second line of the images in the second image group. This indicates that the position reaches the position (the position from the top in the sub-scanning direction Dy to the ninth position) by dividing the second line image of the image 2 into 10 equal parts. That is, of the numerical values stored in the combined position storage area 162d, the integer part indicates the scanning line, and the decimal part is the sub-scan of the image captured by the scanning line next to the scanning line indicated by the integer part. The ratio from the head position in the direction Dy to the end of coupling is shown.

  Returning to FIG. 5, the control unit 163 includes a combined position detecting unit 164, a combined position estimating unit 169, and an image combining unit 170.

  The combined position detection unit 164 detects a position where each image read by the line sensors 141, 142,..., 148 is combined with the next image in the main scanning direction Dx of the same scanning line as the image. Here, the combined position detection unit 164 includes a correlation detection region setting unit 165, a correlated detection region setting unit 166, a correlation calculation unit 167, and a combined position specifying unit 168.

  The correlation detection area setting unit 165 sets a matching area for a set of images including images read by one or more scanning lines by the line sensors 141, 142,.

  Here, in a specific line sensor, a set of images read by one or more scanning lines is represented by N image groups (N is an index assigned to each image group in order to identify each image group. In the present embodiment, a natural number is assigned in order from 1 to each image group). For example, one image group is a set of images including images read by one or more scanning lines in the line sensor 141, and two image groups are read by one or more scanning lines in the line sensor 142. A group of three images is a set of images including an image read by one or more scanning lines in the line sensor 143. A group of four images is one in the line sensor 144. A set of images including images read by one or more scanning lines. The five image group is a set of images including images read by one or more scanning lines in the line sensor 145. Six image groups Is a set of images including images read by one or more scanning lines in the line sensor 146, and seven image groups are one or more in the line sensor 147. A set image including the image read by the scanning lines of the, 8 images, in the line sensor 148, is a set of images including the image read by one or more scan lines.

  FIG. 7 is a schematic diagram illustrating an example of a matching region set by the correlation detection region setting unit 165 for the N image group. Here, in FIG. 7, a plurality of rectangular cells having the same shape arranged from the top to the bottom indicate the images included in the N image group, and the order from the top to the bottom of the images included in the N image group. Indicates the order of reading by the line sensor (the order of young scanning lines).

  Then, as shown in FIG. 7, the correlation detection area setting unit 165 sets the first line as the attention line Al among the lines that are not determined lines or reserved lines as will be described later in the N image group. A line Al, a target line Ol including a predetermined number n of lines following the target line Al, and a reserved line (not shown in FIG. 7) are set as a matching region Mr. Here, the reserved line indicates a line for which specification of the combined position is suspended because the combined position with the adjacent image cannot be detected when the line is set as the target line. Further, the fixed line indicates a line in which the coupling position with the adjacent image is fixed.

  Further, the correlation detection area setting unit 165 sets two lines including the attention line Al and a line immediately below the attention line Al as the edge detection area Er. Ep indicates the edge detection position.

  In the example illustrated in FIG. 7, the target lines Ol are five lines, but the number of target lines Ol is not limited to this. Further, the number of lines in the edge detection area Er is not limited to two.

  Returning to FIG. 5, the correlated detection region setting unit 166 selects the next image captured in the main scanning direction Dx with respect to the image included in the image group in which the correlation detection region setting unit 165 has set the matching region. A plurality of matching areas are set for the included image group.

  For example, the correlated detection region setting unit 166 has the same size (the same number of lines) as the matching region from the image region included in the first line among the image regions that are not the combined lines as described later. Based on the region, a plurality of regions having different sizes, in which the region end position is changed from -m line to + m line size with the region start position fixed, are set as the matching regions. Note that m is a matched region generation coefficient, and an arbitrary natural number is set in advance. It is also assumed that a difference value for changing the end position of the region from the −m line to the + m line size is set in advance. Here, the to-be-coupled line refers to an image area of the to-be-matched area that is confirmed to be coupled to the target line Al (determined line) included in the matching area.

  FIG. 8 is a schematic diagram illustrating an example of a matched region set for the image group by the correlated detection region setting unit 166. Here, in FIG. 8, a plurality of rectangular cells of the same shape arranged from the top to the bottom indicate images included in the image group, and the order from the top to the bottom of the images included in the image group is as follows. It is assumed that the order of reading by the line sensor (the order of young scanning lines) is shown.

Then, the correlated detection region setting unit 166 is the same as the matching region, assuming that the matching region set by the correlation detection region setting unit 165 has 6 lines and the matching region generation coefficient m is set to 3. and the matching area Tr ₀ size, 1 line and large the matching area Tr ₁ than the matching area, two lines and large the matching area Tr ₂ than the matching area, three lines larger the matching area Tr ₃ than the matching area And a matching region Tr- ₁ that is one line smaller than the matching region, a matching region Tr- ₂ that is two lines smaller than the matching region, and a matching region Tr- ₃ that is three lines smaller than the matching region are set. . Here, a plurality of matching regions whose sizes change in units of one line are provided, but the unit of change is not limited to this, and may be two or more lines or less than one line.

  Returning to FIG. 5, the correlation calculation unit 167 performs, for each combination of the matching region set by the correlation detection region setting unit 165 and each of the plurality of matching regions set by the correlated detection region setting unit 166, A value specifying the degree of correlation between them is calculated.

  For example, if the size of the matching region set by the correlation detection region setting unit 165 and the matching region set by the correlated detection region setting unit 166 are the same, the correlation calculation unit 167 sets the matching region as the matching region. A difference value between the image of the line included and the image of the line included in the matching region corresponding to the position of the line included in the matching region is calculated, and the difference values of all the lines are summed.

  In addition, the correlation calculation unit 167 determines the detection region when the size of the matching region set by the correlation detection region setting unit 165 and the matching region set by the correlated detection region setting unit 166 are different. The line included in the matching region set by the correlation detection region setting unit 165 after the size of the matching region set by the setting unit 166 matches the size of the matching region set by the correlation detection region setting unit 165 The difference value between the image and the image included in the matching area corresponding to the position of the line included in the matching area is calculated, and the difference values of all lines are summed.

  Note that the value that specifies the degree of correlation calculated by the correlation calculation unit 167 is not limited to such a difference value. For example, in the matching area line that is the same size or the same size as described above. The correlation coefficient may be calculated from the pixel value of the included image and the pixel value of the image in the matching area corresponding to the line in the matching area, and the correlation coefficient calculated for all lines may be summed. .

  There are various methods for converting the region size. For example, the size conversion may be performed by primary conversion. Furthermore, instead of the image data (pixel values, etc.) included in the line, the value for specifying the correlation may be calculated by creating data of less than one line by interpolation, and the entire matching area and the entire matching area may be calculated. And a value for specifying the degree of correlation may be calculated.

  When the combined position specifying unit 168 determines that the degree of correlation between the matching region and the specific matching region is higher than the value for specifying the degree of correlation calculated by the correlation calculating unit 167, the degree of correlation is the highest. A matching region having a high value is selected, and an image position of the matching region that is coupled to the target line Al included in the matching region is specified.

  With reference to FIGS. 9 to 14, a description will be given of a process in which the combined position specifying unit 168 specifies the image position of the matched region that is combined with the target line Al included in the matching region.

FIG. 9 (a) and (b) are to be matching area Tr ₀ of the same size as the matching area Er, it is a schematic diagram for explaining the processing for specifying an image position for coupling the target line Al. Here, in FIGS. 9A and 9B, a plurality of rectangular cells having the same shape arranged from the top to the bottom indicate images included in the image group, and above the images included in the image group. The order from the bottom to the bottom indicates the order read by the line sensor (the order in which scanning lines are young).

As shown in FIG. 9A, the coupling position specifying unit 168, when the size of the matching region Er and the matching region Tr ₀ is the same, as shown in FIG. 9B, the matching region Tr An image region Ia having the same size as the target line Al is specified as a region to be combined with the target line Al from the leading position of ₀ , and information specifying the end position Ep of the image region Ia is set as a combined position. Note that the combination position includes the number of images (images acquired in one scanning line) included from the start position of the image group in which the matching area Tr ₀ is set to the end position Ep of the image area Ia, and the end position Ep. Is specified by a value obtained by adding the ratio from the start position to the end position Ep of the image including the.

  When the line below the target line Al is set as the next target line, the next matching region is assigned from the line below the end position Ep.

FIGS. 10A to 10C are schematic diagrams for explaining processing when specifying an image position where the target line Al is coupled to the matching region Tr- ₂ having a smaller size than the matching region Er. Here, in FIGS. 10A to 10C, a plurality of rectangular cells having the same shape arranged from the upper side to the lower side indicate images included in the image group, and above the images included in the image group. The order from the bottom to the bottom indicates the order read by the line sensor (the order in which scanning lines are young).

As shown in FIG. 10A, the coupling position specifying unit 168, when the size of the matching region Tr- ₂ is smaller than the matching region Er, as shown in FIG. The image included in Tr- ₂ is enlarged to the same size as the matching area Er (6/4 times in the examples of FIGS. 10A to 10C), and the start position of the enlarged matching area ETr- ₂ And the image area Ia ₀ having the same size as the noticed line Al is used as an area to be joined with the noticed line Al, and the image included in the enlarged matched area ETr- ₂ is reduced as shown in FIG. in (here, 4/6-fold) when returning to its original size, and the binding position information identifying the end position Ep ₁ of the reduced image region Ia ₁ at the same rate.

In the case where the line below the target line Al as the next line of interest, the matching area of the next is assigned a line below the end position Ep _1.

Figure 11 (a) ~ (c) is to be matching area Tr ₂ larger size than the matching area Er is a schematic diagram for explaining the processing for specifying an image position for coupling the target line Al. Here, in FIG. 11, a plurality of rectangular cells having the same shape arranged from the top to the bottom indicate the images included in the image group, and the order from the top to the bottom of the images included in the image group is as follows. It is assumed that the order of reading by the line sensor (the order of young scanning lines) is shown.

Coupling position specifying unit 168, as shown in FIG. 11 (a), when than the matching area Er large size of the matching area Tr _2, as shown in FIG. 11 (b), the matching area Tr reducing the image contained in the ₂ to the same size as the matching area Er (in the example of FIG. 11 (a) ~ (c) , 6/8 -fold), the line of interest from the reduced head position of the matching area RTr ₂ By setting the image area Ia ₀ having the same size as Al as the area to be combined with the target line Al, as shown in FIG. 11C, by enlarging the image included in the reduced matching area RTr _-2 (here So when returning to 8/6 times) its original size, and the binding position information identifying the end position Ep ₁ of the enlarged image region Ia ₁ at the same rate.

In the case where the line below the target line Al as the next line of interest, the matching area of the next is assigned a line below the end position Ep _1.

Figure 12 (a) and (b) are the same size of the matching area Tr ₀ and matching area Er having pending line Sl, the processing for specifying an image position for coupling the hold line Sl and the target line Al It is the schematic to explain. Here, in FIG. 12, a plurality of rectangular cells having the same shape arranged from the top to the bottom indicate images included in the image group, and the order from the top to the bottom of the images included in the image group is as follows. It is assumed that the order of reading by the line sensor (the order of young scanning lines) is shown. FIGS. 12A and 12B show a case in which three reserved lines S1 are included.

As shown in FIG. 12A, the coupling position specifying unit 168, when the size of the matching region Er and the matching region Tr ₀ is the same, as shown in FIG. Each image area Ia having the same size as each of the reserved line S1 and the target line Al from the head position of ₀ is specified as an area connected to the reserved line S1 and the target line Al, and the end position Ep of each image area Ia The information for identifying is set as the coupling position.

  When the line below the target line Al is set as the next target line, the next matching region is assigned from the line below the end position Ep.

FIGS. 13A to 13C are diagrams for specifying an image position that joins the reserved line S1 and the target line Al to the matching area Tr- ₂ having a smaller size than the matching area Er including the reserved line S1. It is the schematic explaining a process. Here, in FIGS. 13A to 13C, a plurality of rectangular cells having the same shape arranged from the top to the bottom indicate images included in the image group, and above the images included in the image group. The order from the bottom to the bottom indicates the order read by the line sensor (the order in which scanning lines are young). FIGS. 13A to 13C show a case in which three reserved lines S1 are included.

As shown in FIG. 13A, the combined position specifying unit 168, when the size of the matching region Tr- ₂ is smaller than the matching region Er, as shown in FIG. The size of Tr- ₂ is enlarged to the same size as the matching region Er (9/7 times in the examples of FIGS. 13A to 13C), and from the head position of the enlarged matching region ETr- ₂ As shown in FIG. 13C, the image area Ia ₀ having the same size as each of the reserved lines S1 and the target line Al is defined as an area coupled to each of the reserved lines S1 and the target line Al. by reducing the matching area ETr _-2 (here, 7/9-fold) when returning to its original size, binding information identifying the end position Ep ₁ of the reduced image region Ia ₁ at the same rate Position.

In the case where the line below the target line Al as the next line of interest, the matching area of the next is assigned a line below the end position Ep _1.

Figure 14 (a) ~ (c) is to be matching area Tr ₂ larger size than the matching area Er including the hold line Sl, when specifying the image position to couple the respective hold lines Sl and the target line Al It is the schematic explaining the process of. Here, in FIGS. 14A to 14C, a plurality of rectangular cells having the same shape arranged from the upper side to the lower side indicate images included in the image group, and above the images included in the image group. The order from the bottom to the bottom indicates the order read by the line sensor (the order in which scanning lines are young). Further, FIGS. 14A to 14C show a case in which three reserved lines S1 are included.

Coupling position specifying unit 168, as shown in FIG. 14 (a), when than the matching area Er large size of the matching area Tr _2, as shown in FIG. 14 (b), the matching area Tr ₂ is reduced to the same size as the matching region Er (9/11 times in the examples of FIGS. 14A to 14C), and each of the reduced matching target regions RTr ₂ is started from the start position. An image region Ia ₀ having the same size as the reserved line S1 and the target line Al is set as a region that is connected to each reserved line S1 and the target line Al, and as shown in FIG. by expanding _-2 (here, 11/9-fold) when returning to its original size, and the binding position information identifying the end position Ep ₁ of the enlarged image region Ia ₁ at the same rate .

In the case where the line below the target line Al as the next line of interest, the matching area of the next is assigned a line below the end position Ep _1.

  Returning to FIG. 5, the coupling position specifying unit 168 controls processing for storing the coupling position calculated as described above in the coupling position information storage unit 162.

  The combined position estimation unit 169 combines the images corresponding to the first reserved line when the number of images set as the reserved lines by the combined position specifying unit 168 reaches a predetermined number in one image group. The process which estimates the coupling position to perform is performed.

  Here, the coupling position deviates from a predetermined interval because the paper to be imaged is bent as shown in FIG. 4, or the distance between the line sensor and the original changes during the conveyance for some reason. It is time to do. Normally, when the paper to be conveyed is bent or warped, the direction is the sub-scanning direction, and the change in the paper is locally highly correlated with the rate of change in the areas adjacent to the target imaging area.

  For example, in FIG. 1, when any paper change occurs at the position of the imaging area 33, there is a high possibility that the tendency appears in the imaging area 32 and the imaging area 34. Therefore, if there is an image whose combined position is fixed in the vicinity, the combined position estimation unit 169 estimates the combined position of the image on the reserved line from the state of change in the combined position information of the image. In addition, the process in the joint position estimation part 169 is demonstrated in detail below.

  The image combination unit 170 combines the images based on the combination position information stored in the combination position information storage unit 162. The processing in the image combining unit 170 will be described in detail below.

  The image combining device 106 described above can be realized by a computer including a CPU (Central Processing Unit) and a memory, for example. For example, the storage unit 161 can be realized by the CPU using a memory, and the control unit 163 can be realized by executing a predetermined program by the CPU. However, the image combining device 106 according to the present embodiment is not limited to that realized by software on a computer system. For example, it may be realized in hardware by an integrated logic IC such as ASIC (Application Specific Integrated Circuits) or FPGA (Field Programmable Gate Array), or may be realized in software by a DSP (Digital Signal Processor) or the like. It may be a thing.

  15 and 16 are flowcharts showing image processing in the image combining device 106. FIG.

  First, the combined position specifying unit 168 sets 1 to an index N indicating an image group, and sets 1 to an index i indicating a line to be processed (S10).

  Next, the coupling position specifying unit 168 initializes an index h indicating the number of processing lines to 0 (S11).

  Next, the correlation detection region setting unit 165 sets a matching region Mr in the N image group (S12). For example, when the line above the line indicated by the index i is not a reserved line in the N image group, the correlation detection area setting unit 165 and the line indicated by the index i A preset number of target lines Ol from the next scanning line is set in the matching region Mr. The correlation detection area setting unit 165 also scans the line preceding the line indicated by the index i when the line above the line indicated by the index i is a reserved line in the N image group. Are set in the matching area Mr, the reserved line that continues from, the line indicated by the index i, and the preset number of target lines Ol from the scanning line next to the line indicated by the index i.

  The correlation detection area setting unit 165 sets the attention line specified by the index i in the column of the combined position storage area 162d specified by the image group sequence 162b specified by the index N in the combined position information table 162a. Whether or not to include a reserved line in the matching area can be determined based on whether or not there is a reserved line flag at a position above the column of the combined position storage area 162d specified by the corresponding line position row 162c. .

  Further, the correlation detection area setting unit 165 sets the scanning line specified by the index i as the attention line Al in the matching area Mr set in step S12, the attention line Al, and the next scanning of the attention line Al. The line is set as the edge detection area Er (S13).

  Next, the correlated detection region setting unit 166 sets a predetermined number of matching regions in the N + 1 image group including the next image in the main scanning direction of the image included in the N image group (S13). ). For example, the correlated detection region setting unit 166 uses a matching region generation coefficient determined in advance from a matching region having the same size as the matching region set in step S12 and the size of the matching region set in step S12. A matching area having a size that is obtained by increasing or decreasing a predetermined difference value until the specified size is reached is set.

  Next, the combination position specifying unit 168 determines whether or not the edge detection area Er set in step S13 includes a characteristic part in the image (S15). If a characteristic part is included (Yes in step S15), the process proceeds to step S16. If a characteristic part is not included (No in step S15), the process proceeds to step S20. The characteristic portion referred to here indicates an edge, a character, a pattern, or the like of an image that is easy to understand as a mark when detecting a superposition position necessary for combining images. That is, it means a portion where the signal level change of the image is large.

  In the present embodiment, a method for detecting a characteristic portion in an image in the edge detection region Er is not particularly limited, but the overlapping position of adjacent images moves only in the sub-scanning direction. If the overlapping position is detected at a position where there is a vertical edge, a highly accurate result can be obtained. Therefore, for example, when the sum of the differences between the signal components in the upper and lower lines is larger than a predetermined value, it is determined that the signal change is large, that is, the edge of the image exists.

  In step S16, the correlation calculation unit 167 calculates a value that specifies the degree of correlation between the matching region Er set in step S12 and each of the plurality of matching regions set in step S14 (S16).

  In both the matching area and the non-matching area, each line is provided with a subline finer than one line in order to increase the coupling accuracy, and each subline is generated by data interpolation when the area size is enlarged or reduced. Data interpolation may be performed using any method such as linear interpolation or bicubic.

  Then, the combined position specifying unit 168 determines whether or not there is a matching region in which a value indicating a correlation stronger than a predetermined threshold is calculated among the values specifying the correlation degree calculated in step S16. (S17). If there is such a matching area (Yes in step S17), the process proceeds to step S18, and if there is no such matching area (No in step S17), the process proceeds to step S20.

  In step S18, the combined position specifying unit 168 selects a matching region in which a value indicating the strongest correlation is calculated among the values specifying the degree of correlation calculated in step S16.

  Then, the joint position specifying unit 168 specifies the joint position of the line from the head position of the matching area set in step S12 to the edge detection position (S19). Here, in this embodiment, since the edge detection area Er is set to two lines and the edge is detected from the difference between the upper and lower lines, the first line of the edge detection area Er, that is, the coupling position to the target line Al is specified. . Note that for the joint position specified in this way, the joint position specifying unit 168 uses the image group sequence 162b specified by the index N in the joint position information table 162a and the line corresponding to the line for which the joint position is calculated. Information for specifying the calculated coupling position is stored in the column of the coupling position storage area 162d identified by the position row 162c.

  On the other hand, in step S20, the coupling position specifying unit 168 sets the attention line Al specified in step S13 as a reserved line. For example, in the combined position information table 162a, the combined position specifying unit 168 specifies the combination specified by the image group sequence 162b specified by the index N and the line position row 162c corresponding to the target line specified by the index i. The reserved line flag is stored in the column of the position storage area 162d. In the present embodiment, the reserved line flag is set by leaving the column of the combined position storage area 162d blank.

  Next, the combined position specifying unit 168 checks whether or not the image group specified by the index N is the maximum value of N (NMAX) (S21). If NO in S21, the process proceeds to Step S22, and if it is the maximum value of N (Yes in Step S21), the process proceeds to Step S23. The maximum value of N is a number obtained by subtracting 1 from the number of line sensors 141, 142,.

  In step S22, the coupling position specifying unit 168 adds 1 to the index N, returns to step S12, and repeats the processing. In step S23, the coupling position specifying unit 168 adds 1 to the index h, and proceeds to step S24.

  In step S24, the coupling position specifying unit 168 confirms whether or not the index h is the maximum value (hMAX) of h, and if it is not the maximum value (hMAX) of h (No in step S24), step The process proceeds to S25, and if it is the maximum value of h (hMAX) (Yes in Step S24), the process proceeds to Step S26 in FIG. Here, the value of hMAX is a difference value between the line for calculating the combining position and the line for combining the images. The hMAX value can be set to an arbitrary value (19 in the present embodiment) within a range not exceeding the image size (number of scanning lines). However, as the hMAX value increases, the line of interest and the image are combined. Since the position difference from the line for outputting the result becomes large, and as a result, the capacity of the memory for temporarily storing the image data becomes large, it is desirable not to set the value so large.

  In step S25, the coupling position specifying unit 168 initializes the index N to 1, and adds 1 to the index i. Then, the coupling position specifying unit 168 returns to Step S11 and repeats the process.

  In step S26, the combined position specifying unit 168 checks whether there is a reserved line setting on the line indicated by the value obtained by subtracting the hMAX value from the index i value. (Yes in step S26), the process proceeds to step S27, and if no hold line is set (No in step S26), the process proceeds to step S28. Here, whether or not there is a reserved line in the line indicated by the value obtained by subtracting the hMAX value from the index i value is obtained by subtracting the hMAX value from the index i value in the combined position information table 162a. It can be determined by whether or not there is a reserved line flag in the column of the combined position storage area 162d of the line position row 162c corresponding to the value.

  In step S27, the combined position estimation unit 169 calculates the combined position of the reserved line set in the line indicated by the value obtained by subtracting the hMAX value from the index i value, and sets it as the confirmed line. The calculation process in step S27 will be described in detail below.

  In step S28, the image combining unit 170 combines the images included in the line indicated by the value obtained by subtracting the value obtained by adding 1 to the hMAX value from the value of the index i according to the value indicated in the combined position information table 162a. To do. The combining process in step S28 will be described in detail below.

  Next, the joint position specifying unit 168 checks whether or not the line specified by the index i is the maximum value of i (iMAX) (S29). If the line is not the maximum value of i (step S29). No), the process proceeds to step S30, and if i is the maximum value (Yes in step S29), the process proceeds to step S31. Here, the maximum value of i is the total number of scanning lines in the sub-scanning direction.

  In step S30, the coupling position specifying unit 168 initializes the index N to 1, adds 1 to the index i, and subtracts 1 from the index h. Then, the coupling position specifying unit 168 returns to Step S11 in FIG. 15 and repeats the process.

  In step S31, the image combination unit 170 combines the images that have not been combined yet according to the values indicated in the combination position information table 162a. When the reserved line flag is set in the combined position storage area 162d of the combined position information table 162a, the combined position estimation unit 169 calculates the combined position, and the image combining unit 170 combines the calculated combined positions. I do.

  Here, a method of calculating combined data by the combined position estimation unit 169 will be described. FIGS. 17A and 17B are examples of the combined position information table 162a.

  First, the combined position information table 162a is as shown in FIG. 17A, and the third and subsequent rows in the three image group column are reserved lines, and the combined positions of the images indicated by these reserved lines Is indefinite.

In step S27 of FIG. 16, when the combined position estimation unit 169 must calculate the combined position of the image corresponding to the third line of the three image group sequence, first, in the combined position information table 162a. A group of images adjacent to the right from the line immediately above the line for which the coupling position is calculated to the line for which the coupling position is calculated (in this case, from the second line to the third line) Is calculated for each image using the following equation (1).

  The values calculated by the combined position estimation unit 169 using the expression (1) are shown in the deviation amount table 180a shown in FIG. In the deviation amount table 180a, the value of S (N, L) calculated using the equation (1) is stored in the column corresponding to the N image group and the Lth line. For example, since the third line of the first image group is combined with the second 2.75 line of the second image group, the amount of deviation from the second image in the third line of the first image group is 2.75−3.0 = −0. .25 lines. Similarly, the amount of deviation from the third image group in the third line of the second image group is 0.25 lines, and the amount of deviation from the fifth image group in the third line of the fourth image group is 0.25 lines. The amount of deviation from the fourth image group in the third line of the third image group is blank because there is no information on the coupling position.

Next, the combined position estimation unit 169 calculates the amount of change ΔS of the combined position shift amount in each image by the following equation (2).

  The values calculated by the coupling position estimation unit 169 using the equation (2) are shown in the deviation amount change table 181a shown in FIG. In the deviation amount change table 181a, the value of ΔS (N, L) calculated using the equation (2) is stored in the column corresponding to the N image group and the L line. For example, a change in the amount of deviation between the second line and the third line of one image group is (−0.25) − (− 0.1) = − 0.15. Similarly, the change in the shift amount between the second line and the third line in the two image group is 0.15 line, and the change in the shift amount between the second line and the third line in the four image group is 0.15 line. The change in the amount of deviation between the second line and the third line of the three image groups is left blank because there is no information on the coupling position.

  Next, the combined position estimation unit 169 estimates a change in the shift amount of the image corresponding to the reserved line for which the combined position is calculated from the change in the peripheral shift amount. In the first place, since the amount of deviation corresponds to the distance to the document surface, a change in the amount of deviation represents a change in the distance to the document surface. Since the original has a continuous surface, if the original surface distance changes at a certain position, it is considered that the original surface distance also changes in the vicinity. Therefore, a change in the amount of deviation of the reserved line is estimated from a change in the amount of deviation of the fixed line adjacent to the reserved line that is the target of calculation of the combined position.

  For example, in the shift amount change table 181a, the combined position estimation unit 169 calculates the average value of the shift amounts of the left and right images when the left and right sides of the reserved line that is the target of the combined position are not reserved lines. Hold line value.

  Further, for example, in the shift amount change table 181a, the combined position estimation unit 169 may use the shift amount of the shift amount of the image that is not the reserved line as it is when either one of the left and right sides is not the reserved line. A value corrected by a predetermined correction value such as multiplying the change amount of the deviation amount by a half may be used.

  On the other hand, in the shift amount change table 181a, the combined position estimation unit 169 determines that the same line in two or more separated image groups when both the left and right sides of the reserved line for which the combined position is calculated are reserved lines. It is estimated from the amount of change in the amount of deviation of the image belonging to. For example, among the images belonging to the same line as the reserved line for which the combined position is to be calculated, the amount of change in the shift amount of the image closest to the reserved line may be used as it is. A value obtained by correcting the change amount of the shift amount with a predetermined correction value may be used.

  In addition, when there are two images that are closest to the reserved line among the images belonging to the same line as the reserved line for which the combined position is calculated, for example, three images are separated from the reserved line. If the image is a fixed line, a value obtained by correcting the average value of these values with a predetermined correction value may be used. Further, as the correction value here, it is desirable to use a correction value that corrects the change amount of the shift amount to be smaller as the image group moves away from the reserved line.

  In addition, in the deviation amount change table 181a, the combined position estimation unit 169 has no change in the amount of deviation, that is, ΔS = 0 when both the left and right sides of the reserved line for which the combined position is calculated are reserved lines. It is good.

  At this time, when the optical system shown in FIG. 1 is used, when the position of the document surface changes, the position where the adjacent image is read in the document changes in a different direction. Therefore, the estimated value must consider the sign. For example, as shown in FIG. 19, in the shift amount change table 181a, the change in the shift amount between the left and right adjacent images of the third reserved line in the three image group is both 0.15, so the average value is also 0.15. It becomes. However, since the change in the shift amount of the adjacent images is in the reverse direction, −0.15 with the sign reversed is an estimated value of ΔS (3, 3). On the other hand, since the change in the amount of deviation is the same in two distant images, there is no need to reverse the sign. That is, when the image group to which the image belongs is divided into the first image sensor unit group 141, 143, 145, 147 and the second image sensor unit group 142, 144, 146, 148, the same group. The value of the image belonging to the image may be used as it is, but the value of the image belonging to a different group needs to be used after inverting the sign.

  FIG. 20 shows a deviation amount change table 181b that stores the amount of change in the deviation amount of the reserved line, which is the calculation target of the combined position, calculated by the combined position estimation unit 169 as described above.

Then, the combined position estimation unit 169 uses the change amount of the deviation amount of the reserved line that is the target of calculation of the combined position obtained as described above, by the reverse calculation of the expression (2), that is, (3) The amount of deviation of the reserved line is calculated by the equation (3).

  Note that the shift amount table 180b shown in FIG. 21 shows the shift amount of the reserved line, which is the calculation target of the combined position, calculated by the combined position estimation unit 169 using Equation (3).

Furthermore, the combined position estimation unit 169 uses the amount of deviation obtained by the expression (3) to perform the reverse calculation of the expression (1), that is, the holding position that is the object of calculation of the combined position by the following expression (4). Calculate the line coupling position.

  As shown in FIG. 17B, the combined position of the reserved line, which is the calculation target of the combined position, calculated by the combined position estimation unit 169 as described above is stored in the combined position information table 162a. The

  As described above, in the present embodiment, when the coupling position is unknown, a coupling position with higher accuracy is estimated by estimating the coupling position that determines the coupling position with the adjacent image based on the correlation of changes in the document shape. Can be derived.

  Next, the combining process in the image combining unit 170 will be described. In the combined position information table 162a, since the i-th line (one-line image data) of each image matches with which line of the image on the right side (position in the sub-scanning direction), the combined position information table 162a. A combined image for one line cannot be generated simply by connecting the images of the lines indicated by the data in order. Therefore, a specific description will be given using the value of the combined position storage area 162d corresponding to the second line of the combined position information table 162a shown in FIG.

  The image combining unit 170 knows from the value of the second line of one image that the second line of the one image group matches the 1.9th line of the two image groups adjacent to the right. An image specified between the end position of the first line and the end position of the 1.9 line of the second image group is enlarged and connected to the image of the line.

  Next, the image combining unit 170 connects the images included in the two image groups and the images included in the three image groups. At this time, the images are stored in the combined position storage area 162d of the second line of the two image groups in FIG. If it is connected with the 2.1st line of the images of the three image groups according to the values that are set, the images are not connected. This is because the second line of the first image group and the 1.9th line of the second image group are already connected, and in order to connect these, it matches the 1.9th line of the second image group. It is necessary to determine the number of lines in the three image groups.

  Here, the line that actually connects the images is referred to as coordinate data, and the coordinate data is simply calculated from the values stored in the combined position storage area 162d by the following equations (5) to (7). C is a value of the coupling position, P is coordinate data, N is a natural number for distinguishing image groups, and i is the number of lines. Note that C (N, i) is the value of the combined position storage area 162d corresponding to the N image group and the i-th line in the combined position information table 162a, and P (N, i) is the N image group and the i-th line. Is the value of the coordinate data for combining the image data with the images included in the N + 1 image group.

  When N = 1, the coordinate data of the second line in FIG. 9 is obtained, P (1,2) = 2, P (2,2) = 1.9, P (3,2) = 2, P (4, 2) = 1.9 and P (5,2) = 2. That is, when the combined image of the second line is generated, the second line of the first image group, the first line of the second image group, the second line of the third image group, and the first line of the fourth image group are 1.9th line. And the second line of the five image groups may be enlarged or reduced and joined together.

  As described above, according to the present embodiment, in each line, the images acquired by the respective image sensor units 141, 142,..., 148 are reduced or enlarged so that the feature portions thereof coincide with each other. In order to connect them together, a smoothly connected image is acquired.

  Further, if the document surface distance changes during reading of a document having no characteristic part in the edge detection area, it may be necessary to rapidly enlarge or reduce the image when the characteristic part appears. Therefore, in the present embodiment, when there is no characteristic part in the image and thus an edge cannot be detected or a correlation between the matching area and the matching area cannot be obtained, the coupling position is determined. Instead, the enlargement / reduction amount per line is reduced even if the coupling position changes suddenly, so that the distortion of the image due to the enlargement / reduction process can be reduced.

  However, if the matching area is set wide when there is a characteristic part in the document, it becomes impossible to follow a minute change in the document surface distance. Therefore, in this embodiment, when there is a characteristic part in the original and the matching area and the area to be matched can be correlated, the coupling position is detected with the minimum matching area, and there is no characteristic part. In addition, by expanding the matching area, distortion when the image is enlarged or reduced is not conspicuous, and by detecting the coupling position following the change in the document surface distance, it is possible to perform accurate image coupling. . In this embodiment, since the continuity of the image is maintained by setting the matching area and the matching area to the line next to the line where the coupling position is determined, the image of the image due to the coupling position detection processing error or the like is maintained. There is no significant deterioration in image quality, such as loss, repetition of the same image, or change of context.

  Furthermore, in this embodiment, the matching region is provided small, and every time one line is input from the reading apparatus, sequential processing for outputting a combined image for one line delayed by a fixed line is performed, so that all image data is stored. Therefore, the frame memory is not necessary, and the cost can be reduced. Further, since it is not necessary to perform a combining process that waits until image reading for one frame is completed, the processing speed can be increased.

  In addition, when the region of the image to be combined (matched region) is detected, the combined position is determined up to the edge detection range. This is because there is less. In other words, the degree of correlation is the highest in the size of the matched region where the adjacent images are best matched. However, it is unclear at this time whether the image has a feature below the edge detection region. When a characteristic image is included, it is useful for detection of the coupling position. When it is not included, it cannot be detected even if a sharp image shift occurs below the edge detection region. Therefore, it is possible to avoid erroneous detection of the combined position by reliably determining the combined position up to the edge detection position where the characteristic image is included.

Modification of the first embodiment.
In Embodiment 1 of the present invention, the imaging optical system 101 is configured by a cylindrical optical cell. However, the present invention is not limited to such an embodiment. For example, a reflection optical system configured by a plurality of optical components is used. Alternatively, the same effect can be obtained with the imaging optical system. FIG. 22 shows an example in which a reflection optical system is used.

  First, the arrangement and optical paths of optical system elements for each cell will be described. FIG. 22 is a diagram showing the imaging optical system elements and main optical paths of the cell 411 in the main scanning direction (X direction). FIG. 23 is a cross-sectional view showing imaging optical system elements and main optical paths of two cells 411 and 412 arranged in the sub-scanning direction (Y direction). The optical system of each cell is an optical system telecentric on the document side.

  The cell 411 belonging to the series 1 includes a first lens 452a and an aperture 453a, a second lens 454a, and a holder for holding them. The cell 412 belonging to the series 2 includes a first lens 452b and an aperture 453b, a second lens 454b, and a holder for holding them. The first lenses 452a and 452b are concave mirrors as a reflection optical system, but are called first lenses because they have the same functions as the lenses. The second lenses 454a and 454b are concave mirrors as a reflecting optical system, but are called second lenses because they have the same functions as the lenses. In the cell 411, a first folding mirror 451a in the horizontal direction is inserted in the middle of the optical path from the document surface to the cell. The folding direction by the first folding mirror 451a is the right side in FIGS. The folding direction by the second folding mirror 451b is the left side in FIG. A second folding mirror 455a is inserted in the middle of the optical path from the second lens 454a to the image sensor 141. The folding direction by the second folding mirror 455a is downward in FIG. 22 and FIG. 23 in the series 1 (direction toward the image sensor 141 on the substrate 404). The folding direction by the second folding mirror 455b is downward in FIG. 23 (direction toward the image sensor 142) in the second series. The optical axis of each cell is obliquely incident on the top plate 103 with respect to the sub-scanning direction. Other cells have the same configuration.

  If the apertures 453a and 453b are arranged at the back focal positions of the first lenses 452a and 452b, a telecentric optical system can be realized on the document side. The transfer magnification may be larger than 1 or reduced smaller than 1, but if it is set to the same magnification, there is a merit that a sensor having a generally distributed resolution can be used. Further, the thickness in the Z direction can be suppressed by the effect of the folding mirror, and a small image reading apparatus can be obtained.

  In the example of FIGS. 22 and 23, the original image of each cell is formed as an inverted image on the corresponding image sensor 141, 142. Therefore, it is necessary to invert the image prior to the combining process. The inversion processing may be performed by assembling a dedicated circuit as preprocessing of the combination processing, or the order in which the image data is stored in the line memory 105 and the order in which the image data is read out from the line memory 105 are reversed. May be performed.

  Here, since the cells 411 and 412 are telecentric optical systems on the document side, as shown in FIG. 23, even if the position of the document changes to 73, 74, and 75, the image transfer magnification with respect to the image sensors 141 and 142 There is a feature that does not change. Further, since the optical axis of each cell 411, 412 and the top plate 103 are perpendicular to the main scanning direction, similarly, the image of each cell 411, 412 does not move in the main scanning direction even if the document position changes. However, with respect to the sub-scanning direction, the optical axis of each cell 411, 412 and the top plate 103 are not perpendicular, so if the position of the document 70 changes to 73, 74, 75, the position where the optical system reads the document shifts. . As a result, the image obtained according to the position of the document moves in the sub-scanning direction.

  At this time, the optical axes of the cell 1 of the series 1 and the cell 412 of the series 2 are inclined in the opposite directions with respect to the sub-scanning direction. Therefore, the direction in which the image moves when the position of the document is changed is also reversed.

  According to the image combining device having the configuration shown in FIGS. 22 and 23, the same function as that of the image combining device according to the first embodiment described above can be realized, and the same effect can be obtained. Further, the image forming apparatus can be thinned.

DESCRIPTION OF SYMBOLS 100: Image reading apparatus, 101: Imaging optical system, 102: Illumination light source, 103: Top plate, 104: Substrate, 141, 142, 143, 144, 145, 146, 147, 148: Image sensor part, 105: Line Memory: 106: Image combiner, 161: Storage unit, 162: Combined position information storage unit, 163: Control unit, 164: Combined position detection unit, 165: Correlation detection region setting unit, 166: Correlated detection region setting Unit, 167: correlation calculation unit, 168: combined position specifying unit, 169: combined position estimating unit, 170: image combining unit.

Claims (20)

First imaging means for acquiring a first image;
A second imaging means for acquiring a second image adjacent to the first image in the main scanning direction;
In a plurality of scanning lines arranged in the sub-scanning direction, a first image group including a plurality of the first images captured by the first imaging unit, and the second image captured by the second imaging unit. Storage means for storing a second image group having a plurality of images;
Control means for calculating and combining a combining position for combining the first image included in the first image group and the image included in the second image group;
The control means includes
In the first image group, a matching region composed of a plurality of the first images acquired from a plurality of scanning lines continuous from the first image at the head position in the scanning line, for which the combined position is not calculated. Correlation detection area setting means for setting
In the second image group, a plurality of sizes different from each other including a matching region in which an image having the same size as the size of the matching region is obtained from the head line in the scanning line where the combined position is not calculated. Correlated detection area setting means for setting a matched area;
Correlation calculating means for calculating a value for specifying a degree of correlation between the matching region and each of the plurality of matching regions;
When the correlation level specified by the value specifying the correlation level indicates a correlation higher than a predetermined correlation level, the matching region having the highest correlation level with the matching region is selected, and the matching is performed. The position in the sub-scanning direction of the image included in the matching area that combines the first images included in the area is specified as the combination position, and the correlation degree specified by the value specifying the correlation degree is In a case where the correlation is not higher than a predetermined correlation degree, a combined position specifying unit that suspends processing for specifying the combined position of the first image included in the matching area;
The combined position specifying means determines the position in the sub-scanning direction of the image included in the second image group, which combines the first images for which the combined position specifying means has suspended the process of specifying the combined position. A combined position estimating means for calculating as the combined position from the combined position of the other first image different from the first image for which the process of specifying the combined position is suspended;
An image combining device comprising: The combination position specifying unit suspends the process of specifying the combination position even when there is no edge portion that can be an edge of an image in the specific image area in the matching area. The image combining device described in 1. The coupling position estimation means includes
The combination position specifying unit and the scan line of the combination position calculated with respect to the first image acquired in the scan line including the first image for which the process of specifying the combination position is suspended. A process of calculating a first deviation amount;
The second calculated for the first image acquired in the scan line one above the scan line including the first image for which the combination position specifying unit has suspended the process of specifying the combination position. Processing to calculate the amount of deviation of
A process of calculating a change amount between the first shift amount and the second shift amount from the first images included in the same first image group;
A change amount corresponding to the change amount calculated from an adjacent image in the main scanning direction with respect to the first image for which the combination position specifying unit has suspended the process of specifying the combination position is specified as the combination position specification. A process in which the means specifies the change amount of the first image for which the process of specifying the combination position is suspended, and calculates the combination position corresponding to the specified change amount;
The image combining device according to claim 1, wherein: The coupling position estimation means includes
When the change amount cannot be calculated from the adjacent image in the main scanning direction with respect to the first image for which the combination position specifying unit has suspended the process of specifying the combination position, the combination position specifying unit For the first image for which the process of specifying the combination position is suspended, a change amount corresponding to the change amount calculated from an image read from the closest position in the main scanning direction is determined as the combination position specification. The image combining apparatus according to claim 3, wherein a change amount of the first image in which processing for specifying the combination position is suspended by a unit is used. The coupling position estimation means includes
When the change amount cannot be calculated from the adjacent image in the main scanning direction with respect to the first image for which the combination position specifying unit has suspended the process of specifying the combination position, the combination position specifying unit The image combination apparatus according to claim 3, wherein there is no change amount of the first image for which the process of specifying the combination position is suspended. The correlation detection area setting means sets the first image at the head position in the scanning line as the attention line among the first images included in the matching area,
The image combination apparatus according to claim 1, wherein the combination position specifying unit calculates the combination position of the first image corresponding to the target line. The binding position specifying means suspends calculation of the binding position when the correlation specified by the value specifying the correlation is less than a predetermined threshold value,
The correlation detection area setting unit is configured to increase the first image included in the matching area and to suspend the calculation of the combined position when the combined position specifying unit has suspended the calculation of the combined position. The first image in the next lower line in the scan line of the first image is the attention line,
The correlated detection area setting means includes a plurality of different sizes including a matched area such that the correlation detection area setting means results in an image having the same size as the size of the matching area obtained by increasing the first image. Set the matching area of
The correlation calculating means calculates a value that specifies the degree of correlation between the matching area and each of the plurality of matched areas.
The image combining device according to claim 6, wherein the processing is repeated until the combined position specifying unit calculates the combined position. The combination position specifying unit suspends calculation of the combination position when there is no edge portion that can be an edge of the image in the specific image region including the target line
The correlation detection area setting unit is configured to increase the first image included in the matching area and to suspend the calculation of the combined position when the combined position specifying unit has suspended the calculation of the combined position. The first image in the next lower line in the scan line of the first image is the attention line,
The correlated detection area setting means includes a plurality of different sizes including a matched area such that the correlation detection area setting means results in an image having the same size as the size of the matching area obtained by increasing the first image. Set the matching area of
The correlation calculating means calculates a value that specifies the degree of correlation between the matching area and each of the plurality of matched areas.
The image combining device according to claim 6 or 7, wherein the processing is repeated until the combined position specifying unit calculates the combined position. If the size of the matching region and the matching region is different, the coupling position specifying unit changes the size of the matching region to the same size as the matching region, and the changed matching target An image having a position and size corresponding to the target line in the region is combined with the target line, and the combined position is determined based on the position and size of the combined image when the changed matching region is returned to the original size. The image combining device according to claim 6, wherein the image combining device is specified. The combination position estimation unit is configured to combine the first image held first when the first number of images for which the combination position specifying unit has suspended calculation of the combination position reaches a predetermined number. The position is calculated. The image combining device according to claim 6, wherein the position is calculated. First imaging means for acquiring a first image;
A second imaging means for acquiring a second image adjacent to the first image in the main scanning direction;
In a plurality of scanning lines arranged in the sub-scanning direction, a first image group including a plurality of the first images captured by the first imaging unit, and the second image captured by the second imaging unit. Storage means for storing a second image group having a plurality of images;
A control unit that calculates and combines a coupling position for coupling the first image included in the first image group and the image included in the second image group is performed. An image combination position calculation method,
The control means includes a plurality of the first images acquired from a plurality of scanning lines continuous from the first image at the head position in the scanning line, in which the combined position is not calculated in the first image group. Correlation detection area setting means for setting a matching area consisting of images;
In the second image group, the control means includes a matching region in which the combined position is not calculated, and the matching region is an image having the same size as the size of the matching region from the head line in the scanning line. A correlated detection area setting processing step for setting a plurality of matched areas of different sizes;
A correlation calculation processing step in which the control means calculates a value specifying a degree of correlation between the matching area and each of the plurality of matching areas;
In the case where the control means shows a correlation higher than a predetermined correlation degree, the matching area having the highest correlation degree with the matching area is selected as the correlation area specified by the value specifying the correlation degree. The position in the sub-scanning direction of the image included in the matching area to be selected and combined with the first image included in the matching area is specified as the combination position, and specified by the value specifying the correlation degree When the correlation degree to be displayed does not indicate a correlation higher than a predetermined correlation degree, a combined position specifying process step for deferring the process of specifying the combined position of the first image included in the matching region When,
The control unit is configured to determine a position in the sub-scanning direction of an image included in the second image group, in which the first image for which the process of specifying the combining position is suspended is combined in the combining position specifying processing step. A combined position estimation process for calculating the combined position from the combined position of the first image different from the first image for which the process of specifying the combined position in the combined position specifying process step is suspended. Steps,
An image combination position calculation method characterized by comprising: The combination position specifying processing step suspends the process of specifying the combination position even when there is no edge portion that can be an edge of an image in the specific image area in the matching area. 11. The image combination position calculation method according to 11. The combined position estimation processing step includes:
The combination position specifying unit and the scan line of the combination position calculated with respect to the first image acquired in the scan line including the first image for which the process of specifying the combination position is suspended. Performing a process of calculating a first deviation amount;
Calculated for the first image acquired in the scan line one above the scan line including the first image for which the process of specifying the combined position is suspended in the combined position specifying process step. Performing a process of calculating a second deviation amount;
Performing a process of calculating a change amount between the first shift amount and the second shift amount from the first images included in the same first image group;
The amount of change corresponding to the amount of change calculated from the adjacent image in the main scanning direction with respect to the first image for which the process of specifying the combining position in the combining position specifying processing step is suspended Performing a process of specifying the change position of the first image for which the process of specifying the combination position in the position specifying process step is suspended and calculating the combination position corresponding to the specified change amount;
The image combining position calculation method according to claim 11 or 12, wherein: The combined position estimation processing step includes:
When the change amount cannot be calculated from the adjacent image in the main scanning direction with respect to the first image for which the process of specifying the combined position in the combined position specifying process step is suspended, the combined position A change amount corresponding to the change amount calculated from the image read from the closest position in the main scanning direction with respect to the first image for which the process of specifying the coupling position in the specifying process step is suspended. The image combination position calculation method according to claim 13, wherein a change amount of the first image for which the process of specifying the combination position is suspended in the combination position specification processing step is used. The combined position estimation processing step includes:
When the change amount cannot be calculated from the adjacent image in the main scanning direction with respect to the first image for which the process of specifying the combined position in the combined position specifying process step is suspended, the combined position The image combination position calculation method according to claim 13, wherein there is no change amount of the first image for which the process of specifying the combination position in the specifying process step is suspended. In the correlation detection region setting processing step, the control means sets the first image at the head position in the scanning line as the attention line among the first images included in the matching region,
The image according to any one of claims 11 to 15, wherein, in the combined position specifying processing step, the control unit calculates the combined position of the first image corresponding to the target line. Bonding position calculation method. In the combined position specifying processing step, the control means suspends calculation of the combined position when the correlation specified by the value specifying the correlation is less than a predetermined threshold,
In the correlation detection region setting processing step, the control means increases the first image included in the matching region when the calculation of the combined position is suspended in the combined position specifying processing step, and The first image of the next lower line in the scan line of the first image for which the position calculation is suspended is set as the attention line,
In the correlated detection region setting processing step, the control unit is configured to match the matching region so that an image having the same size as the size of the matching region obtained by increasing the first image in the correlation detection region setting processing step is obtained. Set multiple matching areas with different sizes, including
In the correlation calculation processing step, the control means calculates a value that specifies a degree of correlation between the matching region and each of the plurality of matching regions.
The image combination position calculation method according to claim 16, wherein the process is repeated until the control unit calculates the combination position in the combination position specification processing step. In the combined position specifying processing step, the control unit suspends calculation of the combined position when there is no edge portion that can be an edge of the image in the specific image area including the target line,
In the correlation detection region setting processing step, the control unit increases the first image included in the matching region when the combined position specifying unit has suspended calculation of the combined position, and the combined position The first image in the line below the scanning line of the first image for which the calculation of
In the correlated detection region setting processing step, the control unit is configured to match the matching region so that an image having the same size as the size of the matching region obtained by increasing the first image in the correlation detection region setting processing step is obtained. Set multiple matching areas with different sizes, including
In the correlation calculation processing step, the control means calculates a value that specifies a degree of correlation between the matching region and each of the plurality of matching regions.
18. The image combination position calculation method according to claim 16, wherein the processing is repeated until the control unit calculates the combination position in the combination position specification processing step. In the coupling position specifying processing step, when the size of the matching region and the matching region is different, the control unit changes the size of the matching region to be the same size as the matching region, An image having a position and size corresponding to the target line in the changed matching area is used as a combined image that is combined with the target line, and the position of the combined image when the changed matching area is returned to the original size and The image combination position calculation method according to any one of claims 16 to 18, wherein the combination position is specified by a size. In the combined position estimation processing step, the control means starts when the number of the first images for which the control means has suspended calculation of the combined position in the combined position specifying processing step reaches a predetermined number. The image combination position calculation method according to any one of claims 16 to 19, wherein the combination position of the first image held on the left is calculated.

Images