JP2013093674A - Image processing apparatus, image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of making a seam of sensor chips to be inconspicuous even when there is variation in output levels between sensor chips.SOLUTION: An image processing apparatus comprises: peripheral pixel change pattern calculation section which obtains a difference of image data between adjacent pixels in a peripheral pixel group including a chip end pixel adjacent to a missing pixel; a comparison pixel change pattern calculation section which obtains a difference of pixel data between adjacent pixels in any comparison pixel group whose position is different from the peripheral pixel group; a change pattern comparison section which compares a change pattern between the peripheral pixel group and the comparison pixel group, and extracts the comparison pixel group whose correlation is highest as a reference pixel group; and a pixel data calculation section which obtains pixel data of a missing pixel based on pixel data corresponding to a chip end pixel in the reference pixel group, pixel data of a pixel adjacent to the reference pixel group and corresponding to the missing pixel, and pixel data of the chip end pixel.

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program. More specifically, the present invention relates to a line by acquiring pixel data of a light receiving element from a line sensor including a plurality of sensor chips each formed with a plurality of light receiving elements. The present invention relates to an improvement in an image processing apparatus that generates a read image.

  A close-contact line image sensor, so-called CIS (Contact Image Sensor), is composed of a plurality of sensor chips arranged in a substantially straight line, and reads a document placed on a document table for one line. It is an optical reading device that sequentially generates images of the minute. Each sensor chip is an image pickup circuit including a one-dimensional array of a large number of light receiving elements, and is mounted on a common substrate. Each sensor chip is connected in the direction in which the light receiving elements are arranged.

  In the current sensor mount technology, if the resolution is high, it is difficult to make the interval between the joint portions of two adjacent sensor chips less than the arrangement pitch of the light receiving elements in the sensor chip. For this reason, there is a problem that the quality of the read image deteriorates at the joint portion of the sensor chip. Therefore, an image processing apparatus has been proposed that generates line read images by interpolating missing pixel data between sensor chips with respect to pixel data read by a line image sensor (for example, Patent Documents 1 and 2).

  Pixel data of missing pixels between sensor chips is obtained by, for example, linear interpolation using pixel data of pixels adjacent to the missing pixels, or pattern matching by luminance comparison between pixel columns. In the method based on linear interpolation, pixel data of missing pixels is obtained by calculating an average value of pixel data for two pixels adjacent to the missing pixels. However, with such a method, although a good interpolation result can be obtained when the read image consists of low frequency components whose luminance changes in a relatively long cycle, the high frequency components whose luminance changes in a short cycle can be obtained. In this case, there is a problem that an appropriate interpolation result cannot be obtained.

  On the other hand, in the method based on pattern matching, for example, for a plurality of comparison blocks that include a comparison target pixel for comparison with a missing pixel and the positions of the comparison target pixels are different, the reference block in which the pixel data in the comparison block includes the missing pixel Is compared with the pixel data within. Pixel data of missing pixels is determined based on a comparison block having the highest correlation of luminance distribution with respect to the reference block. In such a method, even when the read image is composed of high frequency components, a good interpolation result can be obtained. However, due to individual differences among sensor chips, there is a problem that an appropriate interpolation result cannot be obtained when there is a large variation in output level between sensor chips. That is, if there is a large variation in output level between sensor chips, comparison of pixel data between corresponding pixels cannot correctly identify a comparison block having the same luminance change as that of the reference block.

JP 2007-142667 A JP 2010-178174 A

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus capable of improving the image quality of a line-read image generated by interpolating pixel data of missing pixels.

  In particular, the present invention provides an image processing apparatus capable of obtaining a good interpolation result and making the joints of sensor chips less noticeable even when the output level varies greatly between sensor chips due to individual differences. The purpose is that.

  Another object of the present invention is to provide an image processing method used in such an image processing apparatus. Another object of the present invention is to provide an image processing program for causing a computer to function as the image processing apparatus.

  An image processing apparatus according to a first aspect of the present invention acquires pixel data of the light receiving element from a line sensor including two or more sensor chips each having a plurality of light receiving elements formed therein, and extracts missing pixel data between the sensor chips. An image processing apparatus for generating a line-read image by interpolation, a peripheral pixel group including chip end pixels adjacent to missing pixels between the sensor chips and including three or more continuous pixels in the sensor chip The peripheral pixel change pattern calculation unit for obtaining a difference in pixel data between adjacent pixels and obtaining the change pattern of the difference is a comparison pixel group having a position different from that of the peripheral pixel group, and three or more consecutive pixels. In an arbitrary comparison pixel group consisting of the pixels, a difference in pixel data between adjacent pixels is obtained, A comparison pixel change pattern calculation unit for obtaining a change pattern of the difference, a change pattern of the peripheral pixel group, and a change pattern of two or more comparison pixel groups having different positions from each other, and for the peripheral pixel group, A change pattern comparison unit that extracts the comparison pixel group having the highest correlation of change patterns as a reference pixel group, pixel data of a first pixel corresponding to the chip end pixel in the reference pixel group, and adjacent to the reference pixel group And a pixel data calculation unit that obtains pixel data of the missing pixel based on pixel data of the second pixel corresponding to the missing pixel and pixel data of the chip end pixel. The

  In this image processing apparatus, pixel data of a missing pixel is obtained by extracting a comparison pixel group having the highest correlation of change patterns with respect to a neighboring pixel group adjacent to the missing pixel. Since the change pattern is obtained from the difference in pixel data between adjacent pixels in the pixel group and the peripheral pixel group and the comparison pixel group are compared, the output level varies greatly between sensor chips due to individual differences. Even if it exists, a comparison pixel group can be extracted appropriately. That is, since the comparison pixel group having the highest correlation of the change pattern with respect to the peripheral pixel group is correctly specified, a good interpolation result can be obtained, and the joint of the sensor chip can be made inconspicuous.

  In addition to the above configuration, the image processing apparatus according to the second aspect of the present invention is configured such that the change pattern comparison unit extracts, as the reference pixel group, the comparison pixel group whose change pattern matches the peripheral pixel group. . According to such a configuration, a comparison pixel group having the same luminance change pattern as that of the peripheral pixel group can be correctly identified.

  In addition to the above-described configuration, the image processing apparatus according to a third aspect of the present invention is arranged so that the pixel data calculation unit sandwiches the missing pixel when the comparison pixel group whose change pattern matches the peripheral pixel group does not exist. The pixel data of the missing pixel is determined based on the pixel data of the first chip end pixel and the second chip end pixel. According to such a configuration, a good interpolation result can be obtained even if a comparison pixel group having the same luminance change as that of the peripheral pixel group cannot be specified.

  In addition to the above-described configuration, the image processing apparatus according to a fourth aspect of the present invention is configured so that the pixel data calculation unit indicates a change amount of pixel data obtained from the pixel data of the first pixel and the pixel data of the second pixel. The pixel data of the missing pixel is obtained by adding to the pixel data of the end pixel. According to such a configuration, the configuration for obtaining the pixel data of the missing pixel from the pixel data of the first pixel in the reference pixel group, the pixel data of the second pixel adjacent to the reference pixel group, and the pixel data of the chip end pixel is simplified. Can be

  In the image processing apparatus according to the fifth aspect of the present invention, in addition to the above-described configuration, the peripheral pixel change pattern calculation unit assigns the first peripheral pixel group and the second peripheral pixel group arranged with the missing pixel interposed therebetween. A first comparison pixel group and a second comparison pixel that are arranged as a group, the change pattern is obtained for each peripheral pixel group, and the comparison pixel change pattern calculation unit is arranged with a comparison target pixel for comparison with the missing pixel. Each group is set as the comparison pixel group, the change pattern is obtained for each comparison pixel group, and the change pattern comparison unit is different in the change pattern of the first and second peripheral pixel groups from the position of the comparison target pixel. The first and second peripheral pixel groups are compared with the change patterns of the first and second comparison pixel groups. On the other hand, the first and second comparison pixel groups having the highest correlation of the change patterns are extracted as the first and second reference pixel groups, and the pixel data calculation unit calculates the missing pixels based on the first reference pixel group. Obtaining first pixel data, obtaining second pixel data of the missing pixel based on the second reference pixel group, and obtaining pixel data of the missing pixel based on the first and second pixel data. Composed.

  According to such a configuration, the first and second comparison pixel groups having the highest correlation of the change patterns are extracted from the first and second peripheral pixel groups arranged with the missing pixel in between, and the pixel of the missing pixel is extracted. Since the data is obtained, the accuracy of specifying the luminance of the missing pixel can be improved.

  According to a sixth aspect of the present invention, there is provided an image processing method comprising: pixel data missing between the sensor chips with respect to pixel data of the light receiving element acquired from a line sensor including two or more sensor chips each having a plurality of light receiving elements formed therein. Image processing method for generating a line-read image by interpolating a pixel, a peripheral pixel including a chip end pixel adjacent to a missing pixel between the sensor chips and consisting of three or more continuous pixels in the sensor chip In the group, a neighboring pixel change pattern calculating step for obtaining a difference in pixel data between adjacent pixels and obtaining a change pattern of the difference is a comparison pixel group having a position different from that of the neighboring pixel group. In any comparison pixel group consisting of the above pixels, the pixel data between adjacent pixels A comparison pixel change pattern calculation step for obtaining a difference pattern, and a change pattern of the peripheral pixel group, and a change pattern of two or more of the comparison pixel groups having different positions from each other; A change pattern comparison step for extracting, as a reference pixel group, the comparison pixel group having the highest correlation of change patterns for the group, pixel data of the first pixel corresponding to the chip end pixel in the reference pixel group, the reference A pixel data calculating step for obtaining pixel data of the missing pixel based on pixel data of a second pixel corresponding to the missing pixel and pixel data of the chip end pixel, which are pixels adjacent to the pixel group; It is prepared for.

  According to a seventh aspect of the present invention, there is provided an image processing program comprising: pixel data missing between sensor chips with respect to pixel data of the light receiving elements acquired from a line sensor including two or more sensor chips each having a plurality of light receiving elements. Is an image processing program for causing a computer to execute image processing for generating a line-read image by interpolating a chip end pixel adjacent to a missing pixel between the sensor chips, and continuous in the sensor chip In a peripheral pixel group composed of three or more pixels, the pixel data difference between adjacent pixels is obtained, and the peripheral pixel change pattern calculation procedure for obtaining the change pattern of the difference is compared with the peripheral pixel group at different positions. An arbitrary comparison pixel group consisting of three or more consecutive pixels. The comparison pixel change pattern calculation procedure for obtaining a difference in pixel data between adjacent pixels and obtaining the change pattern of the difference, the change pattern of the peripheral pixel group, and two or more of the comparison pixel groups having different positions A change pattern comparison procedure for extracting, as a reference pixel group, the comparison pixel group having the highest correlation of the change pattern with respect to the peripheral pixel group, and the chip end pixels in the reference pixel group. Based on the pixel data of the corresponding first pixel, the pixel adjacent to the reference pixel group, the pixel data of the second pixel corresponding to the missing pixel, and the pixel data of the chip end pixel, the missing pixel And a pixel data calculation procedure for obtaining the pixel data.

  In the image processing apparatus according to the present invention, since the comparison pixel group having the highest correlation of the change pattern with respect to the peripheral pixel group is correctly specified, the output level varies greatly between the sensor chips due to individual differences. Also, a good interpolation result can be obtained. Therefore, the joint of the sensor chip can be made inconspicuous, and the image quality of the line read image generated by interpolating the pixel data of the missing pixel can be improved.

  Further, according to the present invention, it is possible to provide an image processing method used in the image processing apparatus as described above. Further, according to the present invention, it is possible to provide an image processing program for causing a computer to function as the image processing apparatus.

1 is a block diagram illustrating a configuration example of an image reading system 100 including an image processing apparatus 20 according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a configuration example of a change pattern calculation unit 25 in the image processing apparatus 20 of FIG. 1. It is explanatory drawing which showed typically the structural example of the line image sensor 10 of FIG. 1, and the cut surface at the time of cut | disconnecting the line image sensor 10 by the AA cutting line is shown. 2 is a diagram illustrating an example of a line read image generated from pixel data read by a line image sensor. FIG. Across the missing pixel p ₁ is a diagram showing a variation in output level between the sensor chip 11 are arranged. It is the figure which showed an example of operation | movement of the image processing apparatus 20 of FIG. 1, and a mode that the luminance change pattern is compared between a surrounding pixel group and a comparison pixel group is shown. It is the figure which showed an example of operation | movement of the image processing apparatus 20 of FIG. 1, and the pixel data input sequentially in synchronization with a clock signal are shown.

<Image reading system>
FIG. 1 is a block diagram showing a configuration example of an image reading system 100 including an image processing apparatus 20 according to an embodiment of the present invention. The image reading system 100 includes a line image sensor 10 having a plurality of sensor chips 11 and an image processing device 20, and interpolates missing pixel data between sensor chips 11 with respect to pixel data read from a document. A line read image is generated.

  The sensor chip 11 is an imaging circuit in which a large number of light receiving elements are arranged in a substantially linear shape. The sensor chip 11 is a solid-state imaging device such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. A PD (photodiode) is used as the light receiving element.

  The line image sensor 10 is a line array type optical reading device in which a plurality of sensor chips 11 are connected in the arrangement direction of light receiving elements. For example, the line image sensor 10 reads a document placed on a document table and corresponds to the amount of received light. Pixel data indicating a luminance value is generated. In the line image sensor 10, the sensor chips 11 are connected in series, for example, and pixel data generated for each light receiving element is sequentially output in the order of arrangement of the light receiving elements.

The image processing device 20 acquires pixel data from the line image sensor 10 and interpolates pixel data of the missing pixel p ₁ between the sensor chips 11 to generate a read image for one line, that is, a line read image. . The missing pixel p ₁ is a virtual pixel in which a corresponding light receiving element does not actually exist on the line image sensor 10, and one missing pixel p ₁ is virtually arranged between the sensor chips 11.

  In document reading, if the arrangement direction of the light receiving elements is referred to as the main scanning direction, the line image sensor is moved in the sub scanning direction that intersects the main scanning direction every time reading of one line is completed in the main scanning direction. 10 or the original is moved. By repeating such a reading operation for one line, an original image for one page composed of a large number of line reading images can be obtained.

  The image processing apparatus 20 includes a pixel data storage unit 21, a pixel data insertion unit 22, a shift register 23, pixel data calculation units 24 and 27, a change pattern calculation unit 25, a change pattern comparison unit 26, an interpolation timing signal generation unit 28, A delay circuit 29 and a selector circuit 30 are included. The image processing apparatus 20 is configured by a programmable logic circuit such as an FPGA (Field-Programmable Gate Array).

The pixel data storage unit 21 is a buffer memory for temporarily holding pixel data acquired from the line image sensor 10. The interpolation timing signal generation unit 28 generates an interpolation timing signal that defines the timing for interpolating the pixel data of the missing pixel p ₁ and outputs the interpolation timing signal to the pixel data insertion unit 22 and the delay circuit 29.

The pixel data insertion unit 22 reads out pixel data from the pixel data storage unit 21 and sequentially writes the pixel data in the shift register 23, and secures a data area for missing pixels based on the interpolation timing signal. Specifically, predetermined dummy data is inserted as pixel data of the missing pixel p ₁ in a data string of pixel data input / output in synchronization with a predetermined clock signal.

  The shift register 23 shifts the data string of the pixel data every time pixel data is written by the pixel data insertion unit 22, and the pixel data is transmitted to the chips of the selector circuit 30 and the pixel data calculation units 24 and 27 at a predetermined timing. Is output.

Pixel data calculation unit 24, by linear interpolation using pixel data of pixels adjacent to a missing pixel p _1, obtains the pixel data of the missing pixel p _1, and outputs it to the selector circuit 30. Pixels adjacent to the missing pixel p ₁ is a chip end pixels in the sensor chip 11 (described later), the pixel data of the missing pixel p _1, the two that sandwich a missing pixel p ₁ of the chip edge pixel p ₂ by obtaining the average value of the pixel data between the p _3, it is calculated.

The pixel data calculation unit 27 calculates the missing pixel p _{1 based} on the pixel data of the chip end pixels p ₂ and p ₃ adjacent to the missing pixel p ₁ and the luminance change amount between the comparison target pixel and the adjacent pixels described later. Pixel data is obtained and output to the selector circuit 30.

The change pattern calculation unit 25 determines a peripheral pixel group and a plurality of comparison pixel groups whose positions are different from each other for the data string of the pixel data input from the pixel data insertion unit 22, and for each pixel group, A luminance change pattern related to the pixel is obtained. The peripheral pixel group is a pixel group that includes a chip end pixel adjacent to the missing pixel p ₁ and includes three or more pixels that are continuous in the sensor chip 11.

The comparison pixel group is a pixel group having a position different from that of the peripheral pixel group, and is an arbitrary pixel group including three or more continuous pixels. In the change pattern calculation unit 25, the comparison target pixel q ₁ and the adjacent pixels are compared based on the difference between the pixel data of the comparison target pixel q ₁ for comparison with the missing pixel p ₁ and the pixel data of the pixels adjacent to the comparison target pixel q _1. A luminance change amount between pixels is obtained.

The change pattern comparison unit compares the change pattern of the peripheral pixel group with the change patterns of a plurality of comparison pixel groups having different positions, and refers to the comparison pixel group having the highest correlation of the change patterns with respect to the peripheral pixel group. Extract as a pixel group. The change pattern comparison unit 26 repeatedly compares the change pattern of the comparison pixel group with the change pattern of the peripheral pixel group every time the position of the comparison target pixel q ₁ is changed by one pixel, and the comparison result is selected by the selector. It is output to the circuit 30.

The change pattern comparison unit 26 extracts a comparison pixel group whose change pattern matches the peripheral pixel group as a reference pixel group, and notifies the pixel data calculation unit 27 of the position of the comparison target pixel q ₁ adjacent to the reference pixel group. Is done. In the pixel data calculating unit 27 is determined from the comparison target pixel q ₁ of pixel data, compares a pixel adjacent to the target pixel q _1, and pixel data of pixels corresponding to the tip end pixel in the reference pixel group based on the amount of change in brightness, the pixel data missing pixel p ₁ is calculated.

  The delay circuit 29 is a delay time adjustment unit that delays the interpolation timing signal input from the interpolation timing signal generation unit 28 by a predetermined time and outputs the delayed signal to the selector circuit 30. The delay time for delaying the interpolation timing signal is determined in advance based on the time required for each process in the pixel data calculation units 24 and 27, the change pattern calculation unit 25, and the change pattern comparison unit 26.

  The selector circuit 30 is one of the pixel data input from the shift register 23 and the pixel data calculation units 24 and 27 based on the interpolation timing signal input from the delay circuit 29 and the comparison result of the change pattern comparison unit 26. Is selected and output as pixel data constituting a line-read image.

Specifically, pixel data input from the shift register 23 is output as it is to pixels other than the missing pixel p ₁ , and pixels that are input from the pixel data calculation unit 24 or 27 to the missing pixel p _1. Data is output.

At the timing of the pixel data missing pixel p ₁ is input, if the comparative pixel group change pattern matches the surrounding pixel group exists, the pixel data input from the pixel data calculating unit 27 is outputted. That is, pixel data obtained from the pixel data of the chip end pixels p ₂ and p ₃ and the luminance change amount between the comparison target pixel q ₁ and the adjacent pixels is output.

On the other hand, when there is no comparison pixel group whose change pattern matches the peripheral pixel group, the pixel data input from the pixel data calculation unit 24 is output. That is, an average value of pixel data between the chip end pixels p ₂ and p ₃ is output.

  FIG. 2 is a block diagram illustrating a configuration example of the change pattern calculation unit 25 in the image processing apparatus 20 of FIG. The change pattern calculation unit 25 includes a neighboring pixel change pattern calculation unit 251, a comparison pixel change pattern calculation unit 252, and a change amount calculation unit 253.

The peripheral pixel change pattern calculation unit 251 calculates a luminance change amount from the difference in pixel data between adjacent pixels for a peripheral pixel group including chip edge pixels p ₂ and p ₃ adjacent to the missing pixel p _1, and performs the main scanning direction A luminance change pattern related to the pixel position is calculated. Specifically, a first peripheral pixel group and the second peripheral pixel group that sandwich the missing pixel p ₁ and the peripheral pixel group, respectively, for each peripheral pixel group, the luminance change pattern is calculated.

The first peripheral pixel group, a peripheral pixel group disposed on one side of the line image sensor 10 than the missing pixel p _1, the second peripheral pixel group, the other end of the line image sensor 10 than the missing pixel p ₁ This is a peripheral pixel group arranged on the side. The chip edge pixel p ₂ is a chip end pixels in the first neighborhood pixel group, for example, pixel data is input prior to the missing pixel p _1. On the other hand, the chip edge pixel p ₃ is a chip end pixel in the second peripheral pixel group, the pixel data is input later than the missing pixel p _1.

Comparative pixel change pattern calculation section 252 includes a pixel q _2, q ₃ adjacent to the comparison pixel q _1, the plurality of different comparative pixel group the position of comparison target pixel q _1, pixel data between adjacent pixels A luminance change amount is obtained from the difference, and a luminance change pattern relating to the pixel position in the main scanning direction is calculated. Specifically, the first comparative pixel group and the second comparative pixel group that sandwich the comparison pixel q ₁ and each comparative pixel group, for each comparative pixel group, the luminance change pattern is calculated.

The first comparative pixel group, compared than pixel q ₁ is a comparative pixel group disposed on one side of the line image sensor 10, the second comparative pixel group, the line image sensor 10 than compared pixel q ₁ This is a comparison pixel group arranged on the other end side. The pixel q ₂ is a pixel corresponding to the chip end pixel p ₂ in the first comparison pixel group, and pixel data is input before the comparison target pixel q ₁ . On the other hand, the pixel q ₃ is a pixel corresponding to the chip end pixel p ₃ in the second comparison pixel group, and pixel data is input after the comparison target pixel q ₁ .

  Here, it is assumed that the first peripheral pixel group and the second peripheral pixel group have the same number of pixels, and the first comparison pixel group and the second comparison pixel group also have the same number of pixels. Further, each peripheral pixel group and each comparison pixel group have the same number of pixels.

Change amount calculation unit 253 calculates a brightness variation from the difference in pixel data between the pixels q _2, q ₃ adjacent to the comparison pixel q ₁ and compared pixel q _1. Specifically, the pixel data of the comparison object pixel q _1, first luminance change amount is calculated from the difference between the pixel q ₂ of the pixel data, and the pixel data of the comparison object pixel q _1, the pixel q ₃ A second luminance change amount is calculated from the difference from the pixel data.

In changing the pattern comparing section 26, a plurality of pairs of first and second comparative pixel group the position of comparison target pixel q ₁ are different from each other, and the change pattern of the first and second peripheral pixel group, the first and second comparative The change pattern of the pixel group is compared. Then, the first and second comparison pixel groups having the highest correlation of change patterns are extracted as the first and second reference pixel groups with respect to the first and second peripheral pixel groups.

  Here, the first and second comparison pixel groups whose luminance change patterns match the first and second peripheral pixel groups are extracted as reference pixel groups. Whether or not the change patterns match is related to the pixel columns that make up the pixel group, by comparing the amount of change in luminance between corresponding pixels, and whether or not the amount of change in luminance is generally the same for all pixels in the pixel group Determined based on whether or not.

The pixel data calculation unit 27 obtains the first pixel data of the missing pixel p ₁ based on the first reference pixel group, and obtains the second pixel data of the missing pixel p ₁ based on the second reference pixel group. Then, based on the first and second pixel data, the pixel data missing pixel p ₁ is calculated.

The first pixel data is obtained from the pixel data of the comparison target pixel q ₁ sandwiched between the first reference pixel group and the second reference pixel group and the pixel data of the pixel q ₂ adjacent to the comparison target pixel q _1. The obtained first luminance change amount is obtained by adding to the pixel data of the chip end pixel p ₂ adjacent to the missing pixel p ₁ .

On the other hand, the second pixel data includes the second luminance change amount obtained from the pixel data of the comparison target pixel q _{1 and} the pixel data of the pixel q ₃ adjacent to the comparison target pixel q ₁ adjacent to the missing pixel p ₁ . by adding the pixel data of the chip edge pixel p ₃ to be obtained. Pixel data of the missing pixel p _1, by obtaining the average value of the first pixel data and second pixel data are calculated.

The first and second comparison pixel groups may be specified across three or more sensor chips 11, but here, in order to suppress an increase in processing load, the sensor chip to which the first peripheral pixel group belongs 11 and the sensor chip 11 to which the second peripheral pixel group belongs. When a plurality of missing pixels p ₁ having different positions exist in the line image sensor 10, a peripheral pixel group and a comparison pixel group are determined for each of these missing pixels p ₁ , and the luminance change patterns are compared. Is done.

<Line image sensor 10>
FIG. 3 is an explanatory diagram schematically showing a configuration example of the line image sensor 10 of FIG. 1, and shows a cut surface when the line image sensor 10 is cut along an AA cutting line. The sensor chip 11 includes a semiconductor substrate 112 on which a large number of light receiving elements 111 are formed. Each light receiving element 111 are arranged along a straight line in the main scanning direction at a constant arrangement pitch d _1.

  The line image sensor 10 is configured by arranging a plurality of sensor chips 11 on a common substrate 12. The sensor chips 11 are arranged along a straight line in the main scanning direction at a constant interval without overlapping each other.

The interval of the joint portion between the two adjacent sensor chips 11, that is, the interval d ₂ between the chip end pixels corresponding to the light receiving elements 111 formed at the end of the sensor chip 11, is compared with the arrangement pitch d _1. large. In the process of interpolating the pixel data of missing pixels, pixel data for one pixel is inserted as pixel data that is missing between such sensor chips 11. For this reason, each sensor chip 11 is arranged so that the relational expression d ₂ = d ₁ × 2 is established.

FIG. 4 is a diagram illustrating an example of a line read image generated from the pixel data read by the line image sensor 10, and missing pixels are obtained by linear interpolation using the pixel data of the chip end pixels p ₂ and p _3. when obtaining the pixel data of the p ₁ is shown. (A) in the figure shows the case where the image is composed of low frequency components, and (b) shows the case where the image is composed of high frequency components.

When the line-read image is composed of low frequency components, the luminance changes slowly. Therefore, the average value of the pixel data between the _two chip end pixels p ₂ and p ₃ arranged with the missing pixel p ₁ interposed therebetween is determined as the missing pixel. as pixel data is also p _1, the deviation with respect to the waveform of the low frequency component is relatively small.

On the other hand, when the line read image is composed of high frequency components, the luminance changes drastically. Therefore, if the average value of the pixel data between the chip end pixels p ₂ and p _{3 is} the pixel data of the missing pixel p ₁ , depending on the position of the missing pixel p _1, the deviation becomes larger with respect to the waveform of the high frequency components.

In the image processing apparatus 20 according to this embodiment, to the peripheral pixel group adjacent to the missing pixel p _1, missing pixel p ₁ of the pixel data is obtained by extracting the comparative pixel group brightness variation pattern matches. In general, in the case of a read image having a high frequency characteristic, a comparison pixel group whose luminance change pattern matches a peripheral pixel group always exists in one line. Since such a comparison pixel group is extracted and pixel data is interpolated, a good interpolation result can be obtained even when the line read image is composed of high frequency components.

Figure 5 is a diagram showing a variation in output level between the sensor chip 11 that sandwich the missing pixel p _1. There is a variation in output level between the sensor chips 11 arranged across the missing pixel p ₁ due to individual differences of the sensor chips 11.

For example, a sensor chip 11 includes a chip edge pixel p _2, when the sensor chip 11 includes a chip edge pixel p ₃ has read the document of the same luminance, the output range of the one of the sensor chip 11 is L _1, the other The output range of the sensor chip 11 is L ₂ (L ₂ = L ₁ ). A gap having a luminance difference L ₃ exists between the sensor chips 11 at the output level.

If there is variation such as described above to the output level of the sensor chip 11, in the conventional method of comparing the luminance distributions about pixel column it is possible to identify the pixel columns of the same brightness change as the pixel row including the missing pixel p ₁ correctly Can not.

In the image processing apparatus 20 according to the present embodiment, since the luminance change pattern is obtained and the peripheral pixel group and the comparison pixel group are compared, the missing pixel p ₁ is detected even if the output level of the sensor chip 11 varies greatly. It is possible to correctly specify a pixel column having the same luminance change as the pixel column to be included.

FIG. 6 is a diagram illustrating an example of the operation of the image processing apparatus 20 in FIG. 1, and illustrates how luminance change patterns are compared between the peripheral pixel group and the comparison pixel group. Peripheral pixel group G11, G12 is a peripheral pixel group that sandwich the missing pixel _{p 1.} Peripheral pixel group G11 includes a tip end pixel p ₂ disposed adjacent to the missing pixel p _1, the peripheral pixel group G12 may include chip edge pixel p ₃ disposed adjacent to the missing pixel p ₁ Yes.

On the other hand, comparative pixel group G21, G22 is a comparative pixel group that sandwich any comparison pixel q _1. Comparative pixel group G21 includes a pixel q ₂ disposed adjacent to Compare pixel q _1, comparative pixel group G22 includes pixel q ₃ disposed adjacent to the comparison target pixel q _1.

In the process of comparing the luminance change patterns, the change pattern of the comparison pixel group G21 is compared with the change pattern of the peripheral pixel group G11, and the change pattern of the comparison pixel group G22 is compared with the change pattern of the peripheral pixel group G12. The pattern matching process compares performed while at different positions of the target pixel q _1, the change pattern is compared pixel group G21, G22 to match the surrounding pixel group G11, G12 are extracted as the reference pixel group.

The pixel data of the missing pixel p ₁ includes the pixel data of the chip end pixels p ₂ and p ₃ , the pixel data of the pixels q ₂ and q _{3 in} the comparison pixel groups G 21 and G 22 extracted as the reference pixel group, and the comparison obtained from the comparison target pixel _{q 1} of the pixel data between the pixel groups G21, G22. That is, the luminance change amount between the pixels q ₁ and q ₂ is added to the pixel data of the chip end pixel p ₂ to obtain the first pixel data, and the luminance change amount between the pixels q ₁ and q ₃ is calculated as the chip end pixel p. The second pixel data is obtained by adding to the pixel data of ₃ . Then, by obtaining the average value of the first pixel data and second pixel data, the pixel data is obtained missing pixel p _1.

FIG. 7 is a diagram illustrating an example of the operation of the image processing apparatus 20 of FIG. 1, and illustrates pixel data that is sequentially input in synchronization with the clock signal. The interpolation of the pixel data, first, with respect to pixel data sequentially input in synchronization clock signal, based on the missing pixel p _1, the peripheral pixel group G11, G12 are determined. That is, the peripheral pixel group G11 is a pixel position of the missing pixel p ₁ is n, if indicated pixel data and a (n), the pixel data of each pixel in the surrounding pixel group G11 is, from the pixel data a (n) Is a peripheral pixel group input before. The pixel data of each pixel in the surrounding pixel group G11 is missing pixel _{p 1} pixel data sequentially toward the near a (n + 1) is expressed as ~a (n + 8).

The peripheral pixel group G12 is a peripheral pixel group in which pixel data of each pixel in the peripheral pixel group G12 is input after the pixel data a (n). For each pixel in the surrounding in the pixel group G12 is also the pixel data missing pixel _p pixel data a in order from closer to _{1 (n-1)} is expressed as ~a (n-8).

The change pattern of the peripheral pixel group G11 is obtained by obtaining the luminance change amount b (n + k) (k = 1 to 7) between adjacent pixels for each pixel in the peripheral pixel group G11. The luminance change amount b (n + k) is obtained by the following equation (1).
b (n + k) = a (n + k) −a (n + k + 1) (1)

The change pattern of the peripheral pixel group G12 is also obtained by obtaining the luminance change amount b (nk) (k = 1 to 7) between adjacent pixels, as in the case of the peripheral pixel group G11. The luminance change amount b (n−k) is obtained by the following equation (2).
b (nk) = a (nk-1) -a (nk) (2)

For comparative pixel group G21, G22, similarly to the case of the peripheral pixel group G11, G12, based on the comparison pixel _{q 1,} is determined. The pixel data of each pixel in the comparative pixel group G21 may compare the pixel position of the target pixel q ₁ and m, if indicated pixel data and c (m), the pixel data in the order from the side closer to the comparison pixel q ₁ c (m + 1) to c (m + 8).

The change pattern of the comparison pixel group G21 is obtained by obtaining the luminance change amount d (m + k) (k = 1 to 7) between adjacent pixels for each pixel in the comparison pixel group G21. The luminance change amount d (m + k) is obtained by the following equation (3).
d (m + k) = c (m + k) -c (m + k + 1) (3)

On the other hand, pixel data of each pixel in the comparative pixel group G22 is compared pixel _{q 1} pixel data in order from closer to c (m-1) is expressed as ~c (m-8). The change pattern of the comparison pixel group G22 is also obtained by obtaining the luminance change amount d (m−k) (k = 1 to 7) between adjacent pixels, as in the case of the comparison pixel group G21. The luminance change amount d (m−k) is obtained by the following equation (4).
d (m−k) = c (m−k−1) −c (m−k) (4)

The luminance variation between pixels _q 2, _{q 3} adjacent to the comparison pixel _{q 1} the compared pixel _{q 1} Toko e (m), f (m) is obtained by the following equation (5).
e (m) = c (m) -c (m + 1), f (m) = c (m-1) -c (m) (5)

  Next, the change patterns of the peripheral pixel groups G11 and G12 are compared with the change patterns of the comparison pixel groups G21 and G22, and a comparison pixel group whose luminance change pattern matches the peripheral pixel groups G11 and G12 is used as a reference pixel group. Extracted.

  The change pattern comparison method includes luminance change amounts b (n + 7) to b (n + 1) and b (n−1) to b (n−7) of the peripheral pixel groups G11 and G12 and luminances of the comparison pixel groups G21 and G22. This is performed by comparing the change amounts d (m + 7) to d (m + 1) and d (m−1) to d (m−7).

  In the reference pixel group, for all the pixels in the comparison pixel groups G21 and G22, the luminance change amount is within a range in which the error in the luminance change amount between corresponding pixels is equal to or less than a predetermined determination threshold value. Is a comparison pixel group that matches.

Missing pixel _{p 1} pixel data a (n), the chip edge pixels adjacent to the missing pixel _{p 1 p} 2, _{p 3} pixel data a (n + 1), and a (n-1), the luminance variation e (m ), F (m) and the following equation (6).
a (n) = {[a (n + 1) + e (m)] + [a (n-1) -f (m)]} / 2 (6)

In the above equation (6), the change amount e (m) is added to the pixel data a (n + 1) of the chip end pixel p ₂ to obtain the first pixel data, and the pixel data a (n− of the chip end pixel p ₃ is obtained. The second pixel data is obtained by subtracting the change amount f (m) from 1), and the average value of the first and second pixel data is obtained. With this configuration, the pixel data of the pixels q ₂ and q _{3 in} the reference pixel group, the pixel data of the comparison target pixel q ₁ adjacent to the reference pixel group, and the pixel data of the chip end pixels p ₂ and p ₃ are missing. The configuration for obtaining the pixel data of the pixel p ₁ can be simplified.

On the other hand, if there is no comparison pixel group whose luminance change pattern matches the peripheral pixel groups G11 and G12, the pixel data a (n + 1) and a (n-1) of the chip end pixels p ₂ and p ₃ are used. by linear interpolation, the missing pixel p ₁ pixel data a (n) is calculated. That is, the pixel data a (n) of the missing pixel p ₁ is obtained by the following equation (7).
a (n) = {a (n + 1) + a (n-1)} / 2 (7)

  According to the present embodiment, since the change pattern is obtained from the difference in pixel data between adjacent pixels in the pixel group and the peripheral pixel group and the comparison pixel group are compared, between the sensor chips 11 due to individual differences. Even when there is a large variation in the output level, the comparison pixel group can be appropriately extracted. In particular, a comparison pixel group having the same luminance change pattern as that of the peripheral pixel group can be correctly identified. Therefore, a good interpolation result can be obtained and the joint of the sensor chip 11 can be made inconspicuous.

When there is no comparison pixel group whose luminance change pattern matches the peripheral pixel group, the missing pixel is obtained by linear interpolation using the pixel data of the chip end pixels p ₂ and p ₃ adjacent to the missing pixel p _1. the pixel data is calculated p _1. Therefore, even if a comparison pixel group having the same luminance change as that of the peripheral pixel group cannot be specified, a good interpolation result can be obtained.

Further, to the peripheral pixel group G11, G12 are disposed to sandwich the missing pixel p _1, the correlation of the change pattern is determined with the highest comparative pixel group G21, G22 missing pixel p ₁ of the pixel data by extracting, missing The accuracy of specifying the luminance of the pixel p ₁ can be improved.

In this embodiment, for all the pixels in the pixel group, a comparison pixel group in which the luminance change amount matches the peripheral pixel group within a range where the error in the luminance change amount between the corresponding pixels is equal to or less than a certain value. It has been described in the case of extracting and obtaining the pixel data of the missing pixel p _1. However, the present invention is not limited to this method of identifying a comparative pixel group, the highest comparative pixel data of the missing pixel p ₁ by extracting the pixel group correlation change pattern relative to the peripheral pixel group Other configurations may be used as long as desired. For example, for all pixels in the pixel group, an error in luminance change amount between corresponding pixels is obtained, and a comparison pixel group that minimizes the mean square of the error is extracted as a reference pixel group having the highest correlation of change patterns. Such a statistical method is also included in the present invention.

  Further, in the present embodiment, an example has been described in which a comparison pixel group whose luminance change amount matches the peripheral pixel group is extracted as a reference pixel group for all pixels in the pixel group. However, the present invention also includes extracting a comparison pixel group having a luminance change amount that matches a peripheral pixel group as a reference pixel group for a certain number of pixels or more.

Further, in this embodiment, the change pattern of the surrounding pixel group G11 and G12 that sandwich the missing pixel p _1, and the change pattern of the comparative pixel group G21 and G22 are arranged across the compared pixel q ₁ It has been described in the case of obtaining the pixel data of the missing pixel p ₁ by comparing the. However, for one of the surrounding pixel group G11 or G12 only, also included in the present invention seeks to pixel data of the missing pixel p ₁ by comparing the change patterns and comparative pixel group G21 or G22.

100 Image Reading System 10 Line Image Sensor 11 Sensor Chip 111 Photodetector 112 Semiconductor Substrate 12 Common Substrate 20 Image Processing Device 21 Pixel Data Storage Unit 22 Pixel Data Insertion Unit 23 Shift Registers 24 and 27 Pixel Data Calculation Unit 25 Change Pattern Calculation Unit 251 near the pixel change pattern calculation section 252 compares the pixel change pattern calculation section 253 change calculator 26 changes the pattern comparing section 28 interpolation timing signal generator 29 delay circuit 30 selector circuit G11, G12 surrounding pixel group G21, G22 comparative pixel group _{p 1} missing Pixel p ₂ , p ₃ chip end pixel q ₁ Comparison pixel q ₂ , pixel corresponding to q ₃ chip end pixel

Claims (7)

Image processing for obtaining pixel data of the light receiving element from a line sensor including two or more sensor chips each formed with a large number of light receiving elements and interpolating missing pixel data between the sensor chips to generate a line read image In the device
In a peripheral pixel group including three or more consecutive pixels in the sensor chip including the chip end pixels adjacent to the missing pixels between the sensor chips, a difference in pixel data between adjacent pixels is obtained, A surrounding pixel change pattern calculation unit for obtaining a change pattern of the difference;
A comparison pixel group having a position different from that of the peripheral pixel group, and in any comparison pixel group including three or more consecutive pixels, a difference in pixel data between adjacent pixels is obtained, and a change pattern of the difference is obtained. A comparison pixel change pattern calculation unit to be obtained;
The change pattern of the peripheral pixel group is compared with the change patterns of two or more comparison pixel groups having different positions, and the comparison pixel group having the highest correlation of the change pattern with respect to the peripheral pixel group is referred to as a reference pixel group. A change pattern comparison unit extracted as
Pixel data of a first pixel corresponding to the chip end pixel in the reference pixel group, pixel data of a second pixel corresponding to the missing pixel and adjacent to the reference pixel group, and the chip end An image processing apparatus comprising: a pixel data calculation unit that obtains pixel data of the missing pixel based on pixel data of the pixel.   The image processing apparatus according to claim 1, wherein the change pattern comparison unit extracts the comparison pixel group whose change pattern matches the peripheral pixel group as the reference pixel group.   The pixel data calculation unit includes each pixel of the first chip end pixel and the second chip end pixel arranged with the missing pixel interposed therebetween when the comparison pixel group whose change pattern matches the peripheral pixel group does not exist. The image processing apparatus according to claim 2, wherein pixel data of the missing pixel is obtained based on the data.   The pixel data calculation unit adds the amount of change in pixel data obtained from the pixel data of the first pixel and the pixel data of the second pixel to the pixel data of the chip end pixel, and the pixel data of the missing pixel The image processing apparatus according to claim 1, wherein: The peripheral pixel change pattern calculation unit sets the first peripheral pixel group and the second peripheral pixel group arranged across the missing pixel as the peripheral pixel group, and obtains the change pattern for each peripheral pixel group,
The comparison pixel change pattern calculation unit sets each of the first comparison pixel group and the second comparison pixel group arranged with the comparison target pixel for comparison with the missing pixel as the comparison pixel group. Find the above change pattern,
The change pattern comparison unit compares the change patterns of the first and second peripheral pixel groups with change patterns of two or more first and second comparison pixel groups having different positions of the comparison target pixels, Extracting the first and second comparison pixel groups having the highest correlation of change patterns as the first and second reference pixel groups with respect to the first and second peripheral pixel groups;
The pixel data calculation unit obtains first pixel data of the missing pixel based on the first reference pixel group, obtains second pixel data of the missing pixel based on the second reference pixel group, and 5. The image processing apparatus according to claim 1, wherein pixel data of the missing pixel is obtained based on the first and second pixel data. An image for generating a line read image by interpolating pixel data missing between the sensor chips with respect to pixel data of the light receiving elements acquired from a line sensor including two or more sensor chips each having a plurality of light receiving elements. In the processing method,
In a peripheral pixel group including three or more consecutive pixels in the sensor chip including the chip end pixels adjacent to the missing pixels between the sensor chips, a difference in pixel data between adjacent pixels is obtained, A surrounding pixel change pattern calculating step for obtaining a change pattern of the difference;
A comparison pixel group having a position different from that of the peripheral pixel group, and in any comparison pixel group including three or more consecutive pixels, a difference in pixel data between adjacent pixels is obtained, and a change pattern of the difference is obtained. A comparison pixel change pattern calculation step to be obtained;
The change pattern of the peripheral pixel group is compared with the change patterns of two or more comparison pixel groups having different positions, and the comparison pixel group having the highest correlation of the change pattern with respect to the peripheral pixel group A change pattern comparison step extracted as
Pixel data of a first pixel corresponding to the chip end pixel in the reference pixel group, pixel data of a second pixel corresponding to the missing pixel and adjacent to the reference pixel group, and the chip end An image processing method comprising: a pixel data calculation step for obtaining pixel data of the missing pixel based on pixel data of the pixel. An image for generating a line read image by interpolating pixel data missing between the sensor chips with respect to pixel data of the light receiving elements acquired from a line sensor including two or more sensor chips each having a plurality of light receiving elements. In an image processing program for causing a computer to execute processing,
In a peripheral pixel group including three or more consecutive pixels in the sensor chip including the chip end pixels adjacent to the missing pixels between the sensor chips, a difference in pixel data between adjacent pixels is obtained, Neighboring pixel change pattern calculation procedure for obtaining a difference change pattern;
A comparison pixel group having a position different from that of the peripheral pixel group, and in any comparison pixel group including three or more consecutive pixels, a difference in pixel data between adjacent pixels is obtained, and a change pattern of the difference is obtained. A comparison pixel change pattern calculation procedure to be obtained;
The change pattern of the peripheral pixel group is compared with the change patterns of two or more comparison pixel groups having different positions, and the comparison pixel group having the highest correlation of the change pattern with respect to the peripheral pixel group is referred to as a reference pixel group. Change pattern comparison procedure to be extracted as
Pixel data of a first pixel corresponding to the chip end pixel in the reference pixel group, pixel data of a second pixel corresponding to the missing pixel and adjacent to the reference pixel group, and the chip end An image processing program comprising: a pixel data calculation procedure for obtaining pixel data of the missing pixel based on pixel data of the pixel.

Images