JP2005340872A - Image input apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image input apparatus adopting a pixel shift method wherein adjustment of an imaging section for carrying out pixel shift is simply executed with high accuracy. <P>SOLUTION: A mark suitable for adjustment of positioning by the pixel shift method is fixed to a stand on which a photographing object is placed, a shift amount of a relative position between an incident light and a solid-state imaging device is measured by using the mark and the positioning is adjusted. Excellent adjustment can be realized stably at all times independently of the photographing object by locating the mark to a corner of the stand so as to avoid the object from being overlapped on the mark. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image input apparatus having a camera using a pixel shifting method.

In recent years, the resolution of image input devices has been increasing. As the number of pixels of the image sensor increases, the yield decreases and the price increases. There is a pixel shifting method as a method for increasing the resolution of an input image at a low cost without increasing the number of pixels of the image sensor. As a pixel shift method,
A method of moving a CCD chip on a lightweight package is known. Since the moving distance is as small as several microns, high accuracy is required when controlling the moving distance. When positioning the CCD chip, there are errors caused by the structure that moves the package, mounting errors of the structure, and the like, and even if they are moved based on the values calculated from the characteristics of the drive elements, they do not match. Therefore, it is necessary to perform adjustment so that the actual movement amount becomes a predetermined value. It is known that the adjustment is performed by imaging a test chart having a pattern whose movement amount is easy to measure. A technique using a subject has also been proposed (for example, Patent Citation 1).

  Piezo elements are known as drive elements for moving the CCD chip and the package, but the zero point position is shifted even if the voltage is returned to 0 after the voltage is increased due to the problem of hysteresis. Furthermore, since the extension amount varies depending on the temperature depending on the driving element, it is necessary to devise a method for maintaining the accuracy of the movement amount in the long term. Using the test chart or a subject suitable for adjustment when adjusting the pixel shift method does not require adjustment once it is shipped, but it takes time each time the pixel shift is readjusted after shipment. This is a burden on service personnel. In addition, readjustment when a subject suitable for adjustment arrives will not be readjusted if an unsuitable subject continues, so the timing of readjustment will not be known, and eventually the user will be able to Find and place a test chart.

JP-A-9-135381

  The problem to be solved by the present invention is to simply and accurately adjust an imaging unit that performs pixel shifting in an image input apparatus having a pixel shifting method.

  A camera using a pixel shifting method for photographing the object to be imaged at a plurality of positions by shifting a relative position between the image sensor and the object to be imaged by the image sensor, a column for fixing the camera, In an image input apparatus provided with a table on which an object to be imaged is placed, an adjustment mark for moving the image sensor to a predetermined position when shifting pixels is provided on the surface of the image sensor on the table. An image input device characterized by the above.

  According to the present invention, it is possible to achieve stable and good positioning adjustment regardless of the presence or absence of a subject without using a subject or a test chart suitable for adjustment of the pixel shift method.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a schematic diagram showing an image input apparatus of the present invention. The present invention includes a camera 1 using a pixel shifting method, a column 2 for fixing the camera 1, and a document table 3 on which an object to be imaged is placed, and a mark 4 is provided on the document table 3 on the camera 1 side. ing. When the pixel shift is performed using the camera 1 and the movement amount is adjusted, the adjustment is performed using the photographed image of the mark 4 fixed on the document table 3.

Here, a general pixel shift adjustment method using the automatic adjustment apparatus shown in FIG. 4 will be described. The camera 1 is provided with an image sensor 10 so as to be movable. As the image sensor 10, a CCD or the like is used. An object to be imaged is imaged at a reference position before moving the image sensor 10 and stored in the memory 13. Thereafter, the package substrate 11 on which the image sensor 10 is mounted is moved by the x-direction drive device 16 and the y-direction drive device 17 to take an image, and is stored in the memory 13. Then, the movement amount calculation unit 14 calculates the movement amount using the stored image before movement and the image after movement.

  The predetermined movement amount is compared with the calculated movement amount, and the control value given to the x-direction drive device 16 and the y-direction drive device 17 is corrected by the processor 15 so that the difference is reduced. Then, an image is acquired by the image sensor 10 at the position moved by the corrected control value, and the amount of movement from the reference position is calculated again. Then, the adjustment is performed by repeating until the difference between the predetermined movement amount and the calculated movement amount becomes a certain value or less.

  The movement amount calculation unit 14 will be described with reference to FIG. The acquired image stored in the memory 13 enters the preprocessing unit 141. Corrections such as illumination unevenness, sensor sensitivity unevenness, lens distortion, and image distortion are performed on the acquired images before and after the movement. Thereafter, the center of gravity of both images is obtained by the center of gravity calculation unit 142. The calculation formula of the center of gravity (Nx, Ny) of the image in the image area of m pixels × n pixels is performed by the following formula.

  Aij is the pixel value of the i-th pixel in the x direction and the j-th pixel in the y direction in the image area.

  If the center of gravity of the image is a fixed point, the amount of movement can be calculated by determining where the center of gravity of the image before movement has moved after movement. The difference calculation unit 143 obtains the difference between the center-of-gravity coordinates before and after the movement, and sends it to the processor 15.

  By incorporating the parts other than the image sensor 10 and the package substrate 11 shown in FIG. 4 into the camera 1 or the column 2 or the document table 3, the camera of the pixel shift method can be adjusted with the configuration shown in FIG. 1. Further, the movement amount calculation unit 14, the processor 15, and the memory 13 in FIG. 4 may be provided as separate housings, or each may be replaced with a personal computer.

  With regard to the adjustment of the camera by the pixel shifting method, conventionally, it has been considered to place a test chart optimal for calculating the movement amount on the document table 3. However, the expansion amount of the drive element that moves the image sensor 10 may fluctuate due to temperature characteristics or the like. For this reason, since the adjustment is necessary not only once at the time of shipment but also after that, the optimum mark 4 for fixing the movement amount is fixed on the document table 3. Then, the test chart becomes unnecessary and stable and good adjustment is possible.

  The mark 4 disappears when the subject is placed at the reading position 5. Therefore, it is conceivable to fix the marks 41, 42, 43, 44 to the corners of the document table 3 as shown in FIG. By doing so, the adjustment can be performed regardless of the presence or absence of the subject at the reading position 5.

  Further, by arranging the marks 41, 42, 43, and 44 at the four corners, the amount of movement of the four corners can be measured, so that the amount of movement without deviation can be accurately measured at any position. As long as the image sensor 10 is not completely flat on the manuscript or the manuscript table 3, the shift amount is shifted at the four corners. It can be confirmed that the amount of movement is not flat, and adjustment is made so that the amounts of movement of the four corners do not shift. In this case, the adjustment can be automatically performed by applying a voltage to a piezoelectric element (not shown) that supports and fixes the image sensor 10 in parallel with the document table 3 to change the expansion ratio. The piezoelectric element may be a piezo element. In the case of pixel shift, the piezoelectric element extends in a direction parallel to the image pickup element 10 and the document table 3. It extends in a direction perpendicular to the table 3. Moreover, even when not adjusting in parallel, since the average of the four corners is used, it is possible to obtain a balanced and unbalanced movement amount.

  Also, since there is no relation to the presence or absence of the subject 5, the image is always taken and the movement amount is measured based on the mark 4, and the adjustment is automatically performed when the measured movement amount is away from the determined movement amount by a certain threshold value. can do. Here, for example, when the interval between the CCD elements of the image sensor 10 is 4.65 μm, the threshold value can be considered to be 1/10 of 0.465 μm.

  In addition, this automatic adjustment is performed at every shooting or periodically, or when the temperature fluctuates from the time of adjustment, for example, due to the temperature characteristics of the piezo element, the expansion of the piezo element is ±± from the temperature at the time of adjustment. It is also conceivable to carry out when there is a specific condition when there is a temperature fluctuation that fluctuates by 1/10 or more or when there is a designation from the user.

  Here, the number of marks is not limited to four at the four corners, and a plurality of marks may be used. In the case of a plurality of marks, an average value of the movement amounts of the plurality of marks may be used. Conversely, one mark may be used. In any case, it is conceivable that the mark is arranged in a place that is not hidden when the subject 5 is set.

  When carrying this image input device, if there is a column 2, it is often carried with the column 2. Also, when carrying it, it may hit a thing. Therefore, when the support column 2 is bent for some reason, the acquired image of the document table 3 taken from the camera 1 is as shown in FIG. In this case, if the intervals between the marks 41, 42, 43, and 44 fixed at the four corners are known, the distortion of the image is adjusted so that the intervals between the original marks 41, 42, 43, and 44 are based on the intervals. Correction can be performed as shown in FIG. It is also conceivable to use marks for correcting image distortion and marks 41, 42, 43, and 44 for adjusting pixel shift.

  Here, the camera 1 having the control for the special pixel shifting method has been described, but it is considered that the camera 1 has no control for the pixel shifting method.

  For example, it is considered that there is a merit when it can be read by the pixel shifting method using a digital camera that has already been purchased. If the pixel shift adjustment can be performed according to the pitch of the image sensor of various digital cameras, a high-resolution image can be acquired with a low-resolution digital camera, and a higher-resolution image can be acquired with a high-resolution digital. it can.

  FIG. 6 shows a configuration diagram of an image input apparatus according to the present invention.

  The digital camera 6 is fixed downward on the camera mount 7 and the subject 5 on the document table 3 is photographed. For example, the pixel shift driving device may be capable of taking a picture by moving the mounting base 7 of the digital camera 6, and the pixels other than the column 2, the original table 3, and the digital camera 6 may be moved. It is also possible to enable shooting using the shift method. Image information captured by the digital camera 6 is connected to the personal computer 8 by USB, IEEE 1394, or the like, and the processing by the movement amount calculation unit 14 and the processor 15 of the adjustment method described above is performed by software processing by the personal computer 8.

  Here, the signal connection between the digital camera 6 and the personal computer 8 may be wireless. There is no problem if the image information can be transferred. In addition, although the digital camera 6 is activated for shooting from the personal computer 6 here, the user may manually shoot.

  Here, the pixel arrangement by the pixel shift method will be described with reference to FIG.

  Here, nine images are acquired by moving the imaging position.

  First, the first image is acquired at the reference position of the image sensor. The pixel arrangement is shown in FIG. The pixel pitch is arranged at intervals of 3 pixels. Next, the image sensor is moved by 2/3 pixel pitch horizontally. The pixel arrangement of the image is as shown in FIG.

  Further, a pixel arrangement in which the image sensor is moved by 2/3 pixel pitch horizontally is as shown in FIG. Thereafter, the image sensor is returned to the reference position, and moved downward by 2/3 pixel pitch in FIG. The pixel arrangement of the image is as shown in FIG.

  Similarly, FIGS. 8E and 8F are obtained. Similarly, the pixel pitch is further moved down by 2/3 pixel, and FIGS. 8G, 8H, and 8I are obtained. Nine images are taken and all pixels are aligned in the state of FIG. The number of pixels of this image sensor is three times that in the horizontal direction and three times that in the vertical direction.

  Currently, the digital camera 6 is also the same, but the image pickup device of the camera is mainly a single plate type. FIG. 7 shows a typical arrangement example of the color filter array of the single plate type sensor. In the single plate type, light enters one pixel position through one of RGB color filters as shown in FIG. Therefore, only one of RGB signals can be obtained per pixel. Since the color information is represented by information of three colors of RGB per pixel, the other two colors that are not sufficient for the single sensor are generated with reference to surrounding colors.

  FIG. 10 shows an example of an image processing block diagram of the digital camera 6. The image signal photographed by the sensor 61 is digitized by the A / D converter 62 and stored in the memory 63. This is a signal of one of RGB colors per pixel. Thereafter, a plurality of pixels are read out from the signal stored in the memory 63 together with surrounding pixels, and the RGB generation unit 64 generates RGB image signals for each pixel from the color corresponding to the pixel position and the values of the surrounding pixels of the processing target pixel. .

  In the case of a single plate type, the arrangement of pixels after shooting nine images by the pixel shifting method is the RGB pixel arrangement shown in FIG. 9 in consideration of color information.

  When these are generated as 1-pixel RGB image information, for example, when the pixel area of FIG. 9 is viewed in a 2 × 2 pixel area, all the RGB are arranged in the pixel area. One pixel, RGB information is generated using this pixel arrangement.

For example, when the first pixel (RGB) at the upper left is generated, R is R1, G is an average value of G3 and G7, and B is B9 using a pixel area of 2 pixels × 2 pixels at the upper left. The pixel (RGB) on the right side is considered by shifting the pixel area of 2 × 2 pixels to the right by one pixel, R is R2, G is the average value of G3 and G8, and B is B9. Further, in the pixel (RGB) on the right side, R is R2, G is an average value of G1 and G8, and B is B7. As described above, by similarly generating RGB information, image information for three colors of RGB is generated for one pixel.

  Here, when a single digital camera 6 is used to shoot nine subjects 5 by the pixel shift method to increase the resolution, the images taken by the digital camera are captured one by one in the digital camera 6. RGB are generated and stored in the memory 65. With the configuration of FIG. 6, the image stored in the digital camera 6 is transferred to the personal computer 8, and the camera mounting base 7, the support column 2, the document base 3 and the like are moved to photograph the image, and the amount of movement from the reference position is obtained. Adjust the amount of movement. As described above, in this case as well, it is conceivable to use the marks 41, 42, 43, 44 for obtaining the movement amount on the document table 3.

  After the pixel shift adjustment, for example, nine images are shot while moving the solid-state imaging device under the control that enables the pixel shift method described above. However, RGB is generated for each pixel in the digital camera 6 and stored in the memory. Even if the single pixel RGB image information is used to rearrange the pixels according to the shooting positions as described with reference to FIG. 8 to generate one image from a plurality of pieces of image information, the resolution is not high. Rather, the image quality is degraded.

This is because the image captured by the digital camera 6 is generated in RGB per pixel in the state shown in FIG. Therefore, when generating RGB, even if the adjacent pixel area of 2 pixels × 2 pixels in one image is viewed from nine images of the pixel shift method, 2 at intervals of 3 pixels. An image in which RGB is generated in a pixel area of 2 × 2 pixels is obtained. Even if these nine images are correctly rearranged, the RGB pixel positions appear to be reversed from the original order, resulting in image quality degradation. As described above, in the RGB generation at the upper left of the image area in FIG. 9, R is R1, G is the average value of G3 and G7, and B is B9. In the digital camera 6, in the upper left RGB generation, R is R1, G is an average value of G1 which is 3 pixels apart and G1 which is 3 pixels below, and B is B1 which is 3 pixels diagonally below.

  For this reason, there is a method for increasing the number of pixels by using the digital camera 6 and moving the camera by pan-tilt and sticking the edges of the image together without using pixel shifting to increase the resolution of the image on the personal computer 8 side. .

  However, in pan / tilt, a mechanism for moving the digital camera 6 greatly is required, and there may be problems in the accuracy of bonding, vibration, movement time, and the like.

  Accordingly, an image processing configuration as shown in FIG. 11 is proposed as a method of using the digital camera 6 by the pixel shifting method and increasing the resolution of the image on the personal computer 8 side.

In the pixel shifting method, the digital camera 6 is used as shown in FIG. The image is read out from the image memory 81 in which the transferred image information is stored, and the RGB separation unit 82 returns the information of 1 pixel RGB (3 colors) to 1 pixel 1 color 1 pixel R, G, B information. That is, it returns to the image information before generating RGB for each pixel. If the RGB generation method is clearly known in advance, it can be restored. Moreover, it can also be performed by estimating to some extent.

  It is also conceivable that a plurality of return methods (RGB separation methods) are prepared in advance for a plurality of basic RGB generation methods, and the return method (RGB separation method) is switched depending on the specifications of the digital camera 6.

  After returning to the image information before being generated in RGB per pixel, the pixel rearrangement unit 83 rearranges the pixels corresponding to the nine acquisition positions. Here, the rearranged images may be stored in the image memory. By this pixel rearrangement, the pixels are arranged in the state of FIG. 9 here. Thereafter, the RGB generation unit 84 generates image information of one pixel RGB and stores it in the memory 85. This makes it possible to acquire a high resolution image.

  Further, although the configuration of FIG. 11 has been shown to be realized by software such as the personal computer 8, it can also be realized by hardware.

  When the digital camera 6 is used to capture a plurality of times by the pixel shifting method and the image is made high resolution, there are a plurality of captured images in the image memory 81 as shown in FIG. When the image information is created, the image information cannot be rearranged unless the position where the created image information is taken by the pixel shift method is known.

  For this reason, it is conceivable to include position information by the pixel shift method in the file names of the plurality of created image files 811 to 814. For example, it is conceivable to add a number after the file name in the order of shooting. In that case, the position information by the pixel shift method may correspond in order from the small number to the large number, or the position information by the pixel shift method in order from the large number to the small number. It is also possible that It is also possible to add an alphabet after the file name. It is also possible to refer to the created time. The file name can be assigned automatically, or it can be assigned by the user.

  Image information captured by the pixel shifting method may be transferred to a personal computer for each image taken, or a specific number of images may be transferred to a personal computer after taking a specific number of images. .

  At this time, there is a possibility that some digital cameras have not already been photographed images by the pixel shifting method, so it is possible to select only images photographed by pixel shifting and transfer them to a personal computer.

  In addition, as shown in FIG. 13, it is also conceivable to store captured images sequentially in one image file 815.

  A driving device for performing the pixel shifting method by moving either the camera mount 7 or the column 2 or the document table 3 is considered to be expensive because it requires a larger force to move the sensor.

  Therefore, it is also possible to consider an apparatus that manually shoots by performing a pixel shifting method by vibrating the camera mounting base 7 on the upper side of the support column 2 and the attached digital camera 6. The image is acquired while being manually vibrated, the amount of movement is calculated, and only an image whose absolute value of the difference is within a specific threshold with respect to the amount of movement of positioning is stored. In this way, when the images for all the required shooting positions are prepared, the user is notified of this, and the processing is performed as shown in FIG. 11 using the stored images, thereby increasing the resolution at a low cost. It is also possible.

  The digital camera 6 may be a mobile phone with a camera. The mounting base 7 may also serve as a charging device for a digital camera or a mobile phone. Further, if the mount 7 and the support column 2 are movable so that the camera unit can be directed in all directions, it can be used for a monitoring camera. Then, using a digital camera or a mobile phone with a camera, it can be used as a substitute for a scanner as a high-definition image input device or as a monitoring device while charging normally. When used in a monitoring apparatus, a high-definition image can be acquired by always taking a picture while performing the pixel shifting method.

  The image information for monitoring is also converted to RGB for each image, and simply rearranging the pixels does not provide high definition, but the image processing configuration shown in FIG. High-definition images can be acquired.

  Even at this time, it is necessary to devise such that the relationship between the captured images corresponding to the position of the image sensor can be understood. If the position and time of the image sensor at the start of shooting or recording are known, the position of shooting can be determined from the current time.

  In this case, it is conceivable that the image is always taken and the image is stored only when there is no change in the subject or the taken image or when there is a movement. Further, even when the subject moves from the middle, it is conceivable to generate a high-definition image by interpolating from several still images.

1 is a schematic diagram of an image input apparatus according to an embodiment of the present invention. It is a schematic diagram of the image input device which is the other Example of this invention. It is a figure which shows the image imaged from the camera, and the correct | amended image. It is a block diagram of an automatic adjustment device. It is a block diagram of a movement amount calculation unit. It is a schematic diagram of the image input device which is the other Example of this invention. It is a figure which shows a single color filter array. It is a figure which shows the pixel arrangement | positioning for every imaging | photography in a pixel shift method. It is a figure which shows pixel arrangement | positioning of the color filter array of each imaging | photography in a pixel shift method. It is a figure which shows the example of the image processing block of a digital camera. It is a block diagram of image processing of the present invention. It is a figure which shows the several image file acquired by the pixel shift method. It is a figure which shows the image file acquired by the pixel shift method.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera part, 2 ... Support | pillar, 3 ... Document stand, 4, 41, 42, 43, 44 ... Mark, 5 ... Reading position (Subject: Position where document is placed), 6 ... Digital camera, 7 ... Mounting base, 8 ... PC (PC), 10 ... Image sensor, 11 ... Package substrate, 12 ... A / D, 13, 63,
65, 81, 85 ... Memory, 14 ... Movement amount calculation unit, 15 ... Processor, 16, 17 ... Drive device, 61 ... Sensor, 62 ... A / D converter, 64, 84 ... RGB generation unit, 82 ... RGB separation , 83 ... Pixel rearrangement part, 141 ... Pre-processing part, 142 ... Center of gravity calculation part, 143 ... Difference calculation, 811, 812, 813, 814, 815 ... Image file.

Claims (6)

A camera using a pixel shifting method for photographing the object to be imaged at a plurality of positions by shifting a relative position between the image sensor and the object to be imaged by the image sensor, a column for fixing the camera, In an image input device having a table on which an object to be imaged is placed,
An image input apparatus, wherein an adjustment mark for moving the image sensor to a predetermined position when shifting pixels is provided on the surface of the table facing the image sensor. In a state where the object to be photographed is not placed on the table, the image sensor is moved with respect to the table, and shooting is performed by a pixel shifting method.
The amount of movement of the image sensor is measured, and when it is determined that the measured amount of movement has deviated from a value within a specific range, control is performed to move the image sensor to a predetermined position. Item 2. The image input device according to Item 1.   2. The adjustment mark is provided at four corners of the table, and the adjustment mark is also used as a mark for obtaining a coefficient for performing projective transformation on an image obtained by photographing an object to be imaged. Image input device. A camera using a pixel shifting method for photographing the object to be imaged at a plurality of positions by shifting a relative position between the image sensor and the object to be imaged by the image sensor, a column for fixing the camera, In an image input device having a table on which an object to be imaged is placed,
An image input apparatus that adjusts a moving amount for moving the image sensor to a predetermined position by using a barycentric position of an image obtained by photographing an object to be imaged.   The image input apparatus according to claim 4, wherein an object to be imaged for adjusting an amount of movement is an adjustment mark provided on the surface of the imaging device on the table. 6. The adjustment mark is provided at four corners of the table, and the adjustment mark is also used as a mark for obtaining a coefficient for performing projective transformation on an image obtained by photographing an object to be imaged. Image input device.

Images