JP2013051592A - Image pickup device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine an exposure table automatically, and to ensure highly accurate flicker correction in an image pickup device.SOLUTION: An amount of flicker calculation unit 1101 calculates the correction amount of flicker for each exposure field and each color component, based on the captured image data for each color component and each exposure field that are obtained as a result of capturing a flicker detection plate in an image acquisition unit 1107. The flicker calculation unit 1101 calculates the amount of flicker on the basis of the correction amount of flicker. When a determination is made that the amount of flicker exceeds an exposure table switching determination value in an exposure table switching determination unit 1102, an exposure table storage unit 1105 switches the exposure table being read from the exposure table storage unit 1105.

Description

  The present invention relates to an image capturing apparatus and method.

  For example, in a financial institution, a stand-type image reader (stand-type image reader) that is a non-contact type image reader for reading a target medium that cannot be handled by a passbook printer for tax, public money forms, and general forms is known. FIG. 1 is an external view of a general stand type image reader. The stand type image reader is an image scanner having the following features.

For example, as shown in FIG. 1, since the image sensor 101 and the pedestal 102 are separated from each other, for example, a thick form can be read.
An image is generated by receiving the light reflected from the medium surface by the image sensor without having a light source by itself.
A stand-type image reader using an interline CCD (Charge Coupled Device) performs exposure in a plurality of exposure fields to generate a single image.
・ Calibration is performed at the time of installation, and an image with a constant brightness is always acquired by adjusting the exposure time and gain according to the brightness of the installation environment.

FIG. 2 is an explanatory diagram of exposure time control in a stand-type image reader. In the stand type image reader, the exposure time is shortened in a bright environment as shown in FIG. 2A, and the exposure time is lengthened in a dark environment as shown in FIG. 2B. Thereby, a read image having a constant brightness is always obtained. Therefore, several exposure time setting values from a short exposure time to a long exposure time are held as an exposure table.
Example)
10ms, 20ms, 30ms, 40ms, 50ms, 60ms

In a stand-type image reader using an interline CCD, a group of horizontal scanning lines having different exposure start timings is called an exposure field, and exposure consisting of a plurality of exposure fields is performed to generate one image. At this time, in order to avoid the occurrence of image unevenness, the exposure amount of each exposure field must be constant. FIG. 3 is an explanatory diagram of an exposure method using an interline CCD, and shows an example of an interline CCD having three fields. FIG. 3 shows a state in which letters “a” of the alphabet are picked up by the pixel sensor 301 arranged in a lattice pattern in a two-dimensional plane. Each pixel sensor 301 includes a photodiode (not shown) that is a light receiving unit, and a charge coupled device (CCD: Charge Coupled Device) (not shown) for charge transfer in the vertical direction. In addition, a transfer gate (not shown) for transferring the charge accumulated in the photodiode by receiving light to the vertical transfer CCD for a predetermined exposure time is provided. In the interline CCD, as described above, exposure is performed in a plurality of exposure fields to generate one image. That is, in FIG.
The pixels in the first field are exposed at the timing of exposure 1.
The pixels in the second field are exposed at the timing of exposure 2.
Pixels in the third field are exposed at the timing of exposure 3.
Control is performed.

  Here, for example, a fluorescent lamp serving as a light source of a stand-type image reader is a light source driven by an AC power source, and a phenomenon called flicker occurs. For example, a fluorescent lamp normally repeats blinking at twice the power source, that is, 100 Hz if a power source having a frequency of 50 Hz (hertz) is used, and 120 Hz if a power source having a frequency of 60 Hz is used. This frequency is so high that it cannot be perceived by the human eye. When a single image is captured by multiple exposures using an interline CCD under such an AC drive light source, if the frequency of the fluorescent lamp does not match the cycle of the exposure time, The exposure amounts of the plurality of exposure fields of the line CCD are not constant. As a result, striped noise called flicker is generated in the image. FIG. 4 is an explanatory diagram of the flicker countermeasure exposure control in the interline CCD, and shows an example when the power supply frequency is 50 Hz. In order for the exposure amount of each of the plurality of exposure fields of the interline CCD to be constant, the exposure time needs to be an integral multiple of the blinking cycle of the fluorescent lamp. If it is an integral multiple, for example, in the example of FIG. 4, when the brightness of the light source periodically changes (flashes) every 10 ms (milliseconds), the amount of light taken into the photodiode of each pixel for each exposure time That is, the exposure amount is constant as shown in (1), (2), and (3) of FIG.

Therefore, conventionally, as shown below, two types of tables, an exposure table for 50 Hz and an exposure table for 60 Hz, have been prepared for each power frequency of 50 Hz and 60 Hz. These tables were switched to the local frequency.
Example)
50Hz regional table: 10ms, 20ms, 30ms, 40ms,
50ms, 60ms
60 Hz regional table: 8.3 ms, 16.7 ms, 25 ms,
33.3ms, 41.7ms, 50ms

  Image degradation due to flicker as described above does not occur when all pixels are exposed at the same timing with a mechanical shutter, but the provision of a mechanical shutter has the problem of increasing the risk of increased cost and increased failure rate. . Therefore, a configuration that does not use a mechanical shutter is often employed, as in the above-described prior art relating to an interline CCD.

  However, in the above-described conventional technology that does not use a mechanical shutter, the first problem is that the user has to manually determine the correct exposure setting table by determining the power frequency of the installation environment. If an incorrect setting table is selected, flickering occurs and the image quality deteriorates.

  In addition, in the above-described conventional technology that does not use a mechanical shutter, flickering occurs, so that it is not possible to set an exposure time of 10 ms or less in a 50 Hz environment and 8.3 ms or less in a 60 Hz environment. There was a problem that the image could not be acquired.

  As a conventional technique that further improves the above-described conventional technique, the amount of light outside the flicker in the installation environment of the apparatus is detected, and a flicker correction coefficient is created according to the output of the outside light detection and the read address, and this is used to capture image data A device that performs the light amount correction is known (for example, Patent Document 1). This conventional technique automatically changes the flicker correction degree for each area in accordance with the amount of illumination flicker in the installation environment of the apparatus, and performs appropriate external light correction.

  However, this conventional technique simply performs luminance correction by flicker of pixel data, and cannot sufficiently correct color misregistration or the like actually generated by flicker. Further, this prior art has a problem that the exposure table cannot be automatically determined.

  As another conventional technique, one or a plurality of flicker detection pixels for flicker detection, which are also used for image signals, are provided in a plurality of normal pixels for image signals obtained by photoelectrically converting image light from a subject. The pixel array unit 1, an integration circuit for integrating the light receiving element outputs from the flicker detection pixels so that the exposure time of the flicker detection pixels matches the exposure time of the normal pixels, and the light receiving elements from the flicker detection pixels. A flicker detection circuit that determines flicker from the output, and a shutter speed control circuit that specifies flicker based on the flicker information determined by the flicker detection circuit and controls the shutter speed to prevent occurrence of flicker for the specified flicker. The thing is known (for example, patent document 2). In this prior art, the flicker detection circuit determines whether the current light source is in the power cycle (60 Hz cycle) or the power cycle (50 Hz cycle), and the shutter speed control circuit is based on the detection result of the flicker detection circuit. Thus, the flicker noise is removed by selecting either n / 100 sec or m / 120 sec as the shutter speed.

  However, this conventional technique has a problem that it can only select a shutter speed suitable for the power supply cycle and cannot perform high-precision correction by flicker of pixel data.

  As another conventional technique, a common pre-processing unit that performs signal processing common to captured image processing and flicker detection processing on an input image signal, and an image signal for a display image with respect to the image signal from the common pre-processing unit 10 Based on the captured image processing unit that performs processing, the flicker detection preprocessing unit that performs image signal processing for flicker detection on the image signal from the common preprocessing unit, and the image signal from the flicker detection preprocessing unit One having a flicker detection circuit for performing flicker detection is known (for example, Patent Document 3). In this conventional technology, the flicker detection circuit detects the AC power supply frequency (50 Hz or 60 Hz) of the fluorescent lamp from the flicker noise present in the output signal of the flicker detection preprocessing unit, and the shutter speed control circuit converts the shutter speed to AC. Flicker noise is reduced by a method of increasing an integral multiple of the power supply frequency or a method of increasing the gain of a line having a low signal level by a digital gain circuit.

  However, this prior art also has a problem that it can only select a shutter speed that matches the power cycle, and cannot perform high-precision correction by flicker of pixel data.

JP 2001-103250 A JP 2009-212909 A JP 2010-4302 A

  An object of the present invention is to make it possible to automatically determine an exposure table and to perform highly accurate flicker correction.

  In an image capturing apparatus that captures an image of an imaging target set on a pedestal into an exposure field that is a plurality of pixel groups having different exposure start timings and that performs exposure, the image capturing apparatus is installed on the pedestal. An achromatic color or a flicker detection plate of a predetermined chromatic color whose color balance is known that is imaged close to the imaging target, and a captured image for each color component and for each exposure field obtained as a result of imaging the flicker detection plate by the image acquisition unit A flicker amount calculation unit that calculates a flicker correction amount for each exposure field and color component based on the data, and calculates the flicker amount based on the flicker correction amount, and whether the flicker amount exceeds the exposure table switching determination value Sets the combination of exposure table switching determination unit and exposure time in the image acquisition unit An exposure table storage unit that stores a plurality of exposure tables, and an exposure table that switches an exposure table to be read from the exposure table storage unit when the exposure table switching determination unit determines that the flicker amount exceeds the exposure table switching determination value. Based on the exposure table read from the exposure table storage unit via the switching unit and the exposure table switching unit, the exposure time and gain are set in the image acquisition unit, and the image acquisition unit is instructed to image the flicker detection plate again. Includes exposure time / gain setting section.

  According to the present invention, the exposure table can be automatically determined. At the same time, it is possible to perform flicker correction with high accuracy in both pixel luminance correction and color shift correction.

It is an external view of a general stand type image reader. It is explanatory drawing of exposure time control in a stand type image reader. It is explanatory drawing of the exposure system by interline CCD. It is explanatory drawing of the exposure control for a flicker countermeasure in an interline CCD. It is a block diagram of a flicker detection plate. It is a basic flow chart of control operation of this embodiment. It is a figure which shows the example of calculation of a flicker correction amount. It is an external view of this embodiment. It is a block diagram of the flicker detection plate on a base part. It is a block diagram of a camera part, control PC, etc. It is a block diagram of a 1st embodiment. It is a flowchart of a 1st embodiment. It is a block diagram of a 2nd embodiment. It is a flowchart of a 2nd embodiment. It is a block diagram of a 3rd embodiment. It is a figure which shows the example of the flicker amount coefficient in 3rd Embodiment. It is a block diagram of a 4th embodiment. It is a flowchart of a 4th embodiment. It is explanatory drawing of 4th Embodiment.

DESCRIPTION OF EMBODIMENTS Hereinafter, a mode for carrying out the present invention (hereinafter referred to as “the present embodiment”) will be described in detail with reference to the drawings.
This embodiment implements the following functions.
(1) By providing a flicker detection plate outside the form reading area on the pedestal, the flicker pattern of the read image is extracted and it is determined whether or not flicker has occurred with the current settings.
(2) The above flicker pattern is extracted during calibration. When it is determined that flicker has occurred, the exposure table is switched to automatically switch to an exposure table in which flicker does not occur.
(3) A correction amount is calculated from the above-described flicker pattern, and brightness and white balance deviation due to flicker in each exposure field are corrected.

  FIG. 5 is a configuration diagram of the flicker detection plate. As described in (1) above, the flicker detection plate 501 is provided outside the form reading area on the pedestal. When shooting is performed in a flicker generation environment such as under a fluorescent lamp, striped flicker noise is generated in the entire image including the flicker detection plate 501. In the present embodiment, flicker is detected from the image on the flicker detection plate 501.

  In order to accurately calculate the luminance change amount and the color change amount due to flicker, the flicker detection plate 501 is provided with the following color conditions, position conditions, material and processing conditions. In addition, reference is made to the number and size.

<Color conditions>
(1) Achromatic color In order to detect a color shift due to flicker, the color of the flicker detection plate is important, and the balance of the three primary colors: red, green, and blue is uniform. Note that an achromatic color is not necessarily required as long as the color balance is known.
(2) The gradation value is such that saturation (saturation) does not occur. Flicker detection with adjusted gradation value is performed to prevent the accuracy of the correction amount from being reduced due to saturation of the gradation when obtaining the color parameter. A plate 501 is set.

<Position condition>
(1) Provided outside the read guarantee area In the stand-type image reader, the recommended set position of the form is specified by abutment, and the read guarantee area with a margin added to the outside is specified. Since there is a possibility that a form is placed in the read guarantee area, it is inappropriate for flicker detection. Therefore, the flicker detection plate 501 needs to be installed outside the reading guarantee area.
(2) Provided on the upper part of the pedestal When an image is started before the hand leaves the imaging area, an afterimage of the hand may appear in some exposure fields. If there is a reflection, the accuracy of the flicker detection, such as false detection, is reduced. Therefore, the accuracy reduction can be suppressed by disposing the flicker detection plate 501 on the upper side of the pedestal where it is difficult to capture the hand.

<Material and processing conditions>
(1) The material must be resistant to discoloration and abrasion (such as polyacrylic)
This is to suppress color change due to aging, wear, and the like.
(2) Applying coating according to the material This is to reduce the unevenness and reflection on the surface of the flicker detection plate 501, and to reduce the influence on the image.

<About the number and size>
(1) A plurality of flicker detection plates 501 may be provided. Accordingly, when one flicker detection plate is hidden, flicker detection can be performed using another flicker detection plate.
(2) The whole pedestal may be achromatic gray The flicker detection may be performed by regarding a specific coordinate position on the pedestal as a plate.

FIG. 6 shows a basic flowchart of the control operation of the present embodiment using the flicker detection plate 501 described above.
First, an image is taken (step S601).

Next, the flicker amount Y is calculated from the average luminance difference and the white balance in each exposure field of the flicker detection plate 501 (step S602).
Subsequently, it is determined whether or not the flicker amount Y exceeds a determination threshold value (step S603).

If the flicker amount Y does not exceed the determination threshold value, it is determined that there is no flicker (NO in step S603 → step S604).
On the other hand, if the flicker amount Y exceeds the determination threshold value, it is determined that there is flicker (the determination in step S603 is YES → step S605).

The method for calculating the flicker amount Y in step S602 described above will be described in detail below.
In a general CCD, color information is acquired by performing pixel allocation called a Bayer array in which color filters are arranged in pixels at a ratio of R: G: B = 1: 2: 1. Here, R indicates a red component, G indicates a green component, and B indicates a three primary color component of a blue component. Due to the action of the color filter, each pixel cannot acquire information other than the assigned color component (for example, the information assigned to R is G and B information). For this reason, information of each color is generated in a pseudo manner from information on adjacent pixels (convolution). In the present embodiment, by using a Bayer image before performing various corrections such as magnification and inclination, it is possible to accurately extract the amount of noise due to pure flicker.

Here, as an example, the case of an interline CCD having three exposure fields as shown in FIG. 3 will be described. In this case, the horizontal line numbers on the two-dimensional CCD sensor in each exposure field are classified as follows, for example, where n is a natural number.
Horizontal line number of the first field: 1, 4, 7, 10, ... 3n-2
Second line horizontal line number: 2, 5, 8, 11, ..., 3n-1
Third line horizontal line number: 3, 6, 9, 12, ... 3n

In a Bayer array CCD, the same number of G pixels is present in all fields, and in this embodiment, the flicker luminance correction amount is calculated based on the G pixel.
For example, it is assumed that the value of each pixel component changes with a gradation width from 0 to 255. That is, each pixel component is represented by 8-bit data. In this case, when each pixel component value is, for example, G pixel average value of the first field = 100, G pixel average value of the second field = 90, and G pixel average value of the third field = 80, the largest value is obtained. The G pixel average value is generalized and expressed as G_MAX. At this time, the flicker luminance correction amount GX (= G1, G2, G3) of the Xth field (X = 1, 2, 3) of the image is calculated by the calculation represented by the following equation (1).

Flicker luminance correction amount for the first field G1 = G_MAX / G pixel average value = 100/100 = 1.0
Flicker luminance correction amount in the second field G2 = G_MAX / G pixel average value = 100/90 = 1.11.
Third field flicker luminance correction amount G3 = G_MAX / G pixel average value = 100/80 = 1.25
... (1)

  In addition, correction of color balance disruption in each field resulting from the difference in afterglow characteristics of fluorescent lamps is performed as follows.

  In this embodiment, when the flicker detection plate 501 is an achromatic color, the color balance is lost by multiplying the R pixel value and the B pixel value by the correction gain so that R = G = B in all fields. Can be corrected.

That is, the R component flicker color correction amount RX (= R1, R2, R3) of the X field of the image showing the correction gain and the B component flicker color correction amount BX (= B1, B2, B3) of the X field of the image. Is calculated as follows.
R pixel average value of first field: G pixel average value: B pixel average value = 0.9: 1: 0.9
R component flicker color correction amount R1 = G pixel average value / R pixel average value = 1 / 0.9 = 1.11 in the first field
First component B component flicker color correction amount B1 = G pixel average value / B pixel average value = 1 / 0.9 = 1.11
... (2)

Similarly,
R field average value in the second field: G pixel average value: B pixel average value = 0.8: 1: 0.9 R component flicker color correction amount R2 = G pixel average value / R pixel average in the second field Value = 1 / 0.8 = 1.25
Second component B component flicker color correction amount B2 = G pixel average value / B pixel average value = 1 / 0.9 = 1.11
... (3)

Similarly,
R pixel average value in the third field: G pixel average value: B pixel average value = 0.7: 1: 0.7 R component flicker color correction amount R3 = G pixel average value / R pixel average in the third field Value = 1 / 0.7 = 1.43
B component flicker color correction amount B3 = G pixel average value / B pixel average value = 1 / 0.7 = 1.43 in the third field
... (4)

Using the calculation results shown in the above (1), (2), (3), and (4), R is obtained by combining the luminance correction amount and the color correction amount for each color component R, G, and B in the Xth field. The component flicker correction amount FXR (= F1R, F2R, F3R), the G component flicker correction amount FXG (= F1G, F2G, F3G), and the B component flicker correction amount FXB (= F1B, F2B, F3B) are the following (5) ) Can be calculated by the calculation shown by the equation.
R component flicker correction amount FXR = GX × RX
G component flicker correction amount FXG = GX
B component flicker correction amount FXB = GX × BX
... (5)

A calculation example of the flicker correction amount based on the above equations (1) to (5) is shown in FIG.
From the flicker correction amount calculated as described above, the flicker amount Y can be calculated by, for example, the calculation represented by the following equation (6).
Flicker amount Y = Maximum flicker correction amount value = 1.7875 (6)
In addition, instead of the maximum flicker correction amount value, for example, an average value of all flicker correction amounts may be used.

  Here, when no flicker occurs, the average G pixel value of all exposure fields should be substantially the same for the result of imaging the achromatic flicker detection plate 501. Accordingly, when no flicker occurs, the flicker luminance correction amounts GX (X = 1, 1, 2 and 3 fields) calculated by the calculation shown in the above equation (1) should be almost 1. Further, since the flicker detection plate 501 is achromatic, when there is no flicker, the R average: G average: B average of each field should be approximately 1: 1: 1. Therefore, when no flicker occurs, the R component flicker color correction amount RX and the B component flicker color correction amount BX (X = first, second, and second) calculated by the calculations shown in the above equations (2) to (4). 3 fields) should all be nearly 1. Accordingly, when no flicker occurs, the R component flicker correction amount FXR, the G component flicker correction amount FXG, and the B component, which are the sum of the luminance correction amount and the color correction amount calculated by the calculation represented by the equation (5) It can be seen that the flicker correction amount FXB is almost all 1, and the flicker amount Y obtained as the maximum value thereof is close to 1. Conversely, the larger the flicker amount Y is, the stronger the flicker occurs.

  It is determined in step S603 in FIG. 6 whether or not the flicker amount Y calculated as described above exceeds a predetermined determination threshold, thereby determining whether or not flicker has occurred. It becomes possible.

  In specific first to fourth embodiments of the present invention described below, a flicker amount Y that combines the luminance correction amount and the color correction amount calculated as described above is used, so that flicker with high accuracy is used. It is possible to determine whether or not an image has occurred, select an exposure table based on it, and perform flicker correction on a captured image.

FIG. 8 to FIG. 10 are hardware configuration diagrams of the present embodiment implemented as a stand-type image reader common to the first to fourth embodiments described later.
First, FIG. 8 is an external view of this embodiment. The pedestal unit 803 is a table on which documents to be imaged are placed. A support column 804 extends vertically upward from a part of the pedestal 803, and a camera 801 for imaging a document or the like on the pedestal 803 and a lens 802 attached to the camera 801 are provided thereon.

  FIG. 9 is a configuration diagram of the flicker detection plate 501 on the pedestal 803 in FIG. The base part 803 and the column part 804 are the same as those in FIG. The flicker detection plate 501 is the same as that in FIG. As described in <Position Condition> with reference to FIG. 5, in the stand-type image reader, the recommended set position of the form is specified by abutment, and a read guarantee area 901 (see FIG. 9) (region indicated by a broken line). Since there is a possibility that a form is placed in the read guarantee area 901, the flicker detection plate 501 is installed on the upper part of the base outside the read guarantee area 901.

  FIG. 10 is a configuration diagram of the camera unit 801 in FIG. 8 and a control PC connected thereto. The camera unit 801 is the same as that in FIG. 8, and includes an image sensor 1001, a control CPU (central processing unit) 1002, a RAM (random access memory) 1003, a USB (universal serial bus) controller 1004, and an exposure / gain storage unit 1008. Is provided. The image sensor 1001 is an interline CCD type two-dimensional CCD sensor, and converts a captured image received through the lens 802 in FIG. 8 into captured image data of a digital signal. A control CPU 1002 controls the image sensor 1001 to capture an image. The RAM 1003 temporarily stores captured image data input from the image sensor 1001 in addition to a control program executed by the control CPU 1002 and control data. The exposure / gain storage unit 1008 stores an exposure time and a gain that is an input level when the control CPU 1002 takes captured image data from the image sensor 1001. These are set from the control PC 1006. The operation of the control CPU 1002 is a general operation. The USB controller 1004 is an interface circuit that terminates communication connected to the control PC 1006 via the USB interface 1005. A control PC (personal computer) 1006 performs image capture processing by a general financial institution, calculates a flicker amount, determines a power frequency based on the calculated flicker amount, switches an exposure table, sets exposure time and gain, A flicker correction process for the captured image data is executed. Details of these processes will be described later in the description of the first to fourth embodiments. An LCP (liquid crystal display) monitor 1007 displays a captured image captured by the camera unit 801 and processed in the control PC 1006.

FIG. 11 is a block diagram of the first embodiment based on the hardware configuration of the present embodiment from FIG. 8 to FIG.
First, the configuration of the camera unit 801 is the same as that of the camera unit 801 shown in FIG. 10, and includes an image sensor 1001, a control CPU 1002, a RAM 1003, and an exposure / gain storage unit 1008. Here, the image sensor 1001, the control CPU 1002, and the RAM 1003 constitute an image acquisition unit 1107.

  The control PC 1006 corresponds to the control PC 1006 in FIG. 10, and includes a flicker amount calculation unit 1101, an exposure table switching determination unit 1102, an exposure table switching determination threshold storage unit 1103, an exposure table switching unit 1104, an exposure table storage unit 1105, and an exposure / storage unit. A gain setting unit 1106 is provided.

  The flicker amount calculation unit 1101 is for each exposure field and for each color component based on captured image data for each color component and each exposure field obtained as a result of imaging the flicker detection plate 501 (see FIG. 9 and the like) by the image acquisition unit 1107. The flicker correction amount is calculated. Then, the flicker amount calculation unit 1101 calculates the flicker amount based on the flicker correction amount.

  The exposure table switching determination unit 1102 determines whether or not the above-described flicker amount exceeds the exposure table switching determination value stored in the exposure table switching determination threshold storage unit 1103.

The exposure table storage unit 1105 stores a plurality of exposure tables in which combinations of exposure times in the image acquisition unit 1107 are set.
The exposure table switching unit 1104 switches the exposure table read from the exposure table storage unit 1105 when the exposure table switching determination unit 1102 determines that the above-described flicker amount exceeds the exposure table switching determination value.

  The exposure / gain setting unit 1106 sets the exposure time and gain in the exposure / gain storage unit 1008 in the image acquisition unit 1107 based on the exposure table read from the exposure table storage unit 1105 via the exposure table switching unit 1104. . Then, the exposure / gain setting unit 1106 instructs the image acquisition unit 1107 to image the flicker detection plate 501 again.

  FIG. 12 is a flowchart showing the calibration control operation of the control PC 1006 that realizes the function of the block diagram of the first embodiment of FIG. The control operation shown in this flowchart is realized, for example, as an operation in which a CPU (not shown) in the control PC executes a control program stored in a memory (not shown). The control operation of this flowchart is started, for example, by a user operating the control PC 1006 (FIG. 10) or by automatically instructing the start of calibration upon power-on.

  First, the image acquisition unit 1107 of the camera unit 801 reads the exposure time and gain that are initially set in the exposure / gain storage unit 1008, and adjusts the exposure time and gain (step S1201).

  Next, the image acquisition unit 1107 captures an image of the captured image object including the flicker detection plate 501 (step S1202). This processing realizes the function of the image acquisition unit 1107 in FIG.

  Next, a flicker correction amount for each exposure field and each color component is calculated based on captured image data for each color component and each exposure field obtained as a result of imaging the flicker detection plate 501 by the image acquisition unit 1107. Then, the flicker amount Y is calculated based on the flicker correction amount (step S1203). More specifically, out of the captured image data for each of the R, G, and B color components and the first to third exposure fields obtained as a result of imaging the flicker detection plate 501, a predetermined color component, for example, the G component The flicker luminance correction amount for each exposure field is calculated as the ratio of the average luminance value of the captured image data for each exposure field. This corresponds to the calculation represented by the above-described equation (1). Next, the flicker color correction amount for each exposure field and each color component is calculated as the ratio of the average luminance value of the captured image data for each color component for each exposure field. This corresponds to the operations indicated by the above-described equations (2), (3), and (4). Then, the flicker correction amount for each exposure field and each color component is calculated by multiplying the flicker luminance correction amount for each exposure field by the flicker color correction amount for each exposure field and each color component. This corresponds to the calculation represented by the above-described equation (5). Then, the flicker amount Y is calculated as, for example, the maximum value (or average value) of the flicker correction amount. The processing in step S1203 described above realizes the function of the flicker amount calculation unit 1101 in FIG.

  Next, it is determined whether or not the above-described flicker amount Y exceeds the exposure table switching determination value x stored in the exposure table switching determination threshold value storage unit 1103 (step S1204). This process realizes the function of the exposure table switching determination unit 1102 in FIG.

  Next, when it is determined that the flicker amount Y exceeds the exposure table switching determination value x and the determination in step S1204 is YES, the exposure table read from the exposure table storage unit 1105 is switched (step S1205). This exposure table is, for example, an exposure table corresponding to each of the power supply frequencies of 50 Hz and 60 Hz, and a combination of exposure times is set. The processing in step S1205 realizes the function of the exposure table switching unit 1104 in FIG.

  After the processing in step S1205, based on the exposure table read from the exposure table storage unit 1105, the exposure time and gain are set in the exposure / gain storage unit 1008 in the image acquisition unit 1107 (steps S1205 → S1201). Then, the exposure time and gain are readjusted, the flicker detection plate 501 is photographed again, and the processes after step S1201 are repeatedly executed. This control process realizes the function of the exposure / gain setting unit 1106 in FIG.

  In the above iterative process, if it is determined that the flicker amount Y does not exceed the exposure table switching determination value x and the determination in step S1204 is NO, it is determined that no flicker has occurred. Then, without changing the exposure table, various correction information relating to, for example, magnification and inclination is generated (steps S1204 → S1206), and the calibration process is terminated.

According to the first embodiment described above, flicker can be prevented without using a mechanical shutter, and cost reduction, device failure rate reduction, and power consumption reduction can be expected.
In addition, it is possible to automatically select an optimum exposure table according to the power supply frequency of the installation environment that has been manually selected in the past, thereby reducing the amount of work and selection errors.

  FIG. 13 is a block diagram of the second embodiment based on the hardware configuration of the present embodiment from FIG. 8 to FIG. In FIG. 13, blocks having the same functions as those in the first embodiment of FIG. The configuration in FIG. 13 is different from the configuration in FIG. 11 in that an exposure count section 1301 and an exposure count storage section 1302 are further provided.

  When the exposure table switching determination unit 1102 determines that the flicker amount does not exceed the exposure table switching determination value stored in the exposure table switching determination threshold storage unit 1103, the exposure number counting unit 1301 performs the image acquisition unit 1107. The flicker detection plate 501 is instructed to image again under the same conditions up to a predetermined number of exposures stored in the exposure number storage unit 1302.

  FIG. 14 is a flowchart showing a calibration control operation of the control PC 1006 that realizes the function of the block diagram of the second embodiment of FIG. Steps having the same processing as in the first embodiment of FIG. 12 are assigned the same step numbers. The control operation shown in this flowchart is realized, for example, as an operation in which a CPU (not shown) in the control PC executes a control program stored in a memory (not shown), as in the case of FIG. As in the case of FIG. 12, the control operation of this flowchart is started by, for example, a user operating the control PC 1006 (FIG. 10) or by automatically instructing the start of calibration upon power-on.

  14 differs from the flowchart of FIG. 12 in step S1401, a counter (not shown) in the control PC 1006 is initialized to 0, for example. Thereafter, when the flicker amount Y does not exceed the exposure table switching determination value and the determination in step S1204 is NO, the counter is incremented by 1 (step S1402), and whether the counter value exceeds the predetermined number of exposures Z. Whether or not is determined (step S1403). If the determination in step S1403 is NO, the image acquisition unit 1107 is instructed to image again under the same conditions of the flicker detection plate 501 (steps S1403 → S1202). The above processing realizes the function of the exposure number counting unit 1301 in FIG.

If the number of exposures reaches Z and the determination in step S1403 is YES, the process proceeds to step S1206 described above, and the flicker correction process ends.
Even when shooting under the same conditions, the flicker intensity has a certain degree of randomness. For this reason, flicker can be reliably detected by the function or control operation of the second embodiment.

  FIG. 15 is a block diagram of the third embodiment based on the hardware configuration of the present embodiment from FIG. 8 to FIG. In FIG. 15, blocks having the same functions as those in the first embodiment of FIG. The configuration of FIG. 15 is different from the configuration of FIG. 11 in that it further includes a flicker coefficient multiplication unit 1501 and an exposure-flicker amount coefficient storage unit 1502. The flicker coefficient multiplication unit 1501 images the flicker detection plate 501 using the exposure table switching determination value stored in the exposure table switching determination threshold storage unit 1103 when the exposure table switching determination unit 1102 determines exposure table switching. The predetermined coefficient is multiplied by k in accordance with the exposure time.

  In general, flicker appears stronger as the exposure time is shorter and weaker as the exposure time is longer. Therefore, the exposure table automatic selection accuracy can be improved by weighting the exposure table switching determination value according to the length of the exposure time.

FIG. 16 shows an example of the flicker amount coefficient at each exposure. The flicker coefficient multiplication unit 1501 refers to, for example, the table of FIG. 16 stored in the exposure-flicker amount coefficient storage unit 1502 corresponding to the exposure time of the exposure table currently imaging the flicker detection plate 501. The coefficient k is determined. Then, the flicker coefficient multiplication unit 1501 gives this coefficient k to the exposure table switching determination unit 1102. The exposure table switching determination unit 1102 sets a value obtained by multiplying the exposure table switching determination value stored in the exposure table switching determination threshold storage unit 1103 by a coefficient k as a new exposure table switching determination value.
The coefficient k may be arbitrarily set regardless of the example of FIG.

  FIG. 17 is a block diagram of the fourth embodiment based on the hardware configuration of the present embodiment from FIG. 8 to FIG.

  First, the configuration of the camera unit 801 is the same as that of the camera unit 801 shown in FIG. 10, and includes an image sensor 1001, a control CPU 1002, a RAM 1003, and an exposure / gain storage unit 1008. Here, the image sensor 1001, the control CPU 1002, and the RAM 1003 constitute an image acquisition unit 1107.

  The control PC 1006 corresponds to the control PC 1006 in FIG. 10, and the flicker amount calculation unit 1101, flicker correction amount storage unit 1701, flicker occurrence determination unit 1702, flicker determination threshold storage unit 1703, flicker correction unit 1704, and output image generation unit 1705. Is provided.

  The flicker amount calculation unit 1101 is for each exposure field and for each color component based on captured image data for each color component and each exposure field obtained as a result of imaging the flicker detection plate 501 (see FIG. 9 and the like) by the image acquisition unit 1107. The flicker correction amount is calculated. Then, the flicker amount calculation unit 1101 calculates the flicker amount based on the flicker correction amount. This is the same as the flicker amount calculation unit 1101 in the first to third embodiments.

The flicker correction amount storage unit 1701 stores the flicker correction amount for each exposure field and color component calculated by the flicker amount calculation unit 1101.
The flicker occurrence determination unit 1702 determines the occurrence of flicker depending on whether the flicker amount calculated by the flicker amount calculation unit 1101 exceeds the flicker correction determination value stored in the flicker determination threshold storage unit 1703.

  When the flicker occurrence determination unit 1702 determines that the flicker amount exceeds the flicker correction determination value, the flicker correction unit 1704 is for each exposure field and for each color component obtained by the image acquisition unit 1107 imaging the imaging target. Flicker correction is performed on the captured image data based on the flicker correction amount for each exposure field and each color component stored in the flicker correction amount storage unit 1701.

  The output image generation unit 1705 outputs the image after flicker correction to the LCP monitor 1007 in FIG.

  FIG. 18 is a flowchart showing the control operation of the imaging process of the control PC 1006 that realizes the function of the block diagram of the fourth embodiment of FIG. Steps having the same processing as in the first embodiment of FIG. 12 are assigned the same step numbers. The control operation shown in this flowchart is realized, for example, as an operation in which a CPU (not shown) in the control PC executes a control program stored in a memory (not shown). The control operation in this flowchart is started when, for example, a user operating the control PC 1006 (FIG. 10) instructs the above-described calibration operation and subsequent imaging start of the imaging target.

  First, the image acquisition unit 1107 captures an image to be imaged including the flicker detection plate 501 (step S1202). This process is the same as the process of step S1202 of FIG. 12 in the first embodiment, and realizes the function of the image acquisition unit 1107 of FIG.

  Next, a flicker correction amount for each exposure field and each color component is calculated based on captured image data for each color component and each exposure field obtained as a result of imaging the flicker detection plate 501 by the image acquisition unit 1107. Then, the flicker amount Y is calculated based on the flicker correction amount (step S1203). This process is the same as the process in step S1203 of FIG. 2 in the first embodiment.

The calculated flicker correction amount for each exposure field and each color component is stored in the flicker correction amount storage unit 1701 in FIG.
Next, occurrence of flicker is determined based on whether or not the flicker amount calculated in step S1203 exceeds the flicker correction determination value stored in the flicker determination threshold storage unit 1703 (step S1801).

  If flicker occurs and the determination in step S1801 is YES, the image acquisition unit 1107 stores the captured image data for each exposure field and each color component obtained by imaging the imaging target in the flicker correction amount storage unit 1701. Flicker correction is performed based on the flicker correction amount for each exposure field and each color component (step S1802). This process implements the function of the flicker correction unit 1704 in FIG.

  When the determination in step S1801 is NO or after the processing in step S1802, various corrections relating to, for example, magnification and inclination of the image are performed on the input captured image data (step S1803), and the image is output ( Step S1804). The processing in steps S1803 and S1804 realizes the function of the output image generation unit 1705 in FIG.

  FIG. 19 is an explanatory diagram of the flicker correction process in step S1802 of FIG. The flicker correction amount F1R for each exposure field and each color component illustrated in FIG. 19A, which is the same as the example of FIG. 19, by the arithmetic processing represented by the above-described expressions (1) to (5) in step S1203. F1G, F1B, F2R, F2G, F2B, F3R, F3G, and F3B are calculated. Correction processing is performed on image data to be imaged that has been shot simultaneously with the flicker detection plate 501. First, for the pixel data group of the first field in the first row, the R pixel is multiplied by the flicker correction amount F1R, and the G pixel is multiplied by the flicker correction amount F1G. For the pixel data group in the second field of the second row, the B pixel is multiplied by the flicker correction amount F2B, and the G pixel is multiplied by the flicker correction amount F2G. For the pixel data group of the third field in the third row, the R pixel is multiplied by the flicker correction amount F3R, and the G pixel is multiplied by the flicker correction amount F3G. For the pixel data group of the first field in the fourth row, the B pixel is multiplied by the flicker correction amount F1B, and the G pixel is multiplied by the flicker correction amount F1G. For the pixel data group of the second field in the fifth row, the R pixel is multiplied by the flicker correction amount F2R, and the G pixel is multiplied by the flicker correction amount F2G. For the pixel data group in the third field of the sixth row, the B pixel is multiplied by the flicker correction amount F3B, and the G pixel is multiplied by the flicker correction amount F3G.

  As described above, in the fourth embodiment, the flicker correction amount can be calculated for each exposure field and for each color component, and the captured image data can be corrected. As a result, highly accurate flicker correction can be performed, and flicker components in the captured image data are removed.

  This means that even if some flicker occurs using a short exposure time shorter than the blinking cycle of the fluorescent lamp, it can be removed. As a result, the illuminance range in which the apparatus can be operated can be expanded as compared with the conventional apparatus, and reading can be performed in a brighter environment with the same apparatus configuration.

101, 1001 Image sensor 102 Pedestal 301 Pixel sensor 501 Flicker detection plate 801 Camera unit 802 Lens 803 Pedestal unit 804 Post unit 901 Read guarantee area 1002 Control CPU
1003 RAM
1004 USB controller 1005 USB interface 1006 Control PC
1007 LCP monitor 1008 Exposure / gain storage unit 1101 Flicker amount calculation unit 1102 Exposure table switching determination unit 1103 Exposure table switching determination threshold storage unit 1104 Exposure table switching unit 1105 Exposure table storage unit 1106 Exposure / gain setting unit 1301 Exposure count counting unit 1302 Exposure number storage unit 1501 Flicker coefficient multiplication unit 1502 Exposure-flicker amount coefficient storage unit 1701 Flicker correction amount storage unit 1702 Flicker occurrence determination unit 1703 Flicker determination threshold storage unit 1704 Flicker correction unit 1705 Output image generation unit

Claims (9)

In an image capturing apparatus that captures an image capturing target set on a pedestal with an image acquisition unit including an image sensor that performs exposure by dividing an exposure field that is a plurality of pixel groups having different exposure start timings,
A flicker detection plate of a predetermined chromatic color that is installed on the pedestal and is imaged in the vicinity of the imaging target and whose color balance is known;
A flicker correction amount for each exposure field and each color component is calculated based on captured image data for each color component and each exposure field obtained as a result of imaging the flicker detection plate by the image acquisition unit, and the flicker A flicker amount calculation unit for calculating a flicker amount based on the correction amount;
An exposure table switching determination unit that determines whether or not the flicker amount exceeds an exposure table switching determination value;
An exposure table storage unit that stores a plurality of exposure tables in which combinations of exposure times in the image acquisition unit are set;
An exposure table switching unit that switches the exposure table read from the exposure table storage unit when the exposure table switching determination unit determines that the flicker amount exceeds an exposure table switching determination value;
Based on an exposure table read from the exposure table storage unit via the exposure table switching unit, an exposure time and a gain are set in the image acquisition unit, and the image acquisition unit is instructed to image the flicker detection plate again. Exposure time / gain setting section
An image pickup apparatus comprising: When the exposure table switching determination unit determines that the amount of flicker does not exceed the exposure table switching determination value, the image acquisition unit re-images the flicker detection plate under the same conditions up to a predetermined number of exposures. Further including an exposure count section for instructing
The image capturing apparatus according to claim 1, wherein A flicker coefficient multiplier for multiplying the exposure table switching determination value by a predetermined coefficient according to an exposure time when the flicker detection plate is imaged;
The image capturing apparatus according to claim 1, wherein the image capturing apparatus is a part of the image capturing apparatus. In an image capturing apparatus that captures an image capturing target set on a pedestal with an image acquisition unit including an image sensor that performs exposure by dividing an exposure field that is a plurality of pixel groups having different exposure start timings,
A flicker detection plate of a predetermined chromatic color that is installed on the pedestal and is imaged in the vicinity of the imaging target and whose color balance is known;
A flicker correction amount for each exposure field and each color component is calculated based on captured image data for each color component and each exposure field obtained as a result of imaging the flicker detection plate by the image acquisition unit, and the flicker A flicker amount calculation unit for calculating a flicker amount based on the correction amount;
A flicker correction amount storage unit for storing a flicker correction amount for each exposure field and each color component;
A flicker occurrence determination unit that determines the occurrence of flicker depending on whether the flicker amount exceeds a flicker correction determination value;
When the flicker occurrence determination unit determines that the flicker amount exceeds the flicker correction determination value, the image acquisition unit captures images for each exposure field and each color component obtained by imaging the imaging target. A flicker correction unit that performs flicker correction on the image data based on the flicker correction amount for each exposure field and each color component stored in the flicker correction amount storage unit;
An image pickup apparatus comprising: The flicker amount calculation unit captures a predetermined color component for each exposure field among the captured image data for each color component and each exposure field obtained as a result of imaging the flicker detection plate by the image acquisition unit. A flicker luminance correction amount for each exposure field is calculated as a ratio of average luminance values of image data, and a ratio of average luminance value of captured image data for each color component is calculated for each exposure field and the color component for each exposure field. The flicker color correction amount for each exposure field is calculated, and the flicker luminance correction amount for each exposure field is multiplied by the flicker color correction amount for each exposure field and each color component, thereby flickering for each exposure field and each color component. Calculate the correction amount,
The image capturing apparatus according to claim 1, wherein the image capturing apparatus is an image capturing apparatus. The flicker detection plate is installed on the pedestal at the upper part outside the range of the read guarantee area, which is a range that guarantees the imaging of the imaging target.
An image capturing apparatus according to claim 1, wherein the image capturing apparatus is an image capturing apparatus. The flicker detection plate is obtained by making the entire pedestal surface the achromatic color or the predetermined chromatic color.
An image capturing apparatus according to claim 1, wherein the image capturing apparatus is an image capturing apparatus. In an image capturing method for capturing an image of an imaging target set on a pedestal with an image acquisition unit including an image sensor that performs exposure by dividing an exposure field, which is a plurality of pixel groups having different exposure start timings,
For each color component obtained as a result of imaging a flicker detection plate of a predetermined chromatic color with a known achromatic color or color balance, which is installed on the pedestal and imaged close to the imaging target, and the exposure field Calculating a flicker correction amount for each exposure field and each color component based on each captured image data, calculating a flicker amount based on the flicker correction amount,
Determining whether the flicker amount exceeds an exposure table switching determination value;
When the exposure table switching determination unit determines that the flicker amount exceeds the exposure table switching determination value, from an exposure table storage unit that stores a plurality of exposure tables in which a combination of exposure times in the image acquisition unit is set Switch the exposure table to read,
Based on an exposure table read from the exposure table storage unit via the exposure table switching, an exposure time and a gain are set in the image acquisition unit, and the image acquisition unit is instructed to image the flicker detection plate again. To
An image capturing method characterized by the above. In an image capturing method for capturing an image of an imaging target set on a pedestal with an image acquisition unit including an image sensor that performs exposure by dividing an exposure field, which is a plurality of pixel groups having different exposure start timings,
For each color component obtained as a result of imaging a flicker detection plate of a predetermined chromatic color with a known achromatic color or color balance, which is installed on the pedestal and imaged close to the imaging target, and the exposure field Calculating a flicker correction amount for each exposure field and each color component based on each captured image data,
Calculate the flicker amount based on the flicker correction amount,
Stores the flicker correction amount for each exposure field and each color component,
The occurrence of flicker is determined based on whether or not the flicker amount exceeds a flicker correction determination value,
When the flicker occurrence determination unit determines that the flicker amount exceeds the flicker correction determination value, the image acquisition unit captures images for each exposure field and each color component obtained by imaging the imaging target. Performing flicker correction on the image data based on the stored flicker correction amount for each exposure field and each color component;
An image capturing method characterized by the above.