JP2005142638A - Image pickup apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect the quantity of unbalance generated between a plurality of output terminals of an image pickup element, so as to automatically make the plurality of outputs uniform on the basis of the detection result. <P>SOLUTION: This apparatus can be set at a first prephotographing mode for photographing an object selected by a photographer under predetermined conditions as photographing before starting actual main exposure, and a second prephotographing mode for performing photographing with a shutter closed by using an illumination means. Correlation correction between the plurality of outputs of a photographed image in the actual main exposure is executed using a result of image correspondence relation of the first and second prephotographing modes. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  In an image sensor used in a digital camera or the like, the present invention automatically determines the output level between a plurality of outputs when the data is read from a plurality of outputs at the same time. The present invention relates to a method of correcting so as to remove.

  Conventionally, this type of digital still camera has a configuration as shown in FIG. In the case of the configuration in this figure, the overall control circuit 100 detects a change in the state of the camera operation switch 101 (configured by the camera main switch and release switch) by the photographer himself and starts supplying power to the other circuit blocks. To do.

  A subject image within the photographing screen range is formed on the image sensor 104 through the main photographing optical systems 102 and 103, and an electric signal from the image sensor 104 is sequentially applied to each pixel via the CDS / AGC circuit. The A / D converter 106 converts the signal into a predetermined digital signal.

  Here, the image sensor 104 is driven in a predetermined manner by the output of the driver circuit 107 for horizontal driving and vertical driving for each pixel based on a signal from the timing generator 108 that determines the overall driving timing, thereby obtaining an image. Generate signal output.

  Similarly, the CDS / AGC circuit 105 and the A / D conversion circuit 106 that perform analog processing on the output from the image sensor 104 and convert it to a predetermined signal level operate based on the timing from the timing generator 108. .

  The output from the A / D conversion circuit 106 is input to the memory controller 115 via the selector 109 that selects a signal based on the signal from the overall control CPU 100, and all signal outputs are transferred to the frame memory 116 here. . Therefore, in this case, all the pixel data for each photographing frame is temporarily stored in the frame memory 116, so that in the case of continuous shooting, all the writing operations are performed in the frame memory 116.

  After the shooting operation is completed, the contents of the frame memory 116 storing the shooting data are transferred to the camera DSP 110 via the selector 109 under the control of the memory controller 115. In the camera DSP 110, RGB color signals are generated based on the pixel data of the shooting data stored in the frame memory.

  In a state before normal shooting, this result is transferred to the video memory 111 periodically (for each frame), so that a finder display or the like is performed via the monitor display unit 112.

  On the other hand, when the photographer himself performs a photographing operation by operating the camera operation switch 101, each pixel data for one frame is read from the frame memory 116 by the control signal from the overall control CPU 100, and an image is captured by the camera DSP 110. After processing, the work memory 113 is temporarily stored.

  Subsequently, the data in the work memory 113 is compressed by the compression / expansion means 114 based on a predetermined compression format, and the result is stored in the external nonvolatile memory 117 (usually using a nonvolatile memory such as a flash memory).

  Conversely, when observing image data that has already been photographed, the data compressed and stored in the external memory is decompressed into data for each photographing pixel through the compression / expansion means 114, and the result is stored in the video memory 111. This can be done through the monitor display means 112.

  As described above, in a normal digital camera, the output from the image sensor 104 is converted into actual image data through a process processing circuit in almost real time, and the result is output to a memory or a monitor circuit.

  On the other hand, in the digital camera system as described above, in order to improve the ability for continuous shooting or the like (for example, to obtain an ability close to 10 frames / second), the reading speed from the image sensor or the frame memory is increased. It is necessary to improve the system including the image pickup device such as increasing the writing speed of the image pickup device data.

  FIG. 4 simply shows a two-output type device structure in which a horizontal CCD is divided into two with an imaging device such as a CCD as one of the improvement methods.

  In the CCD shown in FIG. 4, charges for each pixel generated in the photodiode sections 90 and 91 are transferred to the vertical CCD section at a predetermined timing all at once, and the vertical CCD charges are transferred to the horizontal CCD 92 for each line at the next timing. And 93.

  Here, the horizontal CCD 92 transfers the charge toward the left amplifier 94 every transfer clock, and the horizontal CCD 93 transfers the charge toward the right amplifier 95 every transfer clock. The photographed image data is read out divided into two right and left with the center of the screen as the boundary.

  Usually, the amplifier is built in the CCD device, but the relative accuracy of the two amplifiers does not always coincide completely because the amplifiers are located far away in terms of layout. For this reason, the output after the amplifier is passed through separate CDS / AGC / AD circuits 4 and 5 on the left and right sides, and then adjusted using some external adjustment means to ensure matching of the left and right outputs.

  In addition, as a method of improving the image resolving power while enhancing the continuous shooting capability of the camera, after shooting the shooting screen with a separate imaging device for each area, combining each imaging device output, There is a method for generating a captured image (see Patent Document 1).

In the case of this method, the image itself is arranged so that the image captured by each image sensor has some overlap with the image of the adjacent image sensor, and by performing image processing so that the overlapping part matches, It is devised so that the joint is not conspicuous when connecting multiple screens.
JP-A-6-141246

  As described above, as an image sensor that can realize high-speed readout, a method of simultaneously reading out signals from two or more outputs can be applied to a future digital camera using a more silver salt camera (already with a single-lens reflex type silver salt camera, 8 frames / It is an indispensable technology to bring it closer to a product with a spec of seconds.

  However, having a plurality of outputs is advantageous in terms of speed, but it is clearly disadvantageous from the viewpoint of matching of output levels as compared with one having only one output.

  A simple manual adjustment method, such as analog adjustment in the conventional CDS / AGC circuit section and digital adjustment that combines both channels with the output after A / D conversion, has been considerably adjusted in the manufacturing process. Even so, for example, the value of the VR resistance itself changes due to the change in the environment, and the tendency of the temperature characteristics of the CDS / AGC circuit to completely match is very low.

  Normally, when such a reading method of the image sensor is performed, if the relative accuracy of the left and right outputs exceeds ± 1%, the imbalance of the boundary is clearly understood on the screen.

  In the case of the method disclosed in Patent Document 1, the uniformity of the entire image can be easily overcome as long as the correlation between the portions can be determined because of the overlap between the images. As shown in FIG. 4 described above, in the case of shooting using an image sensor that performs simultaneous readout from a plurality of outputs, in principle, image data obtained by shooting the same shooting area between a plurality of outputs is not included. It seems difficult to adopt such an image processing method.

  Therefore, in the present application, when a camera or the like using such an image sensor having a plurality of outputs is configured, attention is paid to a specific method for automatically removing imbalance between a plurality of channels in a camera shooting sequence.

  As a method to solve the above problems, pre-photographing (with no incident light from the subject) using internal lighting such as LED before photographing, and the unbalanced amount between multiple channels from the imaging data at that time A method is proposed in which an offset and a gain value for correcting the discontinuity of an image generated between channels are set in advance from the unbalanced amount, and a subsequent correction for main exposure photography is executed. ing.

  However, in this method, the left and right unbalance amounts are calculated as they are from the photographed image that has been irradiated in the presence of uneven illumination of the LEDs (the light emission luminance distribution of the element is not necessarily uniform due to manufacturing variations, etc.) Then, there is a problem that an error occurs in the calculation result of the parameter for originally adjusting the left and right.

  An object of the invention according to the present application is to accurately detect an unbalance amount generated between a plurality of output terminals of an image sensor, and to automatically perform alignment between a plurality of outputs based on the detection result. Is to make up.

  In order to achieve the above object, the invention according to the present application forms an object image, converts the image data into an electrical signal for each pixel, and has a plurality of output terminals for outputting the signal to the outside. Means, processing means for individually processing a plurality of outputs from the imaging means, determination means for discriminating a correlation between data in a predetermined image area with respect to the output from the processing means, In an imaging apparatus that is disposed in the vicinity of the imaging unit and has an illumination unit that illuminates the imaging unit with a predetermined amount of light, the subject selected by the photographer is selected as a predetermined target for shooting before actual actual exposure starts. The first pre-shooting mode for shooting under conditions and the second pre-shooting mode for shooting in the shutter closed state using the illumination means can be set. Using image correlation results for the second pre-photographing mode, and executes the actual plurality of correlation correction of output between each other of the captured image at the time of the exposure.

  Further, the invention according to the present application generates an electrical signal by photoelectric conversion with respect to light incident on the image plane, and outputs the electrical signal as a plurality of signals corresponding to a plurality of regions in the imaging screen, A plurality of processing means for performing predetermined processing on each signal output from the imaging means, and a determination for determining a correlation between image data in each of the plurality of areas based on the output of each processing means And image data generated by a first shooting mode for shooting the subject under a predetermined condition before performing the main shooting, and a subject. Based on the correlation with the image data generated by the second imaging mode in which the illumination is performed using the illumination light with the light blocked, the previous image data obtained by the actual imaging And performing correction between the image data of each region.

  In other words, artificial illumination that is different from the photographic light is incorporated into the device, the image sensor is illuminated, multiple outputs from the image sensor at that time are processed separately, and the correlation for each output is determined to determine the correlation between multiple outputs. The unbalance amount is calculated, and then a uniform luminance image obtained by performing constant illumination on a white paper or the like is taken.

  For example, in a state in which the camera itself is set to a shooting mode (one of adjustment modes) different from normal shooting, illumination that is incorporated in the device in a state where shooting light is not incident on an image sensor having two outputs When the signal obtained from one of the outputs at that time is A and the other signal is B, the signal of only the pixel portion located near the boundary on the image sensor screen is taken out from A and B. The correlation between the two outputs is calculated to calculate the gain ratio between them.

  Subsequently, in the same adjustment mode, a photographed object illuminated with a blank paper or the like under a uniform light source is used, and an actual shutter exposure operation is performed without the internal illumination, and both output signals A and The same correlation calculation is performed from B, and the gain ratio between them is calculated.

  In this case, since the gain ratio corresponding to the left and right unbalance amounts is calculated based on an image obtained by photographing a blank sheet in a predetermined environment, the gain ratio calculated under this condition is a calculated gain error due to uneven illumination. It is possible not to include it.

  On the other hand, an error caused by uneven illumination light emission of the LED inevitably occurs in an image using the illumination means inside the camera that has been photographed at the same time.

  Therefore, in a state where the adjustment mode is not set, calibration shooting is performed by periodically turning on the illumination inside the camera while the SW1 is held before the main shooting of the camera, and a gain ratio between the outputs is calculated. Like that.

  In this way, the gain ratio calculated from the result of shooting a uniform subject such as white paper in the adjustment mode, the gain ratio calculated from the result of shooting using the internal illumination in the same adjustment mode, and calibration before normal shooting different from the adjustment mode The imbalance correction for the actual actual captured image is executed using the gain ratio calculated from the result of photographing using the internal illumination at the time of photographing.

  According to the imaging apparatus according to the present application, by setting an adjustment shooting mode different from the original shooting mode, the left and right unbalance amounts calculated from the shot image with only the internal illumination (using LEDs, etc.) in this mode. And the left and right unbalance amounts calculated from the photographed image for a uniform brightness subject such as a blank sheet by selecting the photographer's confidence are executed under substantially the same environmental conditions.

  Furthermore, in the original shooting mode, the left and right unbalance amounts are calculated from the shot image with only internal lighting (using LEDs, etc.) periodically before the actual actual shooting starts, and then the darkness when the shutter is closed. By performing shooting, pre-shooting is performed in an environment where the environmental conditions of the camera are almost equal to the actual shooting.

  Since the correction operation for the actual actual captured image is executed by combining the above-described shooting results, even if there is uneven internal illumination (LED emission unevenness) or the usage environment of the camera (change in temperature, humidity, etc.) Even in the case of a change, it is possible to accurately correct the imbalance due to reading of a plurality of channels of the image sensor.

  Examples of the present invention will be described below.

  FIG. 1 is a block diagram showing the overall hardware configuration of the present invention. In FIG. 1, an image sensor 1 having two outputs (CH1 and CH2) is constituted by a driver circuit 2 (configured by 2a and 2b, respectively). By being driven, it operates at a predetermined frequency, and is configured to output the captured image data separately on the left and right in the form of dividing the entire screen vertically into two. TG / SSG3 is a timing generation circuit that outputs a vertical synchronization signal VD and a horizontal synchronization signal HD. A driver circuit is driven based on a predetermined clock of the timing generation circuit and at the same time a timing signal ( RD, WR, SPL1, SPL2).

  The image output of the right half of the image sensor 1 is input to the CDS / AGC / AD circuit 5 via the CH1 output, and is included in the output of the CCD or the like by performing a method such as a known correlated double sampling. The AGC circuit for amplifying the output up to a predetermined signal level is operated while removing reset noise and the like.

  After this, the above signal is amplified to a predetermined magnification (changes in conjunction with the ISO sensitivity of the normal camera) by an internal AGC circuit, and then converted into a digital signal by inputting to the internal A / D converter. An output of AD-CH1 is obtained.

  Similarly, the image output of the left half of the image sensor is input to the CDS / AGC / AD circuit 4 via the CH2 output, and is included in the output of the CCD or the like by performing the same method of correlated double sampling or the like here. The AGC circuit for amplifying the output to a predetermined signal level is activated while removing the reset noise and the like.

  After this, the above signal is amplified to a predetermined magnification (changes in conjunction with the ISO sensitivity of the normal camera) by an internal AGC circuit, and then converted into a digital signal by inputting to the internal A / D converter. An output of AD-CH2 is obtained.

  After the left and right outputs from the image sensor 1 are separately converted into digital data in this way, both images are sequentially stored in the memory 9 via the memory controller 8.

  Further, the outputs of AD-CH1 and AD-CH2 are simultaneously input to the unbalance amount calculation circuit 18, where the unbalance amount of both outputs is calculated by a method described later, and the unbalance between the two outputs is corrected. The optimum correction amount is determined.

  Since the memory controller 8 can continuously execute reading and writing to the memory 9 in normal time division, the data written in the memory at different timings is written in the order in which the data from the image sensor is written to the memory in the order of writing. It is possible to read.

  When all the images for one frame are written in the memory 9, and when the unbalance amount calculation for the photographic image of the frame is completed by the unbalance amount calculation circuit 18 and the parameter correction coefficient calculation by a predetermined calculation described later is completed, The image temporarily stored in the memory 9 is read out and image processing for actual picture creation is started.

  First, the image on the left side of the screen of the image sensor 1 is continuously read from the memory 9 under the control of the memory controller 8 and input to the offset adjustment circuit 12. Here, a predetermined offset output OF2 calculated and set by the unbalance amount calculation circuit 18 is connected to the other input of the offset adjustment circuit 12, and both signals are added inside the offset adjustment circuit 12.

  Next, the output of the offset adjustment circuit 12 is input to the gain adjustment circuit 14, where the other input of the gain adjustment circuit 14 is a predetermined gain output GN2 calculated and set by the unbalance amount calculation circuit 18. Are connected, and both signals are multiplied inside the gain adjustment circuit 14.

  The output of the gain adjustment circuit 14 is input to the color processing circuit 17 via the image composition circuit 16 and a final image is obtained by executing normal color interpolation processing, matrix calculation processing, γ processing, and the like.

  At the same time that the image on the left side of the screen is read out for one line, the image on the CH1 side on the right side of the image sensor 1 is read out continuously from the memory 9 under the control of the memory controller 8 and input to the offset adjustment circuit 13. . Here, a predetermined offset output OF1 calculated and set by the unbalance amount calculation circuit 18 is connected to the other input of the offset adjustment circuit 13, and both signals are added inside the offset adjustment circuit 13.

  Next, the output of the offset adjustment circuit 13 is input to the gain adjustment circuit 15. Here, the other input of the gain adjustment circuit 15 is a predetermined gain output GN1 calculated and set by the unbalance amount calculation circuit 18. Are connected, and both signals are multiplied inside the gain adjustment circuit 15.

  The output of the gain adjustment circuit 14 is input to the image composition circuit 16. Here, after the data after correction processing for the image on the left side of the screen is input to the color processing circuit 17, the image is inverted (1 The pixel data read from the memory is input to the color processing circuit first, and the pixel data read from the memory is input to the color processing circuit first). The processed data is input to the color processing circuit 17, where normal color interpolation processing, matrix calculation processing, γ processing, and the like are executed to form a final image.

  Similarly, for the next line, the data on the left side of the screen is first input to the color processing circuit, and then the data on the right side of the screen is input to the color processing circuit to complete an image for one frame.

  In this way, the image data output after the unbalance amount generated between the two outputs is corrected by the unbalance amount calculation circuit is converted into one image data by the image composition circuit 16 (the left and right outputs are converted into one output). Then, it is assumed that predetermined color processing (color interpolation processing, γ conversion, etc.) is performed by the color processing circuit 17 in the next stage.

  The memory controller circuit 8 to the color processing circuit 17 (except for the memory 9) surrounded by a dotted line portion A in FIG. 1 are usually composed of a one-chip IC.

  Next, the overall control will be described. When the photographing operation SW 22 is operated by the photographer, the overall control CPU 19 detects this and first turns on the LED 11 via the driver circuit 10. This LED 11 is arranged in the camera body as shown by 11a or 11b in FIG. 2, and as shown by a dotted line in FIG. 2, a light shielding member 27 (see FIG. 2) that shields light from the subject from entering the image sensor. Usually, even if a mechanical shutter, a mirror, etc.) are in a light-shielded state, they are arranged at a position where a certain amount of light can be incident on the image sensor.

  10 and 11 show the layout configuration of the electronic camera of the present invention. FIG. 10 shows the configuration of the entire camera, and FIG. 11 is an enlarged view of a shutter device 44 portion to be described later of the camera in FIG. In the figure, 31 is an electronic still camera body, and 32 is a photographic lens for forming a subject image on an imaging surface, which is detachably attached to the electronic still camera body 31. The photographing lens 32 includes an imaging lens 33 for forming an object image on an imaging surface, and a lens driving device 34 for driving the imaging lens 33, and a diaphragm for performing exposure control. A blade group 35 and a diaphragm driving device 36 for driving the diaphragm blade group 35 are configured. Although the imaging lens 32 is simplified in the drawing, it is composed of one or a plurality of lenses, and may be a single focal length (fixed focal point) lens, zoom lens or step. A focal length variable like a zoom lens may be used.

  Reference numeral 37 denotes a main mirror for guiding a subject image formed by the photographing lens 32 to the focusing screen 38 and transmitting a part thereof to the focus detection device 43 through a sub-mirror 42 described later. The main mirror 37 is configured to be movable by a mirror driving device (not shown) to a position where a subject image can be observed with a viewfinder and a retreat position where the subject mirror retracts from the optical path of the subject light beam during photographing.

  Reference numeral 38 denotes a focusing screen on which a subject light beam guided by the photographing lens 32 is reflected by the main mirror 37 and is imaged. A subject image is formed on the focusing screen 38 during viewfinder observation.

  Reference numeral 39 denotes an optical member that converts and reflects an object image formed on the focusing screen 38 into an erect image. In this embodiment, a penta roof prism is used. Reference numeral 40 denotes an eyepiece device that causes the subject image converted and reflected into an erect image by the penta roof prism 39 to reach the eyes of the photographer.

  Reference numeral 41 denotes a photometric device that measures the luminance of a subject image formed on the focusing screen 38 during viewfinder observation via the penta roof prism 39. The electronic still camera 31 of the present embodiment is an output signal of the photometric device. Based on the above, exposure control at the time of exposure is performed.

  Reference numeral 42 denotes a sub mirror for reflecting subject light transmitted through a part of the main mirror 37 and guiding the subject light to a focus detection device 43 disposed on the lower surface of a mirror box (not shown).

  The sub-mirror 42 is interlocked with the main mirror 37 and a mirror driving mechanism (not shown) of the main mirror 37, and when the main mirror 37 is at a position where a subject image can be observed with a finder, a precursor focus detection device It is configured to be movable to a position where the subject light is guided to 43 and to a retreat position where the subject light is retracted from the optical path of the subject luminous flux during photographing.

  Reference numeral 43 denotes a focus detection device, which controls the lens driving device of the photographing lens 32 based on an output signal of the focus detection device 43 and performs focus adjustment with the imaging lens.

  Reference numeral 44 denotes a shutter device that mechanically controls the incidence of the subject luminous flux on the imaging surface. The shutter device 44 blocks the subject luminous flux during finder observation, and retracts from the optical path of the subject luminous flux in response to the release signal during imaging, and retracts from the optical path of the subject luminous flux during finder observation. And a focal plane shutter having a rear blade group 44b that shields the subject luminous flux at a predetermined timing after the start of traveling of the front blade group 44a. In the vicinity of the aperture opening of the shutter device 44, a notch or a through hole for projecting light emitted from the LED elements 23a and 23b described later to the leading blade group 44a is formed.

  Reference numeral 45 denotes an image pickup device (corresponding to the image pickup device 1 in FIG. 1) that picks up a subject image formed by the photographing lens 32 and converts it into an electrical signal. A known two-dimensional image pickup device is used for the image pickup element 45. There are various types of imaging devices such as a CCD type, a MOS type, and a CID type, and any type of imaging device may be employed. However, in this embodiment, there are two photoelectric conversion elements (photosensors). An interline CCD image sensor that is arranged in a dimension and outputs signal charges accumulated in each sensor through a vertical transfer path and a horizontal transfer path is employed. The image sensor 45 has a so-called electronic shutter function for controlling the accumulation time (in shutter seconds) of charges accumulated in each sensor.

  In the present invention, as shown in FIG. 12, the image pickup element 45 is protected by a cover glass 45b, which is an optical protective member for protecting the image pickup area 45a of the entire screen, and is vertically arranged on the right half face 45c and the left half face 45d. The image data is divided into two so that each captured image data can be output simultaneously.

  Reference numeral 46 denotes an electric board that electrically and mechanically couples the image pickup element 45 and LEDs 23a and 23b (corresponding to LEDs 11a and 11b in FIG. 2) described later to hold them.

  Reference numerals 23a and 23b denote light projecting means for projecting illumination light onto the imaging region 45a of the image sensor 45 according to the present invention. In the present invention, LED elements are used. As shown in FIGS. 11 and 12, the LED elements 23a and 23b are extensions of a dividing line 45e that divides the imaging region 45a into a right half face 45c and a left half face 45d in the vicinity of the upper and lower side surfaces of the image pickup element 45. The LED elements 23 a and 23 b are arranged so as to project light toward the shutter device 44.

  The luminous fluxes of the LED elements 23 a and 23 b are projected onto the imaging region 45 a of the imaging element 45 with the imaging element 45 side of the leading blade group 44 a of the shutter device 44 as a reflecting surface.

  Usually, a front blade group of a shutter device of a camera using a silver salt film as a recording medium is subjected to antireflection coating to prevent fogging of the film due to stray light. However, the electronic still camera according to the present invention is configured to control the exposure time by controlling the accumulation time (shutter time) of charges accumulated in each sensor by the electronic shutter function of the image sensor 45. Therefore, when the accumulation by the image sensor 45 is started, the leading blade group 44a is in an open state, so that anti-reflection coating on the leading blade group 44a is unnecessary in order to prevent the stray light from being fogged into the imaging region.

  Accordingly, in order to efficiently project the luminous flux of the LED elements 23a and 23b to the imaging region 45a of the image sensor 45, the leading blade group 44a of the shutter device 44 of the electronic still camera of the present invention is high It is desirable to use a material having reflectivity, or to perform surface treatment such as painting or plating with high reflectivity. Further, in order to illuminate the imaging region 45a of the image sensor 45 as widely as possible, it is desirable that the leading blade group 44a of the shutter device 44 has a diffusion characteristic. In the present invention, in order to achieve the above two conditions, the surface on the image sensor 45 side of the leading blade group 44a is semi-glossy white-tone paint or semi-glossy gray-tone paint. Even if one of these conditions is achieved, a sufficient lighting effect can be obtained.

In the present embodiment, the luminous flux of the LED elements 23a and 23b is directly projected and illuminated. However, a mask member having a specific pattern is provided near the light emitting portion of the LED elements 23a and 23b. An optical member that forms an image of this pattern on the imaging region may be arranged, and a specific pattern may be projected instead of the illumination light.
As shown in FIG. 11, in the present invention, the LED elements 23 a and 23 b are held by the electric board 46 that is a holding member of the imaging element 45 and are electrically connected. The holding member is held by the shutter device 44, the camera body (not shown), etc., and the electrical connection is made by connecting to the electric board 46 or other circuit board (not shown) by means of a flexible printed board or lead wire. May be.

  Reference numeral 48 denotes a filter member that removes high-frequency components of photographing light that cause noise, and is integrally held on the cover glass 45 b of the image sensor 45. The filter member 48 is made of a material having birefringence characteristics such as quartz and lithium niobate.

  Next, a specific concept of the unbalance amount calculation circuit 18 will be described using the pixel arrangement shown in FIG.

  FIG. 3 shows a so-called Bayer array of color filters, each composed of Gr (G of R line), R, B, and Gb (G of B line). The readout left screen is arranged symmetrically with the center of the image sensor as the boundary.

  When the Gr, R, B, and Gb pixels surrounded by the diagonal lines in FIG. 3 are used as one unit and they are simply added to generate a simple luminance signal (Y = Gr + R + B + Gb), The sum of the simple luminance signals located near the screen boundary (assuming that the number of units is m) is represented by YLi and YRi on the left screen and the right screen, respectively.

  The simple luminance signal calculated by the above method is sequentially read out along the j direction in FIG. 3, and the result is expressed by the relationship between the j direction and the simple luminance signal. The solid line and the dotted line in FIG. The result is as shown.

  In this way, the two methods described above are the average value of data in a predetermined range existing in the left half and the average value of data in a predetermined range existing in the right half of the pixel data output from the image sensor until now. Is to determine the correlation and correct the imbalance between the two outputs of the image sensor accordingly.

  Next, before performing the actual photographing operation on the actual subject, the left and right unbalance amounts are calculated by combining the automatic calibration photographing using the LED 11 incorporated in the camera device and the adjustment mode different from the normal photographing operation. The calculation sequence and the operation sequence of the left / right imbalance correction for the actual captured data will be described with reference to the flowcharts of FIGS. 5, 6, 7, 8 and 9.

  FIG. 5 shows the main shooting sequence. First, in step 150, it is determined whether or not SW1, which is one of the operation switches 22 of the camera, is in an ON state. If this is still OFF, the state of this SW1 is determined. Although the detection is continued, if the photographer detects that the SW1 is turned on, the process proceeds to the next step 151.

  In step 151, it is determined whether or not the mode setting means 20 shown in FIG. 1 is set to the adjustment mode. If the adjustment mode is set, the process proceeds to step 152 to execute the adjustment shooting sequence.

  The adjustment shooting sequence will be described using the flowchart of FIG.

  In step 200, the above-described LED 11 is turned on, and illumination with a predetermined illuminance is performed directly on the imaging element 1 from the vertical direction of the imaging surface.

  Next, in step 201, a predetermined trigger signal is given to the TG / SSG3 to start an accumulation operation by the image pickup device 1, and in step 202, this accumulation operation (in this case, accumulation for a predetermined time) is waited for. .

  When it is detected in step 202 that the accumulation operation has been completed, the LED is immediately turned off in step 203. Subsequently, in step 204, the data accumulated in the actual image sensor 1 is read out, and the memory controller 8 in FIG. The data is stored in the buffer memory 9 and at the same time, as described above, the image data in a predetermined area near the left and right screen boundaries is also taken into the unbalance amount calculation circuit 18 from the outputs of AD-CH1 and AD-CH2.

  In subsequent step 205, when it is detected that reading of the accumulated data is completed, next, in step 206, an actual gain amount calculating operation in the unbalance amount calculating circuit 18 is started.

  This gain amount calculation operation will be described using the flowchart shown in FIG.

  In step 250, first, the registers YLG and YRG used for internal calculation are initialized to zero.

  Next, in step 251, the value of the counter j used for the internal repetitive operation is reset to 0. Subsequently, in step 252, the value of the counter i used for the internal repetitive operation is similarly reset to 0, and the internal registers YLi and YRi is similarly initialized to 0.

  In step 253, as described with reference to FIG. 3, Grij, Rij, Bij, and Gbij in the positions of the predetermined i-th column (initially 0th column) and j-th row (initially 0th row) on the left screen are added to YLi. Then, the integrated value in the i direction of the left screen simple luminance signal near the center boundary is calculated.

  Similarly, in step 254, as described with reference to FIG. 3, Grij, Rij, Bij, and Gbij at predetermined i-th column (initially 0th column) and j-th row (initially 0th row) positions on the right screen are converted to YRi. In addition, the integrated value in the i direction of the right screen simple luminance signal near the center boundary is calculated.

  After incrementing the internal counter i by 1 in step 255, it is determined in step 256 whether or not the counter value has reached the predetermined value m. If it has not yet reached, the process returns to step 253 again, and the next Grij, The operation of adding Rij, Bij, Gbij to YLi and YRi is repeated.

  Thus, a total of m simple luminance integration values from 0 to m−1 in the i direction are calculated for each of the left screen and the right screen.

  Subsequently, in step 257, it is determined whether or not the left screen simple luminance integrated value YLi is greater than or equal to a predetermined level γ (a value that is somewhat large). If it is greater than or equal to γ, the process proceeds to step 258 to the left of the internal register YLG. The screen simple luminance integrated value YLi is integrated, and the right screen simple luminance integrated value YRi is integrated into the internal register YRG.

  On the other hand, if YLi is smaller than the predetermined level γ in step 257, the process proceeds to step 259 to increment the internal counter j by 1, and then in step 260, it is determined whether or not the value of j has reached n. If not, the process returns to step 252 again, and the operations from steps 252 to 259 described above are repeated.

  The above operation is repeated in step 260 until the value of j becomes equal to n, and integration is performed in the internal registers (YLG and YRG) up to n-1 rows (the last read-out row of the image sensor) in the j direction shown in FIG. The value will be counted.

  Next, in step 261, the ratio between the internal registers YLG and YRG (YLG / YRG) is set in the internal register GN1S as the left / right gain ratio. Similarly, in step 262, a predetermined value 1 is set in the internal register GN2S.

  The above is the explanation about the calculation of the left / right unbalance amount and the calculation method of the left / right gain ratio based on the calculation result. In addition to this method, the predetermined value 1 is set to GN1S in step 261, and the internal register is set in GN2S in step 262. There is also a method of setting the ratio between YRG and YLG (YRG / YLG) as the left / right gain ratio.

  Returning again to the flowchart of FIG. 6, in step 207, the left / right gain ratio data GN1S calculated in the gain ratio calculation flowchart described in FIG. 7 is reset to another internal register GN1X, and GN2S is reset to another internal register GN2X.

  Subsequently, at step 208, the accumulation operation by the image pickup device 1 is started by giving a predetermined trigger signal to the TG / SSG 3 with the LED 11 not turned on, and at step 209, this accumulation operation (in this case, for a predetermined time). Wait until accumulation is complete.

  For this operation, prepare a subject that the photographer will irradiate blank paper, etc., under a certain irradiation condition in advance (at least the subject brightness incident on the central part of the effective screen of the image sensor keeps approximately the same level on the left and right) In addition, the subject is photographed under a predetermined exposure condition.

  When it is detected in step 209 that the accumulation operation has been completed, a shutter closing operation is immediately performed in step 210. Subsequently, in step 211, the data accumulated in the actual image sensor 1 is read out, and the memory controller 8 in FIG. The data is stored in the buffer memory 9 and at the same time, as described above, the image data in the predetermined area near the left and right screen boundaries is also taken into the unbalance amount calculation circuit 18 from the outputs of AD-CH1 and AD-CH2.

  In subsequent step 212, when it is detected that the reading of the accumulated data is completed, next, in step 213, the actual unbalance amount calculation circuit 18 starts the gain amount calculation operation.

  This gain amount calculation operation is exactly the same as the flowchart of FIG. 7 described above, and detailed description thereof is omitted here. In this case, a subject of uniform brightness such as white paper outside is used instead of internal illumination. In the same way, GN1S and GN2S are calculated as the left and right gain ratios.

  Returning again to the flowchart of FIG. 6, in step 214, the left / right gain ratio data GN1S calculated in the gain ratio calculation flowchart described in FIG. 7 is reset to another internal register GN1Y, and GN2S is reset to another internal register GN2Y.

  The above is the explanation of the adjustment shooting sequence. In this sequence, the left and right gain ratios are calculated by shooting using the internal LED as the illumination source, and the uniform brightness of blank paper or the like is selected by the photographer. The left and right gain ratios are also calculated when the subject is photographed, and these two photographings almost coincide with the photographing environment conditions of the camera.

  Returning to the flowchart of FIG. 5 again, after the adjustment photographing sequence is completed in step 152, the process proceeds to step 150 again to determine whether or not the SW1 of the camera is ON.

  On the other hand, if the camera mode setting means 20 is not set to the adjustment mode in step 151, the process proceeds to step 153, where the LED 11 as the internal illumination starts to be turned on, and the imaging surface 1 is directly in the vertical direction of the imaging surface. To illuminate at a predetermined illuminance.

  Next, in step 154, a predetermined trigger signal is given to TG / SSG3 to start the accumulation operation by the image pickup device 1, and in step 155, this accumulation operation (in this case, accumulation for a predetermined time) is waited for. .

  When it is detected in step 155 that the accumulation operation has been completed, the LED is immediately turned off in step 156. Subsequently, in step 157, the data accumulated in the actual image sensor 1 is read out, and the memory controller 8 in FIG. The data is stored in the buffer memory 9 and at the same time, as described above, the image data in a predetermined area near the left and right screen boundaries is also taken into the unbalance amount calculation circuit 18 from the outputs of AD-CH1 and AD-CH2.

  In subsequent step 158, when it is detected that reading of the accumulated data is completed, next, in step 159, an actual gain amount calculating operation in the unbalance amount calculating circuit 18 is started.

  Since the operation for calculating the gain amount is exactly the same as that described in the flowchart of FIG. 7, a detailed description thereof will be omitted. However, the calculated left / right gain ratio data is obtained in step 160 in which GN1S is stored in the internal register GN1Z and GN2S is set Reset to internal register GN2Z.

  Next, in step 161, a predetermined trigger signal is given to the TG / SSG3 to start the accumulation operation by the image pickup device 1, and in step 162, it waits until this accumulation operation (in this case, accumulation for a predetermined time) is completed. .

  When it is detected in step 162 that the accumulation operation is completed, the data accumulated in the actual image pickup device 1 is immediately read out in step 163, and the data is stored in the buffer memory 9 via the memory controller 8 in FIG. As described above, image data in a predetermined area near the left and right screen boundaries is also taken into the unbalance amount calculation circuit 18 from the outputs of AD-CH1 and AD-CH2.

  In subsequent step 164, when it is detected that the reading of the accumulated data is completed, in step 165, an actual offset amount calculating operation in the unbalance amount calculating circuit 18 is started.

  The offset amount calculation operation will be described with reference to the flowchart of FIG.

  In step 300, the registers YLO and YRO for use in internal operations are first initialized to zero.

  Next, in step 301, the value of the counter j used for the internal repetitive operation is reset to 0. Subsequently, in step 302, the value of the counter i used for the internal repetitive operation is similarly reset to 0, and the internal registers YLi and YRi is similarly initialized to 0.

  In step 303, as described in FIG. 3, Grij, Rij, Bij, and Gbij in the positions of the predetermined i-th column (initially 0th column) and j-th row (initially 0th row) on the left screen are added to YLi. Then, the integrated value in the i direction of the left screen simple luminance signal near the center boundary is calculated.

  Similarly, in step 304, as described with reference to FIG. 3, Grij, Rij, Bij, and Gbij in the predetermined i-th column (initially 0th column) and j-th row (initially 0th row) position on the right screen are converted into YRi. In addition, the integrated value in the i direction of the right screen simple luminance signal near the center boundary is calculated.

  After the internal counter i is incremented by 1 in step 305, it is determined in step 306 whether or not the counter value has reached the predetermined value m. If it has not yet reached, the process returns to step 303 and the next Grij, The operation of adding Rij, Bij, Gbij to YLi and YRi is repeated.

  Thus, a total of m simple luminance integration values from 0 to m−1 in the i direction are calculated for each of the left screen and the right screen.

  Subsequently, in step 307, it is determined whether or not the left screen simple luminance integrated value YLi is equal to or less than a predetermined level ε (a value close to 0), and if it is equal to or less than ε, the process proceeds to step 308 to the internal register YLO. The left screen simple luminance integrated value YLi is integrated, and the right screen simple luminance integrated value YRi is integrated into the internal register YRO.

  On the other hand, if YLi is greater than the predetermined level ε in step 307, the process proceeds to step 309 and the internal counter j is incremented by 1. Then, in step 310, it is determined whether or not the value of j has reached n. If not, the process returns to step 302 again and the operations from step 302 to 309 described above are repeated.

  The above operation is repeated in step 310 until the value of j becomes equal to n, and integration is performed in the internal registers (YLO and YRO) up to n-1 rows (the last read-out row of the image sensor) in the j direction shown in FIG. The value will be counted.

Next, at step 311, the result of subtracting the value of YRO from the value of the internal register YLO is divided by 4 ij to calculate the unbalance amount of the left and right images as the average value of the left and right differences, and this result is calculated as the internal value. The register OF1S is set, and then in step 312, the internal register OF2S is set to 0.
The above is the explanation about the operation of calculating the offset amount in step 165. In this way, until the actual actual photographing is started, the calibration photographing is periodically performed and the gain amount calculating operation by the LED illumination is performed. An offset amount calculation operation by photographing is executed.

  Returning to the flowchart of FIG. 6 again, in step 166, it is determined whether or not the release switch operation for performing the actual photographing is actually performed by the photographer (in this case, expressed as SW2). If it is determined that it has not been done, the process returns to step 151 and the above-described operation is repeated.

  On the other hand, if it is determined in step 166 that the release switch has been operated, the process proceeds to step 167 where the mechanical shutter is opened via a shutter drive circuit (not shown) and a predetermined trigger is applied to TG / SSG3. The accumulation operation by the image sensor 1 is started by giving a signal.

  When it is detected in step 168 that the accumulation operation has been completed, the mechanical shutter is immediately closed in step 169. Subsequently, in step 170, the data accumulated in the actual image sensor 1 is read, and the memory controller 8 in FIG. The data is stored in the buffer memory 9.

  Next, in step 171, a correction amount setting for performing a correction operation on the main captured image stored in the buffer memory 9 is performed. This operation will be described with reference to the flowchart of FIG.

  First, in step 350, the gain ratio GN1Z value calculated by the above-described method is multiplied by the gain ratio GN1Z value, and the result is divided by the gain ratio GN1X value as the final gain correction amount. The gain ratio GN1 for the gain adjustment circuit 15 in FIG. 1 is set.

  Similarly, in step 351, the gain ratio GN2Y value calculated by the above-described method is multiplied by the gain ratio GN2Z value, and the result is divided by the gain ratio GN2X value to obtain the final gain correction amount. As a gain ratio GN2 for the gain adjustment circuit 14 of FIG.

  Subsequently, in step 352, the value of the offset OF1S calculated by the method described above is set as the offset OF1 for the offset adjustment circuit 13 in FIG. 1, and similarly, in step 353, the value of the offset OF2S calculated by the method described above is set in FIG. This is set as OF 2 for the offset adjustment circuit 12.

  After performing the above correction operation, the process proceeds again to step 172 in the main photographing sequence of FIG. 6, where each adjustment circuit, image composition circuit 16, and color processing circuit 17 surrounded by the dotted line A of FIG. The image processing operation passed through is started.

1 is an overall system configuration diagram of the present invention. It is a figure showing the concept of the calibration concerning the whole this invention. It is a figure showing the pixel arrangement | sequence of the image pick-up element based on the whole this invention. It is the figure which showed the structure which samples the output from the image pick-up element based on the whole this invention. It is a flowchart for demonstrating the operation | movement which concerns on the whole this invention. It is a flowchart for demonstrating the operation | movement which concerns on the whole this invention. It is a flowchart for demonstrating the operation | movement which concerns on the whole this invention. It is a flowchart for demonstrating the operation | movement which concerns on the whole this invention. It is a flowchart for demonstrating the operation | movement which concerns on the whole this invention. It is a schematic block diagram of the camera of this invention. It is the elements on larger scale of the camera of FIG. It is a perspective view of an image sensor. It is a whole system block diagram of a prior art example.

Explanation of symbols

1: Image sensor 4, 5: CDS / AGC / AD circuit 11: LED
18: Unbalance amount calculation circuit

Claims (4)

An imaging unit that forms a subject image, converts the image data into an electrical signal for each pixel, and outputs the signal to the outside, and a plurality of outputs from the imaging unit individually. A processing means for processing; a discrimination means for discriminating a correlation between data in a predetermined image area with respect to an output from the processing means; and a predetermined means for the imaging means that is arranged in the vicinity of the imaging means In an imaging apparatus having illumination means for illuminating with a light amount,
First pre-shooting mode in which the subject selected by the photographer is shot under predetermined conditions as shooting before actual actual exposure is started, and second shooting is performed with the shutter closed using the illumination unit. The pre-shooting mode can be set, and using the image correlation results in the first pre-shooting mode and the second pre-shooting mode, the output between multiple outputs of the actual captured image can be calculated. An image pickup apparatus that performs correlation correction.   2. The imaging apparatus according to claim 1, wherein the first pre-shooting mode includes a first mode for shooting an object having a uniform luminance surface and a second shooting mode for shooting only under internal illumination. .   2. The imaging according to claim 1, wherein the second pre-shooting mode includes a first shooting mode in which shooting is performed only under internal illumination and a second shooting mode in which shooting is performed in a completely light-shielded state. apparatus. Imaging means for generating an electrical signal by photoelectric conversion with respect to light incident on the image plane, and outputting the electrical signal as a plurality of signals corresponding to a plurality of regions in the imaging screen;
A plurality of processing means for performing predetermined processing on each signal output from the imaging means;
Discrimination means for discriminating a correlation between image data in each of the plurality of areas based on an output of each processing means;
Illuminating means for illuminating the imaging means with illumination light,
Image data generated by the first shooting mode for shooting the subject under predetermined conditions before performing the main shooting, and second shooting for shooting using the illumination light of the illumination unit in a state where the subject light is blocked An image pickup apparatus that performs correction between image data of each region of image data obtained by actual photographing based on a correlation with image data generated by a mode.

Images