JP2018207413A - Imaging apparatus - Google Patents

Abstract

To provide an imaging apparatus that can perform exposure control for each pixel, that can acquire a high dynamic range image by synthesizing outputs of a plurality of pixels having different exposures, and that can mitigate image quality deterioration due to image blur even in a scene in which a moving photographic subject is on a component.SOLUTION: An imaging apparatus comprises a control unit that can change exposure for each pixel and that controls the exposure for each pixel on the basis of a photometry result, performs control of different exposure time to each of a plurality of pixels for each pixel, generates an image for each output of each of different exposures using one time imaging after acquiring the present image, performs comparison between different exposure images using these images, comprises a blur amount calculation unit that performs blur determination, performs interpolation processing of a pixel value of an area, in which the blur is detected, from the pixel value of the different exposures when it is determined that a blur occurs, comprises blur correction unit that corrects the value using gain to become a predetermined output level, and after correcting generates the pixel value which synthesized outputs of the plurality of pixels for each pixel group in an image generation unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus.

  In digital cameras and video cameras, CCDs and CMOS image sensors are used as solid-state image sensors. In recent years, in an imaging device capable of controlling the exposure time in units of one or more pixel groups, images are shot with different exposures for each pixel, and a plurality of pixel outputs with the different exposures are combined to produce a high dynamic range image. An imaging apparatus that can obtain the above is disclosed.

  Japanese Patent Application Laid-Open No. 2004-133830 discloses a method of generating a high dynamic range image by controlling different exposure times for each of a plurality of same color pixels constituting a pixel block and synthesizing outputs of the plurality of same color pixels of the pixel block. Is disclosed.

JP 2013-0666140 A

  However, the conventional technique disclosed in Patent Document 1 does not discuss image quality degradation due to image blurring. In a scene where the moving subject is on the composition, if the exposure time differs for each pixel, the subject blur amount differs for each exposure time, and the image quality deteriorates.

  An object of the present invention is to control the exposure for each pixel, to obtain a high dynamic range image by combining outputs of a plurality of pixels having different exposures, and to perform image blur even in a scene where a moving subject is in the composition. It is an object of the present invention to provide an imaging apparatus capable of reducing image quality degradation due to the above.

In order to achieve the above object, an imaging apparatus according to the present invention includes:
In the imaging device capable of changing the exposure for each pixel, the image pickup apparatus has a control unit that controls the exposure for each pixel based on the obtained photometric result, and the control unit is different for each of the plurality of pixels for each pixel. The amount of blur that controls the exposure time, generates an image for each output for each different exposure in one shot after the main image is acquired, compares the images with different exposures using these images, and performs blur determination If it has a calculation unit and it is determined that blurring occurs, the pixel value of the blur detected area is interpolated from pixel values of different exposures, and the value is corrected with a gain so that it becomes a predetermined output level. A blur correction unit is included, and after correction, the image generation unit generates a pixel value obtained by combining outputs of a plurality of pixels for each pixel group.

  According to the imaging apparatus of the present invention, it is possible to control exposure for each pixel, obtain a high dynamic range image by combining outputs of a plurality of pixels having different exposures, and a scene in which a moving subject is on the composition The image quality degradation due to image blur can be reduced.

Overall block diagram of the imaging device Pixel arrangement for each pixel exposure and composition example Image blur generation image example Example flow chart of the first embodiment Example of generated image and corrected image for each exposure according to the first embodiment Explanatory drawing of the correction pixel of 1st Embodiment Correction value calculation example of the first embodiment Pixel array of pixel exposure and composition example in Bayer sensor

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<< First Embodiment >>
FIG. 1 is a block diagram illustrating a functional configuration example of an imaging apparatus using the imaging device of the present embodiment.

  The optical system of the image pickup apparatus includes a photographing lens 0110 that forms an optical image of a subject on the image pickup device 1101, and zoom control, focus control, aperture control, and the like are performed by a lens driving circuit 0109. A plurality of unit pixels are arranged in a matrix on the image sensor 1101.

  The subject image formed on the image sensor 1101 is output from the image sensor 1101 as an electrical image signal. The signal processing unit 1103 performs various corrections on the image signal output from the image sensor 1101 and compresses the image signal.

  The timing generation circuit 1102 outputs a timing signal for driving the image sensor 1101. The overall control / arithmetic unit 1104 performs various calculations and controls the overall operation of the imaging apparatus including the operation of the imaging element 1101. The focus detection unit 1109 performs a focus detection operation.

  Image data output from the signal processing unit 1103 is temporarily stored in the memory unit 1105. A display unit 1106 displays various information and captured images. The external recording unit 1107 records or reads image data. The external recording unit 1107 is a circuit that reads and writes a detachable recording medium such as a semiconductor memory. The operation unit 1108 includes an input device group represented by a switch, a button, a touch panel, and the like, and accepts a user instruction to the imaging apparatus.

  The exposure control unit 1110 performs AE exposure control and exposure control during actual photographing based on the AE result. The blur amount calculation unit 1111 compares the images generated for different exposures in one shooting, and calculates and detects the blur amount. The blur correction unit 1112 interpolates the pixel value of the position where the blur is detected from the pixel value of a different exposure image that is not blurred, and corrects the value by multiplying the gain so as to obtain a desired (predetermined) exposure. The image generation unit 1113 combines a plurality of pixel values with different exposures to generate a high dynamic range image. When the shake correction is performed, the composition process is performed using the output subjected to the shake correction.

  FIG. 2A shows an example of pixel-by-pixel exposure. Here, four pixels with different exposures are scattered throughout to form one group. For example, it is possible to acquire the output of a normal EV exposure pixel of 0 EV, a pixel of +2 EV that is two steps higher than the proper exposure, and a pixel of -2 EV that is two steps lower than the proper exposure and can be acquired by one shooting. ), It is possible to generate an image with a wide dynamic range. Here, in order to simplify the description, all the pixel configurations of the same color are expressed, but the present invention is not limited to this.

  FIG. 3A is an example of an image in which image blur does not occur in part of the screen, and FIG. 3B is an example of an image in which image blur occurs in part of the screen.

  FIG. 4 is a flowchart of this embodiment. Hereinafter, each step will be described.

(S10, S11, S12)
Photometry is performed by AE, and exposure control for each pixel at the time of actual image acquisition is performed based on the photometry result. For example, as shown in FIG. 2 (A), the main image is acquired by being scattered all over so as to form one group of four pixels having different exposures.

  For example, 0EV is an exposure that provides a proper output, and is set based on the result of AE. -2EV represents -2 steps of exposure from the appropriate exposure, + 2EV represents +2 steps of exposure from the appropriate exposure, and each pixel is set to the respective exposure.

(S13)
Next, an output image for each exposure is generated. Examples of output results of each exposure when shooting in this state are shown in FIGS.

(S14, S15, S16)
(S14) The blur amount calculation unit (1111) compares image data acquired at different exposure times for each exposure, and determines whether the blur amount is within a predetermined threshold.

  (S15) If all the blur amounts are within a predetermined threshold, image composition and generation are performed without correction. When the shake correction is performed, the composition process is performed using the output subjected to the shake correction.

(S16)
When the blur amount exceeds the threshold value, the blur correction unit (1112) interpolates the blur portion in the blurred exposure image from the pixel value in the exposure image with less blur and sets the output value to a desired (predetermined) level. Correction is performed (S16). Then, the image generation unit (1113) combines the images so that four pixels with different exposures form one group, and generates an output image.

  Next, S14 to S16 will be described with specific examples.

  As shown in FIG. 5, for example, FIGS. 5A and 5C and FIGS. 5B and 5C are respectively compared. As a result, FIG. The determination results of (A) and (C) are unblurred. Since there is a blurred portion in FIG. 5B, the portion is interpolated from the adjacent pixel values of the corresponding portion in FIG. 5A or 5C, and the output value is set to a desired (predetermined) level. to correct. FIG. 5B 'is an image of an image obtained by correcting the shake detection portion of FIG.

A specific example of the above-described interpolation and correction will be described with reference to FIG.
A pixel: 0 EV exposure pixel value B pixel: +2 EV exposure pixel value C pixel: −2 EV exposure pixel value B ′ pixel: +2 EV exposure pixel value after correction As shown in FIG. In FIG. 6B, the detected portion is corrected to B ′.

FIG. 7 is an example of correction value calculation at the time of correction, and is [average value of interpolation pixels] × [gain (exposure difference)]. For example, when correcting from the exposure image of A, it carries out as follows.
B ′ = (A1 + A2 + A3 + A4) / 4 × 4
For example, when correcting from an exposure image of C, B ′ = (C1 + C2 + C3 + C4) / 4 × 16
Correct as follows. Thereafter, the composition processing is performed using the corrected output.

  As in the above-described embodiment, in an imaging device capable of controlling exposure for each pixel, a scene in which a high dynamic range image can be acquired by combining outputs of a plurality of pixels with different exposures and a moving subject is on the composition It is possible to reduce image quality degradation due to image blur.

  In the present embodiment, the monochromatic sensor has been described as an example for simplification of description, but the present invention is not limited to this. For example, FIG. 8A shows an example in which pixel-by-pixel exposure control is performed for each color (for example, R, Gr, Gb, B) in the case of an RGB Bayer sensor, and R, Gr, Gb and B pixels are included.

  Here, the pixels with different exposures for each color are scattered all over so that four pixels form one group. For example, it is possible to acquire the output of 0 EV pixel of normal proper exposure, +2 EV pixel that is 2 steps higher than the proper exposure, and −2 EV pixel that is 2 steps lower than the proper exposure for each color by one shooting. The blur detection and the blur correction may be performed every time, and the image may be synthesized for each group for each color as shown in FIG.

  In the present embodiment, one pixel group is described for each 2 × 2 pixel array, but the present invention is not limited to this, and any array or grouping may be performed, but the present invention is not limited to this.

  Further, in the present embodiment, a single color sensor has been described as an example for the sake of simplification. For example, in the case of an RGB color sensor, accumulation start and exposure time for each color (for example, R, G, B) are R and There are cases where G and B are the same, and in the case of the same exposure time, it is possible to reduce the calculation processing load by calculating and detecting the amount of blurring for only one color. Not what you want.

  Further, in the present embodiment, an example is shown in which 0EV, + 2EV, and -2EV pixels are compared with each other with different exposure times, and the blur amount is calculated. For example, 0EV and -2EV pixels are calculated. When the accumulation is started at the same time, the exposure time is the same, and the exposure difference is set by the gain setting for each pixel, no blur occurs between the 0EV and -2EV images. For this reason, it is possible to calculate the blur amount only between the 0EV and + 2EV images without calculating the blur amount between the images for each of the 0EV and −2EV pixels. It is not limited to.

  In the present embodiment, an example in which pixels of 0 EV, +2 EV, and −2 EV are simply combined has been shown. However, for example, a combination ratio is weighted for each region and for each exposure according to a predetermined pixel output level. However, the present invention is not limited to this.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

1110 exposure control unit, 1111 shake amount calculation unit, 1112 shake correction unit,
1113 Image generation unit

Claims (6)

In an imaging device that can change the exposure for each pixel,
Based on the obtained photometric result, it has a control unit for controlling the exposure for each pixel,
The controller is
Different exposure times are controlled for each of a plurality of pixels for each pixel,
After acquiring the main image, an image is generated for each output for different exposures in one shooting, and a blur amount calculation unit that performs comparison between images with different exposures using these images and performs blur determination is provided. If it is determined that the pixel value of the region where the blur is detected, an interpolation process is performed from the pixel value of the different exposure, and the blur correction unit corrects the value with a gain so that the value becomes a predetermined output level, After the correction, the image generation unit generates a pixel value obtained by combining the outputs of a plurality of pixels for each pixel group.   The imaging apparatus according to claim 1, wherein the exposure setting value includes a setting based on an exposure time and a gain.   2. The blur determination unit according to claim 1, wherein in the blur amount calculation unit, the blur determination is performed by comparing images of pixel outputs with different exposures and determining whether a motion amount between images with different exposures is within a predetermined threshold. Imaging device.   In the blur amount calculation unit, an image for each output with different exposure time is used as a comparison image, and in the comparison image, even if the exposure is different, the set gain of the pixel is different and the exposure time is the same setting. The imaging apparatus according to claim 1, wherein the blur amount is calculated using any one of the plurality of exposure images having the same exposure setting image.   The interpolation processing of pixel values determined to be blurred by the blur amount calculation unit uses values of one or more adjacent pixels acquired at an exposure different from that of an exposure pixel determined to have a blur. Item 2. The imaging device according to Item 1.   The control unit controls different exposure times for each of a plurality of pixels of the same color for each pixel, generates an image for each output for different exposures in one shooting after acquiring the main image, and uses those images If there is a comparison between images of the same color with different exposures and it is determined that blurring occurs, the pixel value of the area where the blur is detected is interpolated from the pixel values of different exposures of the same color, and the value is set to a predetermined value. The pixel value obtained by combining the outputs of a plurality of pixels of the same color for each pixel group is generated in the image generation unit after the correction is performed with the gain so as to achieve the output level. The imaging device described.