JP2012124800A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of reducing deterioration of an image due to difference of degree of distortion between divided images, by controlling a dividing method of dividing CMOS sensor pixels into a plurality of groups and a synthesis method of the image read by dividing.SOLUTION: An imaging apparatus includes: a CMOS sensor that has a plurality of image pixels receiving light and converting the light to an image signal; drive control means that controls accumulation and reading of the CMOS sensor; and camera signal processing means that performs a development processing for the image signal read out from the CMOS sensor. The driving control means divides the pixels read from the CMOS sensor into a first pixel group and a second pixel group, and the second pixel group is read out after the first pixel group is read out.

Description

  The present invention relates to an image pickup apparatus, and more particularly to control when a pixel of an image pickup element is divided into a plurality of pixels and read out.

  In recent years, video cameras and still cameras using CMOS sensors as image sensors have been proposed. The CMOS sensor has a feature that the CCD image sensor does not have, which can read an image signal of only an arbitrary region. Therefore, in Patent Document 1, the image area is divided into a plurality of sub areas, and the timing for executing the adaptive control is advanced by reading out the area for obtaining parameters necessary for adaptive control such as exposure time control and autofocus in advance. A method has been proposed.

JP 2008-35279 A

  In general, in a CMOS sensor used for moving image capturing, the charge accumulation time of a pixel is controlled by a rolling shutter operation. FIG. 2 shows a timing chart of the rolling shutter operation. In FIG. 2, the reset timing of the image sensor in units of rows is indicated by “black circles”, and the readout timing of the units is indicated by “white circles”. In the rolling shutter operation, the period from reset to read is defined as the accumulation time by sequentially shifting the line to be reset and the line from which the signal is read. Here, the accumulation time is made constant by controlling the period Δt from the reset timing to the read timing to be the same for all lines. However, the absolute time actually accumulated is shifted between the first and last lines of the scanning line. For this reason, there is a problem of rolling shutter distortion in which a captured image is distorted when a moving subject is imaged or when the imaging apparatus itself is moving in panning imaging or the like. Here, in Patent Document 1, distortion is estimated from a plurality of frames, and the distortion is compensated by image processing. However, in order to compensate for the distortion, it is necessary to perform complicated image processing, and the circuit scale of the image processing unit increases. Further, in the method of excluding a frame with a large degree of distortion from the subject of synthesis, there is a problem that the degree of distortion is large in all frames, and the image quality after synthesis is deteriorated when the degree of distortion is different between frames.

According to a first aspect of the present application, a CMOS sensor having a plurality of pixels that receive light and convert it into an image signal, drive control means for controlling accumulation and readout of the CMOS sensor,
Camera signal processing means for performing development processing on the image signal read from the CMOS sensor,
The drive control unit divides pixels to be read from the CMOS sensor into a first pixel group and a second pixel group, and reads the second pixel group after reading the first pixel group.

  The second invention according to the present application is characterized in that the pixels included in the first group are discretely arranged, and the second group is arranged at other positions.

  According to a third aspect of the present application, the camera includes a distortion estimation unit that estimates a distortion amount when the CMOS sensor is driven by a rolling shutter, and the camera signal processing unit is configured to perform a first operation according to a prediction result of the distortion prediction unit. The method of controlling the pixel group and the second pixel group is controlled.

A fourth invention according to the present application includes a motion detection unit that detects a motion of the imaging device itself,
The distortion estimation means estimates the amount of distortion based on the drive control content of the drive control means and the detection result of the motion detection means.

5th invention based on this application is equipped with the image comparison means to compare between images,
The distortion estimation unit estimates a distortion amount based on a comparison result of the image comparison unit.

  A sixth invention according to the present application is characterized in that each pixel of the CMOS sensor includes a color filter that transmits light of different color components, and the first pixel group includes a specific color.

  The seventh invention according to the present application is characterized in that the camera signal processing means performs development processing only with the image signals of the first or second pixel group.

  According to the present invention, by controlling the dividing method for dividing the pixels of the CMOS sensor into a plurality of groups and the combining method of the divided and read images, the image of the image due to the difference in the degree of distortion between the divided images is obtained. Deterioration can be reduced.

1 is a configuration diagram of Example 1. FIG. It is explanatory drawing of rolling shutter control. It is a pixel block diagram of a CMOS sensor. It is explanatory drawing of control of Example 1. FIG. It is explanatory drawing of image composition control. It is explanatory drawing of the control of a prior art example. It is explanatory drawing of the control of Example 2. FIG. It is explanatory drawing of the drive control of Example 2. FIG. FIG. 6 is a configuration diagram of Example 3.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[Example 1]
FIG. 1 is a diagram showing the configuration of the first embodiment of the imaging apparatus of the present invention. The system control means 101 controls the lens control means 102 and the sensor drive control means 103 according to the situation at the time of shooting. The lens control unit 102 controls the position of the focus lens 104. The sensor drive control means 103 controls accumulation and readout of the CMOS sensor 105. The CMOS sensor 105 receives the light, converts it into an image signal, and outputs it to the image memory 106 and the focus determination means 107. The image memory 106 temporarily holds the input image. The focus determination unit 107 performs frequency analysis on the input image signal, calculates a focus evaluation value, and outputs the focus evaluation value to the system control unit 101. The image comparison means 108 detects differences between the plurality of images and outputs them to the image composition means 109. The image synthesis means 109 synthesizes a plurality of image signals and outputs them to the camera signal processing means 110. The camera signal processing unit 110 performs development processing such as aperture correction, gamma correction, and white balance correction on the input image and outputs the processed image to the recording unit 111. The recording means 111 records on a recording medium such as a FLASH memory.

  FIG. 3 shows a pixel configuration of the CMOS sensor 101. The number of pixels of the CMOS sensor 101 is assumed to be 4 horizontal pixels and 4 vertical pixels shown in FIG. 3A, and the color filter is described as a Bayer array of primary colors RGB shown in FIG. 4B. FIG. 4 is an explanatory diagram of sensor drive control according to the present embodiment. In FIG. 4, the reset timing for each pixel of the CMOS sensor is indicated by “black circle”, and the readout timing for each pixel is indicated by “white circle”. The control of this embodiment will be described with reference to FIGS. In this embodiment, S0, S2, S8, and S10 are set as the first group among the pixels shown in FIG. 3, and the other pixels are set as the second group.

  In FIG. 4, one image signal is read from the image sensor at the period of the vertical synchronizing signal VD. In FIG. 4, after the vertical synchronizing signal VD, reading of the first group of pixels S0, S2, S8, and S10 is performed, and then the remaining pixels of the second group of pixels are read.

  Next, processing contents of the read image data will be described. The focus determination unit 107 performs frequency analysis on the pixel data of the first group and calculates a focus evaluation value. The system control means 101 calculates the focus lens position from the focus evaluation value and outputs it to the focus lens control means 102. The image comparison means 108 detects the amount of rolling shutter distortion by pattern matching using the first group of pixels S0, S2, S8, S10 and the second group of pixels S5, S7, S13, S15. Here, in the color filter of FIG. 3, the first group of pixels is G1, the second group of pixels is G2, and Bayer arrangement R, G1, G2, and B are considered as one unit. S0 and S5 are pixels in the same unit. Since the arrangement of one pixel group in the CMOS sensor 101 is very close, the amounts of light received by the G1 and G2 pixels are very close to each other.

  On the other hand, when the absolute time of the accumulation period is different between the first group and the second group by rolling shutter control, the amount of light received may be greatly different due to distortion. Therefore, the amount of distortion due to the rolling shutter is estimated by extracting G2 from the first group image and the second group image and comparing them. If it is determined that the amount of distortion is small, the first group and the second group are combined according to the configuration of the CMOS sensor 101. If it is determined that the value is large, the G1 position is interpolated with the pixel at the G2 position as shown in FIG. As a result, image degradation can be reduced even when the first group and the second group have different degrees of distortion due to rolling shutter control.

[Example 2]
Next, a second embodiment will be described. In the first embodiment, the amount of distortion due to the rolling shutter is estimated, and if it is determined that the amount of distortion is large, interpolation processing is performed during image synthesis. For this reason, an evaluation value for adaptive processing can be acquired quickly. However, when the amount of distortion is large, the image quality is deteriorated as compared with the case of reading an image as one group. Here, the accumulation time of the sensor varies depending on the situation at the time of photographing. For example, the accumulation time is lengthened when the amount of light of an image input at night photography is low, and the accumulation time is shortened when photographing a fast-moving subject such as sports photography.

  In general moving image shooting, the readout cycle from the CMOS sensor is controlled in accordance with the frame rate of moving image recording. For this reason, when the time required for reading from the sensor is short and the accumulation time is short with respect to the moving image recording period as shown in FIG. Therefore, the second embodiment aims to prevent image quality deterioration when there is a margin for driving by changing the driving method when there is a margin for driving the CMOS sensor.

  FIG. 7 is a diagram showing a configuration of the second embodiment in the imaging apparatus of the present invention. In FIG. 7, the same components as those in FIG. 1 showing the first embodiment are denoted by the same reference numerals.

  The motion detection unit 112 detects the motion of the imaging apparatus itself with a gyro sensor or the like and outputs it to the system control unit 101. The system control means 101 estimates the distortion amounts of the first group and the second group from the accumulation time of the CMOS sensor set in the drive control means 102 and the amount of motion of the image pickup apparatus, and calculates the image composition means 106. Control the synthesis method. Further, it is determined whether or not there is a margin for driving the CMOS sensor based on the reading time from the CMOS sensor and the accumulation time. If there is a margin, the driving method for the first group and the second group is changed.

  FIG. 6B is an explanatory diagram of the CMOS sensor drive control of this embodiment. In FIG. 6B, using the period remaining in FIG. 6A, the images of the first group and the second group are read together during the drive control of the second group. As a result, the amount of distortion due to the rolling shutters of the first and second groups becomes the same, and image quality deterioration due to interpolation processing can be prevented.

[Example 3]
Next, a third embodiment will be described. In the first and second embodiments, the amount of distortion caused by the rolling shutter is estimated and controlled. However, since it is difficult to accurately determine the distortion caused by the rolling shutter, the image quality deteriorates if the distortion is erroneously determined. In addition, in an image sensor such as a CMOS sensor, a defective pixel in which the output level of the pixel is an abnormal value may occur. Since this defective pixel may be generated not only at the time of manufacture but also by using an imaging device, correction by signal processing is performed. Therefore, in the third embodiment, the first group of pixels is processed as a defective pixel, thereby preventing image quality deterioration due to erroneous determination of distortion estimation.

  FIG. 8 is a diagram showing the configuration of the third embodiment of the imaging apparatus of the present invention. In FIG. 8, the same components as those in FIG. 1 showing the first embodiment are denoted by the same reference numerals.

  The defective pixel correcting unit 113 interpolates the defective pixel by interpolating the position of the defective pixel set in advance from the surrounding pixels. The system control unit 101 outputs the defective pixel information of the CMOS sensor and the pixel information of the first group to the defective pixel correction unit 113 together. As a result, it is possible to prevent image quality deterioration due to erroneous determination of the amount of distortion caused by the rolling shutter.

101 System control means
102 Lens control means
103 Drive control means
104 Focus lens
105 CMOS sensor
106 Image memory
107 Focus judgment means
108 Image comparison means

Claims (7)

A CMOS sensor having a plurality of pixels that receive light and convert it into an image signal;
Drive control means for controlling accumulation and readout of the CMOS sensor;
Camera signal processing means for performing development processing on the image signal read from the CMOS sensor,
The drive control means divides pixels to be read from the CMOS sensor into a first pixel group and a second pixel group, and reads out the second pixel group after reading out the first pixel group. . The imaging apparatus according to claim 1, wherein the pixels included in the first group are discretely arranged, and the second group is arranged at other positions. Distortion estimation means for estimating the amount of distortion when the CMOS sensor is driven by a rolling shutter,
3. The imaging according to claim 1, wherein the camera signal processing unit controls a synthesis method of the first pixel group and the second pixel group according to a prediction result of the distortion prediction unit. apparatus. Comprising a motion detection means for detecting the motion of the imaging device itself,
The imaging apparatus according to any one of claims 1 to 3, wherein the distortion estimation unit estimates a distortion amount based on a drive control content of the drive control unit and a detection result of the motion detection unit. . Image comparison means for comparing images,
The imaging apparatus according to claim 1, wherein the distortion estimation unit estimates a distortion amount based on a comparison result of the image comparison unit. Each pixel of the CMOS sensor includes a color filter that transmits light of different color components,
The imaging apparatus according to any one of claims 1 to 3, wherein the first pixel group includes a specific color. The imaging apparatus according to claim 1, wherein the camera signal processing unit performs development processing only with the image signal of the first or second pixel group.