JP2022185619A - Imaging device - Google Patents

Abstract

To suppress reduction in the image quality of a dynamic image or live view image after OB clamp processing.SOLUTION: An imaging device comprises: an imaging element which has a pixel included in an effective region and a pixel included in a light-shielding region; a setting unit which sets a reference value on the basis of a signal generated from the pixel in the light-shielding region; and a correction unit which corrects the signal generated from the pixel in the effective region on the basis of the reference value. The setting unit re-sets the reference value when a prescribed imaging condition is satisfied when continuously acquiring the images.SELECTED DRAWING: Figure 1

Description

The present invention relates to an imaging device.

2. Description of the Related Art Conventionally, there has been known an imaging apparatus that corrects noise components and the like based on outputs from pixels in the OB area (for example, Japanese Unexamined Patent Application Publication No. 2002-100002).
However, due to the difference (OB step) between the output of the OB area and the output of the effective area, there is a problem that the brightness of the image is not stable for each frame and the image quality is degraded.

Japanese Patent Application Laid-Open No. 2010-200109

According to the first aspect, an imaging device sets a reference value based on an imaging element having pixels included in an effective area and pixels included in a light-shielded area, and a signal generated from the pixels in the light-shielded area. A setting unit and a correction unit that corrects a signal generated from the pixels of the effective area based on the reference value, the setting unit being configured so that a predetermined photographing condition is met when images are continuously acquired. If satisfied, reset the reference value.

1 is a block diagram schematically showing a configuration example of a main part of an imaging device according to an embodiment; FIG. It is a figure which shows the structure of an image pick-up element typically. FIG. 4 is a diagram schematically showing output values of signals from an imaging element; 4 is a timing chart at the time of moving image shooting; 5 is a flowchart for explaining the operation during moving image shooting or live view image shooting;

An imaging device according to an embodiment will be described with reference to the drawings.
FIG. 1 is a block diagram schematically showing the main configuration of a camera 1, which is an example of an imaging device according to an embodiment. The camera 1 includes a photographing optical system (imaging optical system) 2 , an imaging device 3 , an imaging control section 4 , a memory 5 , a display section 6 and an operation section 7 . The photographing optical system 2 has a plurality of lenses including, for example, a focusing lens (focus lens) and an aperture stop, and forms an image of a subject on the imaging device 3 . Note that the photographing optical system 2 may be detachable from the camera 1 .

The imaging element 3 is an imaging element such as a CMOS image sensor. The imaging device 3 receives the light flux that has passed through the imaging optical system 2 and captures the subject image formed by the imaging optical system 2 . In the imaging element 3, a plurality of pixels having photoelectric conversion units are arranged two-dimensionally (row direction and column direction). The photoelectric conversion unit is composed of a photodiode (PD). The imaging device 3 photoelectrically converts the received light to generate a signal, and outputs the generated signal to the imaging control unit 4 . Details of the imaging device 3 will be described later.

The imaging device 3 is configured such that imaging sensitivity (exposure sensitivity) can be changed, for example, in predetermined steps. The imaging sensitivity is an amount corresponding to the detection sensitivity of charges accumulated in the imaging element 3 or the amplification gain of an amplifier circuit (not shown). An operation (gain change operation) for changing the imaging sensitivity of the image sensor 3 is performed by the user using the operation unit 7 described later.
Further, the imaging device 3 operates in different drive modes when performing still image shooting, moving image shooting, and live view image shooting for displaying a through image (live view image) of a subject on the display unit 6 . In this case, the imaging device 3 varies the number of pixel rows that generate signals for still image shooting, moving image shooting, and live view image shooting. For example, in the case of still image shooting, the image sensor 3 generates signals in all pixel rows, and in the case of moving image shooting, the image sensor 3 generates signals for each predetermined pixel row. In the case of live-view image capturing, the image pickup device 3 generates an image for each predetermined pixel row with fewer pixel rows than in moving image capturing.

The memory 5 is a storage medium such as a memory card. Image data, control programs, and the like are recorded in the memory 5 . Writing data to the memory 5 and reading data from the memory 5 are controlled by the imaging control unit 4 . The display unit 6 displays an image based on image data, information related to shooting such as a shutter speed and an aperture value, a menu screen, and the like. The operation unit 7 includes various setting switches such as a release button, a power switch, a switch for switching various modes, a sensitivity setting switch for a gain change operation for changing the imaging sensitivity described above, a switch for switching the drive mode of the image sensor 3, and the like. etc., and outputs a signal based on each operation to the imaging control unit 4 .

The imaging control unit 4 is composed of a processor such as a CPU, FPGA, and ASIC, and a memory such as ROM and RAM, and controls each unit of the camera 1 based on a control program. The imaging control unit 4 supplies signals for controlling the imaging device 3 to the imaging device 3 to control the operation of the imaging device 3 . The imaging control unit 4 drives the imaging device 3 in the drive mode to capture a subject image and generate a signal when performing still image shooting, moving image shooting, live view image shooting, or the like. .

The imaging control unit 4 performs various image processing on the signal generated by the imaging element 3 to generate image data (still image data, moving image data, live view image data).
The imaging control unit 4 has a setting unit 41 , a correction unit 42 and a control unit 43 . The setting unit 41 sets a reference value (optical black clamp value) based on a signal generated by a pixel in a light shielding area of the imaging device 3, which will be described later. The setting unit 41 resets the reference value when a predetermined photographing condition is satisfied when images are acquired continuously. The correction unit 42 corrects the black level of the signal generated by the pixels in the effective area of the imaging device 3, which will be described later, based on the reference value set by the setting unit 41. FIG. When continuously acquiring images, the control unit 43 changes the predetermined period or the number of frames, which is the period for setting the reference value, based on the imaging sensitivity set in the imaging device 3 . Details of the setting unit 41, the correction unit 42, and the control unit 43 will be described later.
At least one of the setting unit 41 , the correction unit 42 and the control unit 43 may be included in the imaging device 3 .

Next, referring to FIG. 2, the imaging device 3 of this embodiment will be described. 2A and 2B are diagrams schematically showing the configuration of the image pickup device 3. FIG. 2A shows the plane of the image pickup device 3, and FIGS. is a diagram schematically showing a cross-sectional structure of. As shown in FIG. 2A, the image sensor 3 is provided with an effective area R1 formed in the center thereof and an OB area (optical black area) R2 arranged around the effective area R1. A plurality of pixels are arranged two-dimensionally in each of the effective region R1 and the OB region R2.

FIG. 2B schematically shows the cross-sectional structure of the pixels 30a arranged in the effective region R1. The pixel 30a has a microlens 301 that passes light from the imaging optical system 2, and a photoelectric conversion unit 302 that photoelectrically converts the light that has passed through the microlens 301 to generate electric charges. The signal generated by the pixel 30a included in the effective area R1 of the image sensor 3 is an image signal corresponding to the subject image used to form the pattern of the image.

FIG. 2C schematically shows the cross-sectional structure of the pixels 30b arranged in the OB region R2. The pixel 30b has a light shielding portion 303 on the incident side of the microlens 301 in addition to the same microlens 301 and photoelectric conversion portion 302 as the pixel 30a. The light shielding portion 303 is made of, for example, an aluminum film. Note that the light shielding portion 303 may be provided for each pixel 30b, or may be provided for each of a plurality of pixels 30b. Also, the light shielding portion 303 may be arranged below the microlens 301 (between the microlens 301 and the photoelectric conversion portion 302). As described above, since the pixel 30b has the light shielding portion 303, the OB region R2 is a region (light shielding region) where the light flux from the subject does not enter. The signal generated by the pixel 30b included in the OB region R2 is used to remove the dark current component of the signal output from the pixel 30a included in the effective region R1. That is, the signal generated by the pixel 30b included in the OB region R2 is a signal unrelated to the subject image, which is not used for forming the pattern of the image.
Note that the pixels 30b shown in FIG. 2C are not limited to being arranged in the OB region R2. For example, as shown in the cross-sectional view of FIG. 2D, pixels 30c each having a microlens 301 and a light shielding portion 303 but not including a photoelectric conversion portion may be arranged in the OB region R2. Also in this case, the light shielding portion 303 may be provided for each of the plurality of pixels 30 c , or the light shielding portion 303 may be arranged below the microlens 301 .

The setting unit 41 of the imaging control unit 4 sets the reference value based on the signal generated by the pixel 30b included in the OB region R2 at the time of shooting (still image shooting, moving image shooting, live view image shooting). In this case, the setting unit 41 sets the average value of the signals generated by the pixels 30b in the OB region R2 as the reference value.
FIG. 3 is a diagram schematically showing output values of signals generated by the imaging device 3. As shown in FIG. In FIG. 3, the vertical axis indicates the output value of the signal generated by the image pickup device 3, the horizontal axis indicates the position of the pixel of the image pickup device 3 in the horizontal direction (row direction), and the signal generated by the image pickup device 3 is shown. An output value, ie, brightness, is represented by a solid line L1. In this case, the output offset component D corresponds to the output value (that is, the reference value) of the signal generated by the pixel 30b included in the OB region R2.

The correction unit 42 converts the output value of the signal generated by the pixel 30a included in the effective region R1 of the image sensor 3 into the reference value set by the setting unit 41 (that is, the average of the output values of the generated signal). As a result, the correction unit 42 outputs a signal included in the signal range P in FIG. 3, that is, an image signal from which noise due to dark current has been removed (OB clamp processing). It should be noted that the output offset component D, that is, the effect of dark current changes depending on factors such as temperature.

The setting unit 41 resets the reference value set as described above when a predetermined shooting condition is satisfied when continuously acquiring images (that is, during moving image shooting or live view image shooting). . In other words, the setting unit 41 does not reset the already set reference value if the predetermined photographing condition is not satisfied, and the correction unit 42 sets the already set reference value (that is, the reference value used for the immediately preceding frame image). value) is used to correct the image of the next frame.

The photographing condition is at least one of the elapse of a predetermined period of time, the photographing of a predetermined number of frames, the change of the imaging sensitivity, and the change of the drive mode of the image sensor 3 .
When the predetermined period T has passed since the reference value was set as described above, the setting unit 41 again sets a new reference value from the signal value from the pixel 30b included in the OB region R2. In this case, the imaging control unit 4 multiplies the frame rate tr by the number of acquired frames i to calculate the elapsed time tp (=tr×i), and the elapsed time tp is a predetermined period T (T<tp=tr×i). is exceeded, it is determined that the predetermined period T has passed. Note that the imaging control unit 4 may have a timer (not shown), and the predetermined time T may be measured by this timer. Also, the predetermined period T may be a fixed value of several seconds, for example, or may be a value that can be changed by the user. Alternatively, when the number of frames corresponding to the predetermined time T (that is, the number of frames equal to or greater than T/tr) is captured, the setting unit 41 sets a new reference value. In this case, the number of frames may be a fixed value, or may be a value settable by the user.

When the imaging sensitivity is changed, the setting unit 41 again sets a new reference value from the signal from the pixel 30b included in the OB region R2. Note that the control unit 43 changes the predetermined period T (or the number of frames corresponding to the predetermined period T) based on the changed imaging sensitivity. In this case, the control unit 43 sets the predetermined period T to be longer when the imaging sensitivity is set to high.
When the driving mode of the imaging element 3 is changed, the setting unit 41 again sets a new reference value from the signal from the pixel 30b included in the OB region R2.

Next, the timing for setting the reference value during moving image shooting or live view image shooting will be described.
FIG. 4 is a timing chart during moving image shooting. In FIG. 4, the horizontal axis represents time, indicating that the moving image shooting was started at time t0.
When moving image shooting is started, the imaging control unit 4 controls the imaging element 3 to generate a signal in order to generate an image of the first frame. The setting unit 41 sets the reference value based on the signal from the pixel 30b included in the OB region R2. The set reference value is stored in the memory 5 . The correction unit 42 reads the reference value stored in the memory 5, and corrects the signal from the pixel 30a included in the effective area R1 using the read reference value. The imaging control unit 4 performs various image processing on the signal corrected by the reference value, and generates an image of the first frame. The generated image is stored in the memory 5 and displayed on the display section 6 .

After that, until the predetermined period T elapses, the images of the second and subsequent frames are generated in the same manner. That is, for the second and subsequent frames as well, the correction unit 42 uses the reference value stored in the memory 5 (that is, the reference value used when generating the image of the first frame) to determine the effective area. The signal generated by the pixel 30a of R1 is corrected.

After the predetermined period T has passed (time t1=t0+T), the setting unit 41 resets the reference value. In FIG. 4, time t1 is the timing of photographing the n-th frame image. In this case, when generating the n-th frame image, the correction unit 42 uses the reference value (that is, the reference value stored in the memory 5) used when generating the (n−1)-th frame image. value) is used to correct the signal generated by the pixel 30a in the effective area R1.

At time t2, the setting unit 41 sets a new reference value using the signal generated by the pixel 30b in the OB region R2 of the imaging element 3 when generating the (n+1)th frame image. The setting unit 41 overwrites the reference value already stored in the memory 5 with the newly set reference value and stores it. When generating the (n+1)th frame image, the correction unit 42 uses the reference value newly stored in the memory 5 to correct the signal generated by the pixel 30a in the effective area R1. conduct.

Thereafter, during the period from time t2 until the predetermined period T elapses, the correction unit 42 performs correction using the reference value set when generating the (n+1)-th frame image.
In the above description with reference to FIG. 4, the case where the reference value is reset when the predetermined period T has passed was taken as an example, but when the imaging sensitivity is changed or the drive mode of the imaging element 3 is changed, Also when it is set, the setting unit 41 similarly resets the reference value. That is, if the imaging sensitivity is changed or the driving mode of the imaging device 3 is changed while the image of the n-th frame is being captured, the setting unit 41 changes the imaging sensitivity when the image of the (n+1)-th frame is generated. The signal from the image of the OB region R2 of element 3 is used to set the reference value.
Also, although FIG. 4 shows a timing chart for moving image shooting, even in the case of live view image shooting, the reference values are set by the setting unit 41 in the same manner as in the case of moving image shooting, and the correction unit 42 is corrected by

With reference to the flow chart shown in FIG. 5, the operation of the camera 1 during moving image shooting or live view image shooting according to the present embodiment will be described. Each processing of the flowchart shown in FIG. 5 is performed by executing a program in the imaging control unit 4 . This program is stored in the memory and is activated and executed by the imaging control section 4 .

In step S1, the imaging control unit 4 sets the value of the predetermined period T, and the process proceeds to step S2. The value of this predetermined period T can be set by the user operating the operation unit 7 . In step S2, the imaging control unit 4 adds 1 to the value of the frame counter i, and the process proceeds to step S3. In step S3, it is determined whether the value of the frame counter i is 1 or not. If the value of the frame counter i is 1, an affirmative determination is made in step S3, and the process proceeds to step S4. If the value of the frame counter i is greater than 1, a negative determination is made in step S3, and the process proceeds to step S7, which will be described later.

In step S4, the imaging control unit 4 causes the imaging element 3 to perform imaging and generate a signal, and the process proceeds to step S5. In step S5, the setting unit 41 sets a reference value using the signal generated by the pixel 30b in the OB region R2, stores it in the memory 5, and the process proceeds to step S6. In step S6, the correction unit 42 corrects the signal generated by the pixel 30a in the effective area R1 using the reference value set in step S5, and the process proceeds to step S9, which will be described later.

In step S7 after step S3 has been negatively determined, the image pickup control unit 4 causes the image pickup device 3 to pick up an image of the next frame to generate a signal, and the process advances to step S8. In step S8, the correction unit 42 corrects the signal from the pixel 30a in the effective area R1 using the reference value set in step S5, and the process proceeds to step S9. In step S9, the imaging control unit 4 calculates the elapsed time tp, and the process proceeds to step S10. In this case, the imaging control unit 4 multiplies the frame rate tr by the number of acquired frames (that is, the value of the frame counter i) to calculate the elapsed time tp (=tr×i).

In step S10, the imaging control unit 4 determines whether or not the imaging sensitivity has been changed. If the imaging sensitivity has been changed, an affirmative determination is made in step S10, and the process returns to step S2. In this case, the imaging control unit 4 resets the value of the frame counter i to zero. If the imaging sensitivity has not been changed, a negative determination is made in step S10, and the process proceeds to step S11. In step S11, the imaging control unit 4 determines whether or not the drive mode of the imaging device 3 has been changed (for example, whether live view image shooting has been switched to moving image shooting). If the driving mode of the imaging device 3 has been changed, an affirmative determination is made in step S11, and the process returns to step S2. In this case, the imaging control unit 4 resets the value of the frame counter i to zero. If the drive mode of the imaging device 3 has not been changed, a negative determination is made in step S11, and the process proceeds to step S12.

In step S12, the imaging control unit 4 determines whether or not the elapsed time tp calculated in step S9 has exceeded the predetermined period T. As shown in FIG. If the elapsed time tp exceeds the predetermined time T, an affirmative determination is made in step S12, and the process returns to step S2. In this case, the imaging control unit 4 resets the value of the frame counter i to zero. If the elapsed time tp is equal to or shorter than the predetermined time T, a negative determination is made in step S12, and the process proceeds to step S13.

In step S13, it is determined whether or not to end moving image shooting or live view image shooting. When the moving image shooting or the live view image shooting ends, for example, when the user instructs to shoot a still image or turn off the power of the camera 1, an affirmative determination is made in step S13 and the process ends. If moving image shooting or live view mode image shooting is not finished, a negative determination is made in step S13, and the process returns to step S2.

It should be noted that if the photographing condition for resetting the reference value does not include a change in the imaging sensitivity, the processing of step S10 may be skipped. Further, if the imaging conditions for resetting the reference values do not include a change in the drive mode of the image pickup device 3, the processing of step S11 may be skipped.
Further, in the case where the control unit 43 changes the predetermined period T based on the changed imaging sensitivity, if the determination in step S10 is affirmative, the control unit 43 sets a new predetermined period T, and the process proceeds to step S10. It is sufficient to return to S2.
Further, if the elapse of the predetermined period T is determined based on the number of captured frames, the process of step S9 is skipped, and if the value of the frame counter i is equal to or greater than T/tr in step S12, the process should return to step S2.

According to the embodiment described above, the following effects are obtained.
(1) The imaging device 3 has pixels 30a included in the effective area R1 and pixels 30b included in the OB area R2 (light-shielded area). The setting unit 41 sets the reference value based on the signal generated from the pixel 30b in the OB region R2, and the correction unit 42 corrects the signal generated from the pixel 30a in the effective region R1 based on the reference value. . Then, the setting unit 41 resets the reference value when a predetermined photographing condition is satisfied when images are acquired continuously. This makes it possible to perform OB clamp processing using the same reference value when generating images of a plurality of frames during moving image shooting or live view image shooting.
If there is a difference (OB step) in the output value between the signal generated by the pixels in the OB area and the signal generated by the pixels in the effective area, the signal generated by the pixels in the effective area after OB clamp processing. output value is higher or lower than expected. If an OB level difference occurs in each frame when shooting a moving image or a live view image, and the OB clamping process is performed by setting a reference value each time an image of each frame is generated, the generated moving image or live view The brightness of the entire image varies from frame to frame, causing flickering. In the present embodiment, since the same reference value is set for a plurality of frame images and the OB clamping process is performed, compared with the case where the reference value is set each time an image of each frame is generated, It is possible to suppress the occurrence of flickering and suppress deterioration of image quality.

(2) The setting unit 41 resets the reference value each time a predetermined period T elapses or a predetermined number of frames are captured. As a result, it is possible to suppress the occurrence of flickering for each frame compared to the case where the reference value is set for each frame and the clamping process is performed. Level corrections can be made.

(3) The setting unit 41 resets the reference value when the imaging sensitivity is changed. This eliminates the need to secure an area in the memory 5 for storing a reference value for each imaging sensitivity. In addition, since it is not necessary to adjust a fixed reference value for each imaging element 3, the takt time during production can be shortened.

(4) The control unit 43 changes the predetermined period T or the predetermined number of frames based on the imaging sensitivity. As a result, for example, when a high imaging sensitivity is set, noise increases. It is possible to suppress the occurrence of flickering in moving images or live view images due to frequent changes in the reference value.

(5) The setting unit 41 resets the reference value when the driving mode of the imaging device 3 is changed. This eliminates the need to secure an area in the memory 5 for storing the reference value for each drive mode. In addition, since it is not necessary to adjust a fixed reference value for each imaging element 3, the takt time during production can be shortened.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Other aspects conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention.

DESCRIPTION OF SYMBOLS 1... Camera 3... Imaging element 4... Imaging control part 30a, 30b... Pixel 41... Setting part 42... Correction part 43... Control part 303... Light shielding part

Claims (8)

an imaging device having pixels included in the effective area and pixels included in the light-shielded area;
a setting unit that sets a reference value based on a signal generated from a pixel in the light shielding area;
a correction unit that corrects a signal generated from pixels in the effective area based on the reference value;
The imaging device, wherein the setting unit resets the reference value when a predetermined imaging condition is satisfied when the images are continuously acquired. The imaging device according to claim 1,
The imaging device, wherein the correction unit corrects a black level of a signal generated from pixels in the effective area. The imaging device according to claim 1 or 2,
The imaging apparatus, wherein the setting unit resets the reference value each time a predetermined period of time elapses or a predetermined number of frames are captured. In the imaging device according to any one of claims 1 to 3,
The imaging device, wherein the setting unit resets the reference value when the imaging sensitivity is changed. In the imaging device according to claim 3,
The setting unit resets the reference value when the imaging sensitivity is changed,
An imaging apparatus comprising a control unit that changes the predetermined period or the predetermined number of frames based on the imaging sensitivity. In the imaging device according to any one of claims 1 to 5,
The imaging device, wherein the setting unit resets the reference value when the driving mode of the imaging element is changed. In the imaging device according to any one of claims 1 to 6,
the pixels included in the effective area photoelectrically convert light incident through an imaging optical system to generate a signal;
The imaging apparatus according to claim 1, wherein the pixels included in the light-shielding region generate signals without receiving incident light through the imaging optical system. In the imaging device according to claim 7,
The pixel included in the effective area includes a microlens through which light from the imaging optical system passes, and a photoelectric conversion unit that photoelectrically converts the light that has passed through the microlens to generate an electric charge,
The pixel included in the light-shielding region includes a microlens through which light from the imaging optical system passes, a photoelectric conversion unit that photoelectrically converts the light that has passed through the microlens to generate charges, and an incident side of the microlens. and a light shielding unit provided in the imaging device.

Images