JP6355324B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a control method thereof.

  An imaging apparatus equipped with a technique called Computational Photography has been studied. In addition to image brightness and color data, computational photography captures new information that has not been recorded by conventional imaging devices with an image sensor, records it with image data, and uses the recorded information to create an image. This is a technique of performing various processes with a processing LSI. For example, the imaging apparatus records light information (light field) such as information on the incident angle of light from the subject and depth. By using information that could not be recorded by these conventional imaging devices, focus, depth of field by iris, focal length, and the like can be accurately reproduced by subsequent image processing. There has been proposed a camera that can change the focus position of an actually captured image later using a recorded light field.

  As a typical light field acquisition method, a method for acquiring images from multiple viewpoints will be described. For example, a plurality of images can be acquired by changing the focal length with a single imaging device, and the acquired plurality of images can be left as a single image so that the focal length of the image can be freely changed later by image processing. Can be changed. Further, by moving the camera horizontally or vertically and acquiring a plurality of images, it is also possible to acquire subject distance information from the parallax of the images. These methods use a single imaging device (imaging device).

  Patent Document 1 discloses a system that samples and aliases a light field for a three-dimensional display device. Patent Document 2 discloses an imaging apparatus that uses a plurality of lenses having different focal lengths to acquire images having different focal lengths.

JP 2008-257686A JP 2000-32354 A

  By arranging a plurality of imaging devices in an array, it is conceivable to increase the amount of information that can be obtained from the light field by acquiring a plurality of pieces of data while providing the images to be synchronized. However, this technique requires a plurality of imaging devices, increasing power consumption and increasing the size of the device.

  An imaging apparatus that takes a picture using a plurality of lenses having different focal lengths from a single imaging apparatus can be considered, but this imaging apparatus has the following problems. For example, it is assumed that the user selects an output image of a lens having a short focal length as a camera preview image during moving image shooting. When the user manually performs exposure processing while viewing the preview image, the output image of the lens with a longer focal length than the lens with the focal length used for the preview image is the same as the output image of the lens with a shorter focal length under the same exposure conditions. It will be darker. As a result, when the user tries to change the zoom position by editing after shooting, a luminance difference occurs when switching the output image of the lens, resulting in an uncomfortable image as a moving image.

  The present invention provides an imaging apparatus capable of obtaining an image that does not feel uncomfortable when the output image is switched even when the user manually performs exposure editing on an output image obtained through a plurality of lenses having different focal lengths. Objective.

An imaging apparatus according to an embodiment of the present invention includes a plurality of lenses having different focal lengths, an imaging element that photoelectrically converts light incident through each of the lenses, and an output image based on an image signal from the imaging element. Display means for displaying the output image selected by the user as a preview image, the luminance level of the output image selected by the user is calculated as a target value for exposure, and the focal length is different from the lens of the focal length corresponding to the output image. A control unit that controls exposure of an output image corresponding to the lens based on the calculated target value, and the control unit performs exposure control on the output image corresponding to the lens of each focal length with the target value. In addition, a luminance difference occurs between the output image selected by the user and the output image corresponding to a lens having a focal length different from the lens having the focal length corresponding to the output image. To notify a warning if the put away.

  According to the imaging apparatus of the present invention, even when the user manually performs exposure editing on an output image obtained through a plurality of lenses having different focal lengths, an image that does not feel uncomfortable when the output image is switched can be obtained. An object is to provide an imaging device.

It is a figure which shows the structure of the imaging device of this embodiment. It is an example of a functional block diagram of a digital video camera. 3 is a flowchart for explaining an operation process of the imaging apparatus according to the first embodiment. 6 is a flowchart illustrating an operation process of the imaging apparatus according to the second embodiment.

Example 1
FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to the present embodiment. An imaging apparatus shown in FIG. 1 is a digital video camera 101. Of course, the application range of the present invention is not limited to a digital video camera. The present invention is also applicable to a digital single lens reflex camera or a digital still camera.

  In front of the video camera 101, a plurality of lenses and sensors having different focal lengths are arranged in a 4 × 4 matrix. Specifically, as the lens group 102, lenses are arranged in a 4 × 4 matrix. In FIG. 1, the first lens is a lens having a focal length A, the second lens is a lens having a focal length B, the third lens is a lens having a focal length C, and the fourth lens is a lens having a focal length D. It is a lens.

  The lens with the focal length A has the shortest focal length. That is, the lens with the focal length A is the lens on the widest side. In FIG. 1, the focal length becomes longer as the lens moves to the lens having the focal length B, the lens having the focal length C, and the lens having the focal length D. The lens with the focal length D has the longest focal length. That is, the lens with the focal length D is the lens on the most Tele side.

  The digital video camera 101 has a plurality of lenses having different focal lengths and lenses having the same focal length, and records an output image obtained through each lens. Thereby, the zoom position and focus can be changed even after shooting.

  Here, an object of the present invention is to improve a sense of incongruity due to a change in the zoom position after shooting. For this reason, the four lenses having the same focal length related to the focus change after shooting are not necessary for the description. Therefore, the digital video camera 101 will be described below as a video camera having lenses having four different focal lengths.

  FIG. 2 is an example of a functional block diagram of the digital video camera shown in FIG. The lens group 201 includes a plurality of lenses having different focal lengths. In this example, the lens group 201 has four types of lenses whose focal lengths are A, B, C, and D described with reference to FIG.

  The light incident through the lens group 201 is coupled to a solid-state imaging device such as a CMOS sensor 202 and is photoelectrically converted into an electrical signal. One solid-state imaging device is arranged for one lens, and photoelectrically converts light incident through each lens into an electrical signal. The analog signal subjected to photoelectric conversion is converted into a digital signal by the A / D converter 203.

  The signal converted into the digital signal is input to the image signal processing unit 204. The image signal processing unit 204 includes an AGC 205, a signal processing unit 206, and a luminance level detection unit 207. The AGC performs automatic gain control processing (AGC processing) on the input digital signal. The signal processing unit 206 performs various signal processing such as gamma correction and white balance correction on the AGC-processed signal. The luminance level detection unit 207 detects the luminance level of the output signal from each signal processing unit 206. The output from the signal processing unit 206 is input to the recording control unit 208 and the display control unit 210.

  The signal input to the recording control unit 208 is recorded on the recording medium 209. The signal input to the display control unit 210 is displayed by the video signal display unit 210. At this time, the CPU 212 selects a preview image to be displayed. The CPU 212 controls the entire digital video camera. The image sensor control circuit 213 controls the shutter speed of the image sensor based on the signal output from the image signal processing unit 204.

  FIG. 3 is a flowchart illustrating an operation process of the imaging apparatus according to the first embodiment. The process according to this flowchart is executed by the CPU 212 reading and executing a predetermined control program.

  First, the luminance level detection unit 207 detects the luminance level of the subject based on the signal output from the signal processing unit 206 (step S301). At this time, the brightness of the subject is detected by using the center emphasis method from the output of a lens having a short focal length (a lens having a focal length A).

  Next, the CPU 212 calculates a target value for exposure based on the luminance level detected in step S301 (step S302). Subsequently, the CPU 212 executes exposure control (AE) for the output of each lens (step S303). Specifically, the CPU 212 controls the image sensor control circuit 213 to change the shutter speed, or controls the AGC control unit 205 to change the digital gain in the AGC process.

  Next, the CPU 212 selects the output of the lens having the focal length used for the preview image in accordance with the user operation (step S304). Subsequently, the CPU 212 determines whether or not the user has manually performed the exposure process on the selected preview image (step S305). If the user has not performed the exposure process on the selected preview image, the process ends. If the user manually performs an exposure process on the selected preview image, the process proceeds to step S306.

  Next, the luminance level detection unit 207 detects the luminance level of the video used for the preview image (step S306). Then, the CPU 212 calculates an exposure target value of a lens having a focal length whose output signal is not used for the preview image based on the luminance level detected in step 306 (step S307). Then, the CPU 212 uses the calculated exposure target value of each lens as a target value, and performs exposure control on the output of the lens that is not used in the preview image (step S308).

  In step S304, when the CPU 212 selects an output image of the wide-side lens in accordance with a user operation, the CPU 212 may execute the following processing in steps S306 to S308. That is, the CPU 212 controls the exposure of the output image of the tele side lens with the brightness level of the output image on the wide side as an exposure target value.

(Example 2)
The imaging apparatus according to the first embodiment performs AE on the output of another lens based on the luminance level of the preview image of the lens at the focal length selected by the user. However, depending on the luminance of the subject, a luminance difference may occur between a lens with a short focal length and a lens with a long focal length. For example, it is assumed that an output image of a lens with a short focal length is used as a preview image and a subject with low illuminance is photographed. At this time, if it is attempted to brighten the output image within the maximum range of exposure control (maximum gain, long-second exposure), the lens having a long focal length is darker because the amount of incident light is smaller than that of a lens having a short focal length. Therefore, the imaging apparatus according to the second embodiment notifies a warning when a luminance difference occurs between the output images of lenses having different focal lengths.

  FIG. 4 is a flowchart illustrating an operation process of the imaging apparatus according to the second embodiment. The process according to this flowchart is executed by the CPU 212 reading and executing a predetermined control program.

  Steps S401, S402, S404, S406, S407, S408, S409, and S412 are the same as steps S301, S302, S303, S304, S305, S306, S307, and S308, respectively, in FIG.

  In the second embodiment, in step S403, the CPU 212 determines whether a luminance difference is generated between output images corresponding to lenses having different focal lengths based on the exposure target value calculated in step S402 (step S403). S403). That is, the CPU 212 determines whether a luminance difference occurs between output images from lenses having different focal lengths when AE is executed with the calculated exposure target value. If the CPU 212 determines that no luminance difference occurs between output images from lenses with different focal lengths, the process proceeds to step S404. If the CPU 212 determines that a luminance difference occurs between output images from lenses with different focal lengths, the process proceeds to step S405.

  In step S405, the CPU 212 changes the exposure target value calculated in step S402 to an exposure target value that does not cause a luminance difference between output images from lenses having different focal lengths. Then, the CPU 212 executes AE based on the changed exposure target value (step S405), and the process proceeds to step S406.

  In the second embodiment, in step S410, the CPU 212 determines whether a luminance difference is generated between output images from lenses having different focal lengths when AE is executed with the exposure target value calculated in step S409 (step S410). S410). If the CPU 212 determines that there is no luminance difference between output images from lenses with different focal lengths, the process proceeds to step S412. If the CPU 212 determines that a luminance difference occurs between output images from lenses with different focal lengths, the process proceeds to step S411.

  In step 411, the CPU 212 issues a warning to inform the user that a luminance difference occurs between output images from lenses having different focal lengths. For example, the CPU 212 instructs the display control unit 210 to display a warning on the screen.

  The present invention is not limited to the first and second embodiments described above, and various forms within the scope of the present invention are also included in the present invention. A part of the above-described embodiments may be appropriately combined.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

101 Digital Video Camera 201 Lens Group 202 Image Sensor 207 Brightness Level Detection Unit 212 CPU

Claims (6)

Multiple lenses with different focal lengths;
An image sensor that photoelectrically converts light incident through each of the lenses;
Display means for displaying, as a preview image, an output image selected by a user among output images based on image signals from the image sensor;
The brightness level of the output image selected by the user is calculated as a target value for exposure, and an output image corresponding to a lens with a focal length different from the lens with the focal length corresponding to the output image is calculated based on the calculated target value. And control means for controlling exposure ,
When the control unit performs exposure control on the output image corresponding to the lens having each focal length with the target value, the output image selected by the user and the lens having the focal length different from the lens having the focal length corresponding to the output image are selected. An image pickup apparatus that issues a warning when a luminance difference occurs between an output image corresponding to a lens and the output image . The control means determines whether or not a luminance difference occurs between output images corresponding to lenses having different focal lengths when performing the exposure control based on the target value, and corresponds to each of lenses having different focal lengths. The imaging apparatus according to claim 1, wherein a warning is notified when a luminance difference occurs between output images to be output. When the user selects an output image of the wide-side lens, the control means controls the exposure of the output image of the tele-side lens with the brightness level of the wide-side output image as a target value for exposure. The imaging device according to claim 1 or 2, wherein The control means determines whether the user has manually exposed the lens corresponding to the selected output image, and the user manually exposed to the lens corresponding to the selected output image. 4. The imaging apparatus according to claim 1, wherein, when processing is performed, a luminance level of the output image is calculated as the target value. 5. The control means is configured to notify a warning when the user manually performs an exposure process for brightening the output image within a maximum range of exposure control for a lens corresponding to the selected output image. The imaging device according to any one of claims 1 to 4. A control method for an imaging apparatus, comprising: a plurality of lenses having different focal lengths; and an imaging element that photoelectrically converts light incident through each of the lenses,
Accepting an output image selected from an output image based on an image signal from the image sensor by the user as a preview image;
The brightness level of the output image received as the preview image is calculated as a target value for exposure, and the output image corresponding to a lens having a focal length different from the lens having the focal length corresponding to the output image is calculated as the calculated target value. possess a step of controlling exposure on the basis of,
When an output image corresponding to a lens having each focal length is controlled with the target value, an output corresponding to a lens having a focal length different from the output image selected by the user and the lens having a focal length corresponding to the output image. A control method characterized by notifying a warning when a luminance difference occurs with an image .

Images