JP2018014654A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method that addresses the problem in which time for image display or image processing is longer than the time for acquisition or processing of a single image, by performing the image processing after acquiring a plurality of images having different phases from an imaging device.SOLUTION: An imaging apparatus acquires a plurality of images having different phases from an imaging device, and when the imaging apparatus performs image processing on an image having each phase to store, it creates a thumbnail, creates a simple display image, and performs the image processing such as subject discrimination, using only the image having the phase called from the imaging device initially.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an imaging apparatus for a digital camera or a video camera, and more particularly to an imaging apparatus that captures images having a plurality of phases using an imaging element.

  There has been proposed an imaging apparatus that can divide an exit pupil of a photographing lens (imaging optical system) into a plurality of regions and simultaneously photograph a plurality of parallax images corresponding to the divided pupil regions (Patent Document 1). .)

JP-A-2015-119416

  However, performing image processing after acquiring a plurality of images having different phases from the image sensor causes a problem that the time for image display and image processing is longer than that for single image acquisition and image processing. . In addition, even if it is not a main image processed by a plurality of images, if a small image such as a thumbnail, a face position such as face detection, a high image quality such as discrimination of a scene is not required, Even an image taken with one phase has no effect.

In order to solve the above problems, an imaging apparatus according to the present invention provides:
When acquiring multiple images with different phases from the image sensor, and performing image processing on each phase image and saving it, creating thumbnails and displaying them simply using only the image of the phase that was first called from the image sensor Perform image processing such as image creation and subject discrimination, and create a display image using the phase image that was first called from the image sensor, and capture the second and subsequent phase images during image processing such as subject discrimination Call from the element.

  ADVANTAGE OF THE INVENTION According to this invention, in the imaging device which acquires the several image from which an image differs from an imaging element, the time concerning the display and a process at the time of acquiring and processing a several phase image can be shortened.

It is a block diagram which shows the function structural example of the imaging device which concerns on embodiment. It is a figure which shows the structure of an image pick-up element. It is a figure which shows the call of the signal from an image pick-up element. It is a figure which shows the order of signal processing.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiment described below is merely an example, and the present invention is not limited to the configuration described in the embodiment.

  FIG. 1 is a block diagram illustrating a configuration of a focus detection device and an imaging device including an image correction unit according to an embodiment of the present invention.

  In the figure, an image sensor 100 photoelectrically converts an optical image of a subject formed by a photographing optical system into an electric signal. The image sensor 100 is controlled by a central processing unit 103, which will be described later, and takes a still image or a moving image.

  An analog front end (hereinafter referred to as AFE) 101 performs digital conversion on an analog image signal output from the image sensor 100 in accordance with gain adjustment or a predetermined quantization bit.

  A timing generator (hereinafter referred to as TG) 102 controls drive timing of the image sensor 100 and the AFE 101. In the present embodiment, the AFE 101 and the TG 102 are arranged outside the image sensor 100, but they may be configured to be built in the image sensor.

  As described above, the central processing unit (hereinafter referred to as CPU) 103 executes a program for controlling each part of the image sensor.

  The operation unit 104 sets a shooting command, shooting conditions, and the like to the CPU 103.

  The display unit 105 displays captured still images, moving images, menus, and the like.

  The RAM 106 has a function of an image data storage unit that stores image data digitally converted by the AFE 101 and image data processed by an image processing unit 108 described later, and a function of a work memory when the CPU 103 operates. . In this embodiment, these functions are performed using the RAM 106, but other memories may be applied as long as the access speed is sufficiently high and there is no problem in operation. Is possible.

  The ROM 107 stores a program that is loaded and executed by the CPU 103 to control the operation of each unit, and a coefficient group used in various circuits. Here, in the present embodiment, a flash memory is shown, but this is only an example, and other memories can be applied as long as the access speed is sufficiently high and there is no problem in operation. . The image processing unit 108 performs processing such as correction and compression of a captured still image or moving image.

  In addition, a function of separating A image data and B image data, a function of combining A image data and B image data, a function of correcting other images, and a function of generating still images and moving images are provided.

  The AF calculation unit 109 calculates the driving amount of the focus lens using the correlation calculation result output from the correlation calculation unit 120.

  The flash memory 110 is a detachable flash memory for recording still image data and moving image data. In this embodiment, a flash memory is applied as a recording medium, but other data writable nonvolatile memory may be used. Moreover, the form which incorporated these recording media may be sufficient.

  The focal plane shutter 111 adjusts the exposure time during still image shooting. In the present embodiment, the exposure time of the image sensor 100 is adjusted by a focal plane shutter. However, the present invention is not limited to this, and the image sensor 100 has an electronic shutter function, and the exposure time is controlled by a control pulse. The structure which adjusts may be sufficient.

  The focus drive circuit 112 changes the focal position of the optical system, controls the drive of the focus actuator 114 based on the focus detection result of the AF calculation unit 109, and drives the third lens 119 forward and backward in the optical axis direction. Adjust the focus.

  The aperture driving circuit 113 controls the aperture diameter of the aperture 117 by controlling the driving of the aperture actuator 115.

  The first lens 116 is disposed at the tip of the photographing optical system (common optical system) and is held so as to be able to advance and retract in the optical axis direction.

  The diaphragm 117 adjusts the light amount at the time of photographing by adjusting the aperture diameter. The diaphragm 117 and the second lens 118 are integrally moved forward and backward in the optical axis direction, and a zooming function (zoom function) is realized by interlocking with the forward and backward movement of the first lens 116.

  The third lens 119 adjusts the focal point of the photographing optical system by advancing and retreating in the optical axis direction. The correlation calculation unit 120 performs correlation calculation using the pixel signal output from the image sensor 100.

  Next, the configuration of the image sensor 100 will be described with reference to FIG.

  A pixel array of the image sensor 100 is shown in FIG.

  In FIG. 2, the pixel array is a Bayer array, where R is a red color filter, B is a blue color filter, Gr and Gb are green color filters, and the microlens 100a is a microlens array. To do. The photodiodes (PD) 100b and 100c constitute an A image photoelectric conversion unit and a B image photoelectric conversion unit, which will be described later, as photoelectric conversion means for performing photoelectric conversion. Each pixel has a configuration in which one microlens 100a is disposed on the top of two PDs.

  That is, the focus detection pixel includes a plurality of photoelectric conversion units for one microlens. When the imaging area sharing the microlens 100a is one pixel, the pixel array 100a has h pixels arranged in the horizontal direction and v pixels arranged in the vertical direction. Signals accumulated in the PD 100b and PD 100c are converted into a voltage simultaneously or independently by a pixel transfer operation described later, and output to the outside by the above-described readout operation. The PD 100b and the PD 100c have a pupil division configuration, and separate images having a phase difference from each other are incident.

  Therefore, the signals of the PD 100b and the PD 100c can be read independently, the correlation calculation unit 120 can perform the correlation calculation process, and the AF calculation unit 109 can calculate the driving amount of the focus lens using the result. Also, separate phase images having a phase difference from each other can be used for refocus adjustment after shooting. Here, the PD 100c is an A image photoelectric conversion unit, and the PD 100b is a B image photoelectric conversion unit. Although FIG. 2 shows a configuration in which two PDs are arranged for one microlens, the present invention is not limited to this. The present invention can be applied to a configuration in which a plurality of PDs are arranged vertically or horizontally with respect to one microlens, or the pupil can be divided into four or more.

  Next, reading of signals from the image sensor 100 will be described with reference to FIG.

  FIG. 3A shows a case where A image data is read from the A image photoelectric conversion unit and B image data is read from the B image photoelectric conversion unit.

  When a shooting command is input to the operation unit 104, a read switch for A image data from the PD 100 c of the image sensor 100 is turned on, and the A image data is output to the image processing unit 108 via the AFE 101 and the CPU 103. When the reading of the A image data is completed, the A image data reading switch is turned off, and the B image data reading switch from the PD 100b of the image sensor 100 is turned on. Is output. When the reading of the B image data is completed, the B image data reading switch is turned off and the photographing is finished.

  FIG. 3B shows a case where A image data is read from the A image photoelectric conversion unit, and addition data of A image data and B image data is read from the A image photoelectric conversion unit and the B image photoelectric conversion unit.

  When a shooting command is input to the operation unit 104, a read switch for A image data from the PD 100 c of the image sensor 100 is turned on, and the A image data is output to the image processing unit 108 via the AFE 101 and the CPU 103. When the reading of the A image data is completed, the A image data reading switch is turned off.

  Next, the readout switch for the addition signal of the A image data from the PD 100 c of the image sensor 100 and the B image data from the PD 100 b of the image sensor 100 is turned on, and the added data of the A image data and the B image data undergoes image processing via the AFE 101 and the CPU 103. Is output to the unit 108. When the readout of the addition data of the A image data and the B image data is completed, the readout switch for the addition data of the A image data and the B image data is turned off, and the photographing is finished.

  Next, processing of a signal read from the image sensor 100 will be described with reference to FIG.

  FIG. 4A shows signal processing when A image data and B image data are read out.

  As soon as the reading of the A image data (S401) is completed, the image processing unit 108 performs image processing on the A image data, creates a thumbnail of the captured image, and confirms the simple image displayed on the display unit 105 for confirmation immediately after shooting. Creation and creation of an evaluation image of the subject (S402). The subject evaluation image is used for processing of subject tracking, face detection, scene discrimination, and the like.

  At the same time as the reading of the A image data is completed and image processing is started on the A image data, reading of the B image data is started (S403). When the reading of the B image data is completed, the A image data and the B image data are added by the image processing unit 108 (S404), and the added data is subjected to image processing to create a main display image (S405). The main display image is recorded in the flash memory 110 (S406). By performing simple processing that does not require image accuracy using the A image data before creating an image for main display, the processing time from image calling to various processing can be shortened.

  FIG. 4B shows signal processing when the A image data, the addition data of the A image data, and the B image data are read.

  As soon as the reading of the A image data (S407) is completed, the image processing unit 108 performs image processing on the A image data, creates a thumbnail of the captured image, and confirms the simple image displayed on the display unit 105 for confirmation immediately after the shooting. Creation and creation of an object evaluation image (S408). The subject evaluation image is used for processing of subject tracking, face detection, scene discrimination, and the like. At the same time as the reading of the A image data is completed and image processing is started on the A image data, reading of the added data of the A image data and the B image data is started (S409). When the reading of the addition data of the A image data and the B image data is completed, the addition data of the A image data and the B image data is subjected to image processing by the image processing unit 108 and an image for main display is created (S410).

  The main display image is recorded in the flash memory 110. The A image data is recorded in the flash memory 110 again as data for refocusing or the like for adjusting the in-focus position after shooting (S411). By performing simple processing that does not require image accuracy using the A image data before creating an image for main display, the processing time from image calling to various processing can be shortened.

  When the above processing is not an actual captured image processed by multiple images, small images such as thumbnails, face positions such as face detection, and high image quality such as scene discrimination are not required This is possible because there is no influence even with an image taken in one phase. Therefore, the created thumbnail can be used as a thumbnail for the main image, so that a simple image can be displayed quickly, and moving object tracking by live view display between frames during continuous shooting can be easily performed by the imaging apparatus. The subject tracking evaluation is used to determine the focus point at the time of the next shooting during continuous shooting, and the continuous shooting speed can be increased. In face detection and scene determination, the addition data of A image data and B image data is used. The time required for determining parameters for image processing can be shortened.

100 Image sensor, 100a micro lens, 100b, 100c photoelectric conversion unit

Claims (6)

An imaging device in which a plurality of pixels are arranged, and each of the pixels is obtained from an imaging device provided with a plurality of sub-pixels that receive light beams that pass through different pupil partial regions of the imaging optical system. Acquisition means for acquiring an input image
Generating means for generating a plurality of images having different phases from the input image based on a signal of a sub-pixel that receives a light beam passing through the same pupil partial region;
Having a calling means for calling a signal processing means in time series a plurality of signals having different phases;
An image pickup apparatus that performs signal processing before final image generation and storage using an image processed with a signal called first among a plurality of images having different phases. The imaging apparatus according to claim 1, wherein the process before generating and storing the final image is thumbnail creation. The imaging apparatus according to claim 1, wherein the process before generating and storing the final image is creation of a simple display image displayed on the display apparatus. The imaging apparatus according to claim 1, wherein the process before generating and storing the final image is creation of an evaluation image of a subject. The imaging apparatus according to claim 4, wherein the subject evaluation image is used for subject tracking, face detection, and scene discrimination. Of the plurality of images having different phases, the first called phase image is generated based on the signal of the sub-pixel that receives the light beam passing through the same pupil partial region, and the next called phase image includes the plurality of pupil partial regions. 2. The image pickup apparatus according to claim 1, wherein the image pickup apparatus is a combination of signals of subpixels that receive a passing light beam.