JP2011250133A - Imaging apparatus and control method of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To keep a blur amount of the background constant even when a change in distances to a main subject and to a background occurs for an imaging device provided with a diaphragm to control exposure.SOLUTION: In an imaging apparatus, a main subject/background distance calculation part 801 calculates distances to a main subject and to a background, and a memory 12 stores each information on distance to the main subject and to the background calculated when a blur amount of a background image is being designated or when the recording is being started, and information on a focal distance and a diaphragm value. A difference amount between the distances to the main subject and to the background stored in the memory 12 is compared with a difference amount between the distance to the main subject and to the background during photographing as needed and when the compared two amounts are different, a exposure control part 802 controls diaphragm 2 in accordance with the amount of change in the distance to the background to change a depth of field.

Description

  The present invention relates to a technique for controlling the degree of out-of-focus of a background image in a captured image.

A conventional image pickup apparatus performs exposure correction according to the brightness at the time of photographing an object by using an aperture in the photographing lens unit, an ND (Neutral Density) filter, and an electronic shutter that controls the accumulation time of the image sensor. The photographer can arbitrarily set the zoom position and change the focal length by using the zoom function of the image pickup apparatus. The depth of field changes when the F value of the aperture and the focal length change depending on the brightness and zoom position at the time of shooting the subject.
On the other hand, a high-quality image having a stereoscopic effect and a sense of depth is required for the imaging device. However, depending on the shooting environment, the depth of field becomes deeper or the depth of field becomes shallower as the aperture value changes. For this reason, it is difficult to keep the blur amount of the background image constant, and there is a problem that it is difficult to obtain an image with a sense of depth.

  In order to avoid this, the conventional imaging apparatus divides the object scene that appears on the imaging device into a plurality of regions in a two-dimensional manner, and moves the focus position of the optical system in each region, so that the main subject and the background Separate the subject. Means are provided to generate an image with a shallow depth of field by synthesizing an image focused on the main subject and an image taken at the out-of-focus position where the background subject is not in focus. (See Patent Document 1).

JP 2008-244504 A

However, the apparatus disclosed in Patent Document 1 requires an operation for moving the focus position in order to obtain an image of the main subject and an operation for moving the focus position so that the background subject is out of focus. That is, in order to generate an image with a shallow depth of field, it is necessary to move the focus position twice. Therefore, the time required to obtain one image with a shallow depth of field is increased by the time required to move the focus position. Also, moving the focus position twice to generate an image with a shallow depth of field is not suitable when continuous image data acquisition is required, such as moving image shooting or continuous shooting.
SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to keep the amount of blur of a background image constant even when a change occurs in the distance to the main subject and the distance to the background.

  In order to solve the above-described problems, an apparatus according to the present invention is an imaging apparatus including a photographing lens and an aperture for exposure control, and a calculation unit that calculates a distance to a main subject and a distance to a background, and the calculation Storage means for storing information on a distance calculated by the means, a focal length related to the photographing lens, and an aperture value related to the aperture, a distance to the main subject and a distance to the background stored in the storage means Is compared with the difference between the distance to the main subject being photographed and the distance to the background, and if the difference is different, the aperture is controlled according to the amount of change in the distance to the background. Control means for changing the depth of field.

  According to the present invention, even when the distance to the main subject and the distance to the background change, the blur amount of the background image can be kept constant.

It is a block diagram which shows the structural example of an imaging device, in order to demonstrate 1st Embodiment of this invention combined with FIG. It is explanatory drawing which shows an example of the image pick-up element which can acquire distance information with the image information of a to-be-photographed object. It is a figure explaining the basic principle of the pupil division system of a micro lens in an image sensor, and a phase difference detection system. It is a figure explaining a camera signal processing circuit and processing of CPU. It is a figure explaining an example of the interpolation processing method regarding a phase difference detection element. It is a figure explaining the principle of a depth of field. It is explanatory drawing regarding control of the depth of field according to the change of the distance to the main subject and the distance to the background. It is a flowchart which shows the example of control operation. It is a flowchart which shows the control operation example which concerns on 2nd Embodiment of this invention.

[First Embodiment]
FIG. 1 shows an outline of a configuration example of an imaging apparatus according to an embodiment of the present invention. The photographing lens 1 constitutes an imaging optical system for photographing a subject, and an aperture 2 for exposure control limits the amount of incident light so that an appropriate exposure amount of light is incident on the imaging element 3. The imaging device 3 is driven by a timing pulse output from a TG (Timing Generator) circuit 9 and converts incident light into an electrical signal by photoelectric conversion and outputs the electrical signal. The TG circuit 9 is controlled by a CPU (Central Processing Unit) 8 and outputs, for example, a drive pulse having a predetermined voltage amplitude in synchronization with the horizontal synchronization signal HD as a signal having a constant period.
The CDS and A / D circuit 4 receives the output signal of the image sensor 3 and performs a correlated double sampling process and a conversion process from an analog signal to a digital signal. The TG circuit 9 generates and transmits a drive signal to the CDS and A / D circuit 4.
The camera signal processing circuit 5 separates the output signal from the CDS and A / D circuit 4 into a luminance signal and a color signal (Y / C separation), performs video signal processing, color signal processing, gamma correction, etc. Is generated. The video signal processing circuit 6 converts the output signal from the camera signal processing circuit 5 into a video signal and outputs it to the display unit 10 and the recording medium 11. An image memory 7 provided in the video signal processing circuit 6 is a storage device for buffering processing related to image data after encoding or decoding processing. When a recording operation is started in accordance with an operation instruction from the operation unit 13 to be described later, the video signal processing circuit 6 performs an encoding process or a decoding process on the input video signal using the image memory 7. Image data subjected to a predetermined process is recorded on the recording medium 11.

The imaging apparatus includes calculation means for calculating distance information to a subject or calculation means (not shown) for detecting a focus state and calculating a movement amount of a focus adjustment lens or lens group (hereinafter referred to as a focus lens). . An output signal from the calculation means is input to the CPU 8 via the camera signal processing circuit 5. A main subject / background distance calculation unit (hereinafter referred to as a distance calculation unit) 801, an exposure control unit 802, and an optical control unit 803 shown in the CPU 8 represent processing that the CPU 8 interprets and executes a program as functional blocks. A distance calculation unit 801 in charge of distance calculation processing calculates distance information from the imaging device to the main subject and distance information from the imaging device to the background. An exposure control unit 802 controls an aperture value, a shutter speed, and the like so that proper exposure is performed when a still image or a moving image is captured. The optical control unit 803 generates drive signals for a focus lens, a zoom lens, and the like and controls their operations.
When the distance to the background relative to the distance to the main subject is changed based on the distance to the main subject, the distance to the background, and the depth of field information set during the subject shooting, the CPU 8 changes the subject to the main subject. Control the depth of field. As a result, the state of the diaphragm 2 is changed, so that the amount of blurring of the background image is controlled to be constant.
The memory 12 is connected to the CPU 8 and stores various data and programs necessary for the processing of the CPU 8. The operation unit 13 includes a selection switch (blur designation selection switch) for transmitting a photographer's operation instruction to the CPU 8 and designating a blur amount of the background image, a recording switch for instructing start and end of recording, and the like.

Next, a process for generating map data representing the distribution of distance information (hereinafter referred to as distance map) will be described with reference to FIGS. Hereinafter, a distance calculation method in which pixels obtained by pupil division are embedded in the imaging element 3 and these are used as a focus state detection element will be described.
FIG. 2 shows an example of arrangement of focus state detection elements in the image sensor 3. As shown in the enlarged view, a photographing element and a focus state detecting element are arranged. In this example, elements (indicated by squares in a round frame) shown in the pixel rows of the first and second rows, and the fourth and fifth rows are used for imaging. Detection elements (indicated by two rectangles in a circle) at the pixel positions shown in the second column, the sixth column, the tenth column,... Of the third row are used for focus state detection. Although the detection accuracy is improved when the focus state detection elements are continuously provided, the deterioration of the image is increased, so that there is a trade-off between improving the accuracy and improving the image quality. As shown by a round frame, a microlens that efficiently collects incident light is disposed on the front surface of each element.

In the focus state detection element, as shown in FIG. 3A, each light beam is incident on the pair of light receiving elements A and B by dividing the light beam into pupils. As shown in FIG. 2, a first image (hereinafter referred to as an A image) is obtained by combining the outputs of the light receiving elements A (see the left rectangle in the round frame) arranged side by side in the horizontal direction. A second image (hereinafter referred to as a B image) is obtained by combining the outputs of the light receiving element B (see the rectangle on the right side in the round frame).
Next, the focus detection principle for obtaining the focus shift amount of the imaging lens from the A and B images will be described. As shown in FIG. 3B, a subject image (A image) formed on the imaging surface by the light beam passing through the A region of the photographing lens and a subject image (B image) formed by the light beam passing through the B region. The position changes at the time of focusing, front pin, and rear pin. The larger the defocus amount, which is the distance between the imaging surface and the imaging surface, the greater the deviation between the A image and the B image, and the sign of the image deviation amount at the front pin and the rear pin is reversed. The phase difference detection method detects the defocus amount from the image shift using this.

Next, processing of the camera signal processing circuit 5 (see reference numerals 501 to 504) and the CPU 8 (see reference numerals 506 to 512) will be described with reference to FIG.
First, an interpolation process as an image capture of a pixel portion related to phase difference detection will be described. With respect to the image signal read from the image sensor 3, the pixel signal is lost at a portion corresponding to the position of the focus state detection element. For this reason, the pixel interpolation processing unit 501 calculates data corresponding to the position of the focus state detection element by interpolation using data of pixels located around the focus state detection element. The output signal of the pixel interpolation processing unit 501 is sent to the video signal processing unit 502 and becomes a signal that can be handled by the video signal processing circuit 6. As for the interpolation calculation method, as shown in FIG. 5, there is a process of simply averaging signals of pixels of the same color positioned above and below the target focus state detection element. In this example, the focus state detection element is located at the center of 5 rows and 5 columns, and “Exx” (x = 1 to 5) represents the charge level of each pixel. In the simple average method, the value of E33 at the position of the focus state detection element is calculated as an average value of E31 and E35 positioned above and below one pixel. In addition, various methods such as a method of calculating a weighted average value using charge level data of a larger number of pixels on the top, bottom, left, and right centering on the position of the focus state detection element can be employed. Since such a method is already known in the same technique as pixel defect correction, detailed description thereof is omitted.
An AF (autofocus) gate switch 503 indicated by a symbol of an on / off switch in FIG. 4 selects which part of the imaging signal A / D converted by the CDS and A / D circuit 4 is to be subjected to AF signal processing. . The AF gate switch 503 is controlled in accordance with a signal from an AF control unit 512 described later. The TV-AF signal processing unit 504 obtains a value representing the sharpness of the image by processing the signal extracted by the AF gate switch 503 with a band-pass filter or the like to extract a predetermined range of frequency components.

Next, generation of a TV-AF signal will be described. The TV-AF signal is generated, for example, by obtaining the level of a predetermined high-frequency component by filtering the imaging signal. At this time, if the filtering process is performed with the pixel portion corresponding to the position of the focus state detection element missing, a signal containing an error is generated. Therefore, the AF gate switch 503 controls which part of the video signal is subjected to the AF signal processing. That is, in addition to determining the AF area, the switch serves to prevent the TV-AF signal processing from being performed for the horizontal line of the video signal in which the pixel portion corresponding to the position of the focus state detection element is missing. . As a result, a TV-AF signal that is not affected by the missing pixel portion corresponding to the position of the focus state detection element is obtained.
Configuration units 507, 509, and 512 in the CPU 8 represent processing of the CPU 8 as functional blocks. A selector 506, which is illustrated by a changeover switch symbol, distributes the image signals A / D converted by the CDS and A / D circuit 4 to the above-described A image and B image. That is, when the selector 506 is switched to the first state, the phase difference calculation processing unit 507 acquires the data of the A image, and when the selector 506 is switched to the second state, the phase difference calculation processing is performed on the data of the B image. The part 507 acquires. The phase difference calculation processing unit 507 calculates how much the amount of deviation is in the A and B images at each position in the imaging screen, and the amount of deviation at each detected position is two-dimensionally as shown in the table 508. Manage as array data.

The distance map creation processing unit 509 calculates a focusing lens position based on the amount of deviation obtained by the phase difference calculation processing unit 507, and uses the focusing lens position and the distance table 510 to focus each area on the imaging screen. Calculate the distance. The CPU 8 holds, for example, in-focus distance data for discrete focus lens positions for each zoom lens position in the data format shown in the distance table 510. The distance map creation processing unit 509 interpolates the focus distance for the focus lens position from the data in the distance table 510, calculates the distance to the subject for each focus state detection area on the imaging surface, and creates a distance map. To do. The calculation result is managed as two-dimensional array data as shown in a table 511.
The AF control unit 512 performs focusing control by driving the focus lens based on the TV-AF signal and the distance map data. The distance map creation processing unit 509 constitutes the distance calculation unit 801 in FIG. 1, and the AF control unit 512 constitutes the optical control unit 803. The control of the diaphragm 2 is performed by the exposure control unit 802 according to a process described later.

次に、主被写体距離（Ｌｓと記す）と、撮像装置から背景までの距離（以下、背景距離といい、Ｌｈと記す）に対する被写界深度制御について、図７を用いて説明する。ここで距離Ｌｓ及びＬｈが焦点距離ｆよりも非常に大きい（Ｌｓ＞＞ｆ）と仮定しており、式（１）は、「｜Ｄｂ｜≒ｌ」となる。 Next, the principle of the depth of field with respect to the distance to the subject will be described with reference to FIG. “L” in the figure represents the distance from the imaging device to the main subject (hereinafter referred to as the main subject distance). In the same figure, when the subject is in focus with respect to the main subject distance l, the rear depth of field that is in focus at a portion farther from the imaging device than the distance l is “Db”, which is greater than the distance l. The depth of front depth of field that is in focus in the near area is “Da”. If the depth of field is “D”, the focal length of the lens is “f”, the F number of the aperture of the taking lens unit is “Fno”, and the allowable circle of confusion is “σ”, the depth of field is determined from the shooting conditions. Is represented by the following equation.

Next, depth-of-field control with respect to the main subject distance (denoted as Ls) and the distance from the imaging apparatus to the background (hereinafter referred to as background distance, denoted as Lh) will be described with reference to FIG. Here, it is assumed that the distances Ls and Lh are much larger than the focal length f (Ls >> f), and the expression (1) becomes “| Db | ≈l”.

  In the state of FIG. 7A, “Ls = Ls1” and “Lh = Lh1”, and the depth of field with respect to the main subject at this time is “d1”. In this state, when the photographer gives an instruction to hold the current amount of blur of the background image using a designation selection switch or the like provided on the operation unit 13 during shooting, Ls1, Lh1 , D1 are stored in the memory 12. Thereafter, as shown in FIG. 7B, the state changes to “Ls = Ls2” and “Lh = Lh2”. For example, when the background distance Lh is Lh2 (= Lh1 / 2), from the above relational expression, the value of the depth of field d2 is set to the depth of field corresponding to one half of d1. The aperture 2 is controlled by the exposure control unit 802. That is, when the distance Lh becomes shorter after the background image blur amount designation operation than the distance Lh1 immediately after the background image blur amount is designated by the photographer's operation instruction (Lh = Lh2). The aperture control is performed according to the ratio of the distance Lh2 to the distance Lh1. At that time, control is performed to reduce the aperture F value so that the depth of field d2 becomes small, that is, the depth of field becomes shallow. On the other hand, when the distance Lh increases after the blur amount is specified, the aperture F value is set so that the depth of field d2 increases, that is, the depth of field increases, according to the ratio of the distance Lh2 to the distance Lh1. Control to increase the value is performed. With this control, the depth of field is controlled according to the amount of change in the background distance even when the background distance changes with respect to the main subject distance after the photographer specifies the amount of blur in the background image. A background image having a certain amount of blur can be generated.

Next, control for keeping the blur amount of the background image constant will be described using the flowchart of FIG.
After the start of photographing (S1000), the distance map creation processing unit 509 calculates distance information from the subject image currently being photographed, and generates distance map data (S1001). The distance calculation unit 801 determines the main subject based on the distance map data and the focus position, and calculates the main subject distance and the background distance (S1002).
Next, the CPU 8 detects that a defocus amount designation selection unit, for example, a designation selection switch provided in the operation unit 13 is operated (S1003). Next, the CPU 8 stores each data of the main subject distance Ls1 and the background distance Lh1 at this time in the memory 12 (S1004). Next, the CPU 8 acquires focal length detection information obtained from a zoom position detection unit (not shown) and stores it in the memory 12 (S1005). The CPU 8 detects the aperture F value at this time and stores it in the memory 12 (S1006). The CPU 8 calculates depth-of-field information d1 based on the detected aperture F value, focal length f, and main subject distance Ls1, and stores it in the memory 12 (S1007).

  When the background distance Lh changes to Lh2 (see FIG. 7B), the distance map creation processing unit 509 calculates distance information again and updates the distance map. The main subject distance Ls2 and the background distance Lh2 are calculated again using the updated distance map (S1008). The CPU 8 compares | Lh1-Ls1 | and | Lh2-Ls2 | and determines whether or not they are equal (S1009). If the determination result is “| Lh1−Ls1 | = | Lh2−Ls2 |”, shooting continues with the current aperture F value maintained (S1010). On the other hand, if | Lh1-Ls1 | is not equal to | Lh2-Ls2 |, the CPU 8 calculates the ratio “Lh2 / Lh1” between Lh2 and Lh1 (S1011). From this calculation result, the optimum depth of field d2 for the main subject is calculated so that the background image has a certain amount of blur. That is, the value of d2 is obtained from “d2 = (Lh2 / Lh1) × d1” (S1012). After calculating the aperture F value corresponding to the calculated depth of field d2, the CPU 8 performs drive control of the aperture 2 (S1013). With the aperture F value changed, the exposure control unit 802 controls the electronic shutter of the image sensor 3 and changes the ND filter in the photographing lens unit to perform exposure correction (S1014).

  In the first embodiment, a distance map is created during subject photographing, and data of the main subject distance Ls1, the background distance Lh1, the focal length f, the aperture F value, and the depth of field d1 when the blur amount is designated. Is stored in the memory. Then, the main subject distance Ls2 and the background distance Lh2 after the blur amount designation is executed are calculated again. By comparing these with the stored data, the determination processing is performed for the difference amounts | Lh1-Ls1 | and | Lh2-Ls2 | between the main subject distance and the background distance. When a difference occurs in the difference amount, the depth of field is changed in accordance with the amount of change in the background distance (Lh2 / Lh1), thereby making it possible to control the blur amount of the background image to be constant.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the second embodiment, when the start of recording is instructed from a recording standby state, the amount of blur of the background image is controlled to be constant. Since the configuration of the imaging apparatus according to the second embodiment is the same as that of the first embodiment, the flow of the control operation will be described with reference to the flowchart shown in FIG. Hereinafter, S1100 to 1102 different from FIG. 8 will be described, and the same processes as in the first embodiment will be omitted by using the step numbers that have already been used.
After S1002, an operation member such as a recording switch provided in the operation unit 13 is operated in S1100, and recording starts. In S1101, the CPU 8 calculates the main subject distance Ls1 and the background distance Lh1 immediately after the start of recording from the distance map data, stores them in the memory 12, and proceeds to S1005.
If | Lh1-Ls1 | and | Lh2-Ls2 | are not equal in S1009, the processing proceeds to S1102 through the processing of S1011 to 1014. When the photographer operates an operation member such as a recording switch to instruct the imaging apparatus to end recording, the operation ends when the recording operation ends.

  According to the second embodiment, even when the imaging apparatus is remotely operated by a recording start or end trigger signal, the amount of blur of the background image can be controlled to be constant.

1 Shooting Lens 2 Aperture 8 CPU
12 Memory 801 Main subject / background distance calculation unit 802 Exposure control unit 803 Optical control unit

Claims (5)

An imaging device having a photographing lens and an aperture for controlling exposure,
A calculation means for calculating the distance to the main subject and the distance to the background;
Storage means for storing information of the distance calculated by the calculation means, the focal length of the photographing lens, and the aperture value of the diaphragm;
The difference amount between the distance to the main subject and the distance to the background stored in the storage unit is compared with the difference amount between the distance to the main subject being photographed and the distance to the background, and the difference amount is An imaging apparatus comprising: a control unit that controls the diaphragm according to the amount of change in the distance to the background when there is a difference, and changes the depth of field.   2. The storage unit according to claim 1, wherein the storage unit stores information on a distance to the main subject and a distance to the background calculated by the calculation unit when a blur amount of a background image is designated. Imaging device.   The imaging apparatus according to claim 1, wherein the storage unit stores information on a distance to the main subject and a distance to the background calculated by the calculation unit when recording is started.   When the difference amount is different, the control means determines the depth of field according to the ratio of the distance to the background stored in the storage means and the distance to the background calculated by the calculation means after the time point. The imaging apparatus according to claim 2, wherein exposure control is performed by calculating the aperture value corresponding to the depth of field and performing exposure control. A method for controlling an image pickup apparatus having a photographing lens and an aperture for exposure control,
A calculation step for calculating the distance to the main subject and the distance to the background;
A storage step of storing the distance calculated in the calculation step, the focal length related to the photographing lens, and the aperture value related to the aperture;
The difference amount between the distance to the main subject and the distance to the background stored in the storing step is compared with the difference amount between the distance to the main subject being photographed and the distance to the background, and the difference amounts are different. A control method for an imaging apparatus, comprising: a step of changing the depth of field by controlling the diaphragm according to a change amount of a distance to a background.

Images