JP6198395B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an image generation technique in an imaging apparatus capable of shooting a moving image.

  2. Description of the Related Art Conventionally, there has been proposed an imaging apparatus that performs focus adjustment using a contrast in moving image shooting. In recent years, there has been proposed an imaging apparatus that is commonly called a light field camera and that can adjust the focus after acquiring an image.

  For example, Patent Document 1 discloses an imaging apparatus that performs contrast AF using sensors arranged at positions having different optical path lengths. Patent Document 2 discloses an imaging apparatus that can adjust focus after acquiring an image.

JP 2002-365517 A JP 2007-004471 A

  However, with the conventional techniques disclosed in the above-mentioned patent documents, an appropriate image may not always be obtained during moving image shooting. In other words, the technique described in Patent Document 1 can detect the focus position, but when the detected focus is out of focus, an image with a good focus state cannot be obtained immediately. Patent Document 2 discloses a method for obtaining an image in which the focus state is changed after photographing, but does not show a method for obtaining an appropriate image in synchronization with driving of a lens during moving image photographing.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an imaging device that can obtain a recorded image with a good focus state during movie shooting and obtain a lens driving signal. is there.

Imaging device according to the present invention includes an imaging device to obtain a signal containing a light field information, and the imaging element control means for outputting a signal from the imaging element at a constant cycle, the obtained signals from the previous SL imaging element set the development surface, a developing unit that generates an image signal in the developing surface by performing an addition of the signals respectively corresponding to the developing surface, the image signals in a plurality of developing surface obtained from the developing unit It used a calculation means for focus state is calculated the best development plane, based on the calculation result by the calculating means, in advance that have been set drive speed following speed and the direction taking lens focus state becomes better a lens control means for controlling, based on the calculation result by the calculating means, characterized in that it comprises a determining means for focus state to determine the best development plane as the development face of the recording image.

  According to the present invention, it is possible to provide an imaging apparatus capable of obtaining a recorded image with a good focus state during moving image shooting and obtaining a lens driving signal.

It is a figure which shows the structure of the digital camera which is one Embodiment of the imaging device of this invention, and a lens. It is explanatory drawing of an imaging optical system. It is a flowchart which shows operation | movement of the camera of one Embodiment. It is a figure explaining reconstruction of an image. It is a figure explaining the operation | movement with a focus key frame. It is a figure explaining operation | movement with the frame after a focus key frame.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1A is a central cross-sectional view of a digital camera and a photographing lens as an embodiment of the imaging apparatus of the present invention, and FIG. 1B is a block diagram showing an electrical configuration of the digital camera. In FIG. 1A and FIG. 1B, the same reference numerals correspond to each other.

  In FIG. 1A, 1 is a camera body, 2 is a lens unit attached to the camera body 1, 3 is a lens forming a photographing optical system built in the lens unit 2, 4 is an optical axis of the photographing optical system, Reference numeral 6 denotes an image sensor. Further, 9 is a rear display device, 11 is an electrical contact between the camera body 1 and the lens unit 2, 12 is a lens system controller provided in the lens unit 2, 14 is a quick return mirror, and 16 is provided in the camera body 1. It is a finder display section.

  FIG. 1B is a block diagram illustrating an electrical configuration of a digital camera and a lens which are imaging devices. A camera system including the camera body 1 and the lens 2 has an imaging system, an image processing system, a recording / reproducing system, and a control system. The imaging system includes a photographing optical system 3 and an imaging element 6, and the image processing system includes an image processing unit 7. The recording / reproducing system includes a memory unit 8 and a rear display device 9, and the control system includes a camera system control circuit 5, an operation detection unit 10, a lens system control circuit 12, and a lens driving unit 13. The lens driving unit 13 can drive a focus lens, a shake correction lens, a diaphragm, and the like. In the camera of this embodiment, a moving image is generated by acquiring a signal from the image sensor 6 at a constant period.

  The imaging system is an optical processing system that forms an image of light from an object on the imaging surface of the imaging device 6 via the imaging optical system 3. Microlenses are arranged in a lattice pattern on the surface of the image sensor 6 (on the image sensor) to form a so-called microlens array (hereinafter referred to as MLA). In this embodiment, the MLA constitutes a light beam control means. Details of the functions and arrangement of the MLA will be described later with reference to FIG. As will be described later, since the focus evaluation amount / appropriate exposure amount can be obtained from the imaging element 6, the imaging optical system 3 is adjusted based on this signal, so that an appropriate amount of object light is exposed to the imaging element 6. At the same time, a subject image is formed in the vicinity of the image sensor 6.

  The image processing unit 7 includes an A / D converter, a white balance circuit, a gamma correction circuit, an interpolation calculation circuit, and the like, and can generate a recording image. In addition, a developing unit, an image shift unit, an image generation unit, a contrast evaluation unit, a correlation calculation unit, and the like, which are main parts of the present embodiment, can be included. In the present embodiment, it is assumed that these elements are arranged in the camera system control circuit 5.

  The memory unit 8 includes a processing circuit necessary for recording in addition to an actual storage unit. The memory unit 8 outputs to the storage unit and generates and stores an image to be output to the rear display device 9. In addition, the memory unit 8 compresses images, moving images, sounds, and the like using a predetermined method.

  The camera system control circuit 5 generates and outputs a timing signal at the time of imaging. In addition, the imaging system, the image processing system, and the recording / reproducing system are controlled in response to external operations. For example, the operation detection unit 10 detects that a shutter release button (not shown) is pressed, and controls driving of the image sensor 6, operation of the image processing unit 7, compression processing of the memory unit 8, and the like. Further, the state of each segment of the rear display device 9 that displays information is controlled.

  The adjustment operation of the optical system by the control system will be described. An image processing unit 7 is connected to the camera system control circuit 5, and a focal position and a diaphragm position are obtained based on a signal from the image sensor 6. The camera system control circuit 5 issues a command to the lens system control circuit 12 via the electrical contact 11, and the lens system control circuit 12 controls the lens driving unit 13. Further, a camera shake detection sensor (not shown) is connected to the lens system control circuit 12, and in a mode for performing camera shake correction, the camera shake correction lens is controlled via the lens driving unit 13 based on a signal from the camera shake detection sensor.

  The camera system control circuit 5 includes image sensor control means, contrast evaluation means, and development surface control means, which are essential parts of the present embodiment. The developing means is provided in the image processing unit 7. Further, the lens control means includes a camera system control circuit 5, an electrical contact 11, a lens system control circuit 12, and a lens driving unit 13.

  FIG. 2 is a diagram for explaining a main part of the photographing optical system in the present embodiment. In order to implement this embodiment, it is necessary to acquire angle information in addition to the position of the light beam, which is so-called light space information. In the present embodiment, an MLA is arranged in the vicinity of the imaging plane of the photographing optical system 3 for obtaining angle information, and a plurality of pixels are associated with one lens constituting the MLA.

  FIG. 2A is a diagram schematically illustrating the relationship between the image sensor 6 and the MLA 20. FIG. 2B is a schematic diagram showing the correspondence between the pixels of the image sensor and the MLA. FIG. 2C is a diagram showing that pixels provided under the MLA by the MLA are associated with a specific pupil region.

  As shown in FIG. 2A, the MLA 20 is provided on the image sensor 6, and the front principal point of the MLA 20 is arranged in the vicinity of the imaging plane of the photographing optical system 3. FIG. 2A shows a state where the MLA is viewed from the side and from the front of the imaging device, and the lens of the MLA is disposed so as to cover the pixels on the imaging device 6 when viewed from the front of the imaging device. In FIG. 2A, the microlenses constituting the MLA are illustrated in large size so that the microlenses are easy to see, but each microlens is actually only about several times as large as a pixel. The actual size will be described with reference to FIG.

  FIG. 2B is a partially enlarged view from the front of the apparatus of FIG. A grid-like frame shown in FIG. 2B indicates each pixel of the image sensor 6. On the other hand, each microlens constituting the MLA is indicated by thick circles 20a, 20b, 20c, and 20d. As apparent from FIG. 2B, a plurality of pixels are assigned to one microlens. In the example of FIG. 2B, 5 rows × 5 columns = 25 pixels are one microlens. Is provided against. That is, the size of each microlens is 5 × 5 times the size of the pixel.

  FIG. 2C is a diagram in which the image pickup element 6 is cut so that the longitudinal direction of the sensor includes the optical axis of the microlens and the horizontal direction of the drawing. 2, 22, 23, 24, and 25 in FIG. 2C indicate pixels (one photoelectric conversion unit) of the image sensor 6. On the other hand, the figure shown above FIG. 2C shows the exit pupil plane of the photographing optical system 3. Actually, when the direction is aligned with the sensor diagram shown in the lower part of FIG. 2 (c), the exit pupil plane is in the direction perpendicular to the paper plane of FIG. 2 (c). It is changing. In FIG. 2C, one-dimensional projection / signal processing will be described for easy understanding. In an actual device, this can be easily extended to two dimensions.

  Pixels 21, 22, 23, 24, and 25 in FIG. 2C are in a positional relationship corresponding to 21a, 22a, 23a, 24a, and 25a in FIG. As shown in FIG. 2C, each pixel is designed to be conjugate with a specific area on the exit pupil plane of the photographing optical system 3 by the microlens 20. In the example of FIG. 2C, the pixel 21 and the region 31 correspond to the pixel 22 and the region 32, the pixel 23 and the region 33, the pixel 24 and the region 34, and the pixel 25 and the region 35, respectively. That is, only the light beam that has passed through the region 31 on the exit pupil plane of the photographing optical system 3 enters the pixel 21. The same applies to the other pixels. As a result, it is possible to obtain angle information from the passing area on the pupil plane and the positional relationship on the image sensor 6.

  Here, symbols are introduced to simplify the explanation later. As shown in FIG. 2C, it is assumed that the pixel pitch of the image sensor 6 is Δx and the angular resolution is Δθ. Further, the number of angle divisions is Nθ (Nθ = 5 in the example of FIG. 3). The pixel pitch is determined by the shape of the image sensor 6, and Δθ is determined by the range for obtaining the angle of the light beam and the angle division number Nθ. That is, these parameters are determined only by the physical structure (the structures of the image sensor 6 and the MLA 20).

  Processing for obtaining a recorded image in good focus from a signal from the image sensor 6 and obtaining a lens drive signal using the imaging optical system shown in the present embodiment will be described with reference to FIGS. 3, 4, 5, and 6. explain.

  FIG. 3 is a flowchart for explaining the imaging operation of the imaging apparatus of the present embodiment. 3A shows the overall operation of moving image shooting, FIG. 3B shows the operation of the contrast AF unit, and FIG. 3C shows the operation of the developing unit. 3 (d) shows the operation of the image shift means, FIG. 3 (e) shows the operation of the image generation means, FIG. 3 (f) shows the operation of the contrast evaluation means, and FIG. 3 (g) shows the development surface control. The operation of each means is shown.

  Moving image shooting will be described with reference to FIG. Step S1 corresponds to the start of moving image shooting, and corresponds to the case where the operation detection unit 10 in FIG.

  Steps S2 to S11 form a loop. In other words, a video recording end instruction from the user or a video recording end instruction from the device (the remaining battery level is less than the specified value, the remaining recording medium level is less than the specified value, time limit, shutdown due to heat generation, etc.) Until then, a so-called infinite loop is formed.

  Step S3 is image acquisition, and an image signal is acquired from the image sensor 6 at a set time interval (so-called frame rate) (the acquired image is referred to as a frame). The frame rate is set in advance by the user.

  In step S4, it is determined whether it is a focus key frame. The focus key frame here refers to a frame for performing focus detection. In a system for taking moving images, focus detection is often performed at an appropriate frame because of a heavy calculation load. For example, a focus key frame may be set every 15 frames in order to detect the focus in steps of 0.5 seconds at 30 frames / second. The focus key frame frequency at this time may be set depending on the moving speed of the subject. In general, a subject moving at a high speed may be detected finely, and a low speed may be detected roughly. The upper limit of the moving speed is defined from the mode setting etc. and is set from the relationship with the depth of field. If it is a focus key frame in step S4, the process proceeds to step S5. If it is not a focus key frame, the process proceeds to step S7.

  In step S5, the contrast AF routine is called to obtain information on the development surface from which a recorded image with good focus can be obtained. The contrast AF routine will be described later with reference to FIG.

  In step S6, a development surface control routine is called to determine a development surface from which a recorded image with good focus can be obtained in the current frame. The developing surface control routine will be described later with reference to FIG.

  In step S7, a development surface on which a recorded image with good focus is obtained in the current frame is determined based on the information on the development surface on which a recorded image with good focus on the previous focus key frame is obtained and the state of the lens control means. To do. In step S8, an image is generated on the development surface determined in steps S6 and S7.

In step S9, the lens control routine is operated. In moving image shooting, a sharp change in the focus state deteriorates the quality. For this reason, it is desirable to adjust the focus by slowly driving the focus lens when there is a slight focus shift. Step S9 is a block for controlling this. Specifically, an operation of filling a difference between the development surface obtained in step S5 in which a recording signal with a good focus state is obtained and the surface on which the current image sensor is present at a preset speed is performed. Details of the operation will be described later with reference to FIG. 6. In step S10, buffering / compression / recording of an image is performed. The frames obtained in succession are compressed by a predetermined encoding method while being appropriately buffered, and recorded in the memory unit 8.

  Step S12 corresponds to the end of moving image shooting. In FIG. 3A, the compression / recording and the like are finished at the same time as the image acquisition is finished, but this is for easy understanding, and post-processing (Exif setting etc.) is performed as necessary. Also good.

  The contrast AF routine will be described with reference to FIG. Step S13 shows the start of the contrast AF routine. Steps S14 to S17 form a loop, and the image is evaluated by setting the development surface at a predetermined evaluation position. In the example of FIG. 5, this corresponds to setting focus positions of −2, −1, 0, +1, and +2.

  In step S15, a development routine is called to generate an image on the set development surface. In step S16, a contrast evaluation routine is called, and a focus evaluation value is obtained from the contrast value of the image obtained in step S15. Details of the contrast evaluation routine will be described later with reference to FIG.

  In step S18, the development surface with the best focus evaluation value (development surface with the highest contrast) is selected from the focus evaluation values of the development surfaces, and the developed surface is output as a development surface from which a recorded image with good focus can be obtained. In step S19, the process returns to the caller step S5.

  The developing routine will be described with reference to FIG. Step S26 indicates the start of the development routine.

  In step S27, an image shift routine is called, and an amount suitable for the development surface is shifted based on pupil information (incident angle information) that has passed through the acquired image. In the caller step S8, the surface set in steps S6 and S7, and in the caller step S15, it is necessary to generate an image on the focus evaluation surface set in the loop from step S14 to step S17. This is called the development surface. Details of the image shift routine will be described later with reference to FIG.

  In step S28, an image generation routine is called and the image shifted in step S27 is superimposed (added pixel signals) to obtain an image on the development surface. Thereby, so-called development is performed. Details of the image generation routine will be described later with reference to FIG. In step S29, the process returns to step S8 or step S15 of the caller.

  In this embodiment, the method of performing development by image shift and addition has been exemplified, but any method can be used as long as it uses a ray information to generate an image on a surface different from the surface from which the image was acquired. But it can be used as a development routine.

  The image shift routine will be described with reference to FIG. Step S31 indicates the start of the operation of the image shift routine. Steps S32 to S35 form a loop. In step S32, the loop calculation is executed by the number corresponding to the number of pupil divisions. For example, in the example shown in FIG. 2, since it is divided into 5 × 5 = 25, calculation according to each of the 25 pupil positions is performed. As will be described later with reference to FIG. 5, when image reconstruction is considered, even if the reconstruction plane is the same, the amount of image shift is different when the incident angle is different. This is a loop for reflecting this.

  In step S33, the image shift amount in each pupil region on the developing surface being evaluated is calculated based on the information on each pixel and the corresponding pupil region. As described with reference to FIG. 2, the correspondence between the microlens array 20 operating as pupil dividing means and the pixels is fixed and does not change depending on the photographing state. The correspondence between each pixel on the image pickup device 6 and the MLA is stored, and it is known which pupil region receives the light beam of each pixel.

  In step S34, the signal of the pixel that has received the light beam that has passed through the same pupil region (the light beam having the same incident angle) is shifted. For example, pixels 25a and 25b in FIG. 2B correspond to pixels that obtain light rays having the same incident angle. There are as many such pixels as the number of microlenses constituting the MLA. The image shift routine is also illustrated later using FIG. In step S36, the process returns to the caller step S27.

  Details of the operation of the image generation routine will be described with reference to FIG. Step S41 indicates the start of the operation of the image generation routine.

In step S42, the area data for addition in step S45 is initialized (filled with 0). The size of the data area at this time may be as many as the number of MLA, and it is convenient if the data gradation can store the product of the original data gradation and the number of pupil divisions. For example, if the original data is 8 bits and divided into 25, if 13 bits (> 8 bits + log ₂ 25), it is not necessary to consider overflow of data.

  Steps S43 to S47 form a loop. In step S43, loop calculation is executed according to the number of microlenses constituting the MLA. For example, in the example illustrated in FIG. 4, the number of microlens is the number of pixels of the original image sensor ÷ 25 (number of pupil divisions).

  Steps S44 to S46 form a loop. In step S44, loop calculation is executed by the number corresponding to the number of pupil divisions. For example, in the example shown in FIG. 4, since it is divided into 5 × 5 = 25, the luminous fluxes from the 25 pupil positions are added in step S45. If the shift amount is not an integer multiple of the pixel, the addition is performed while being divided internally in addition step S45. What is necessary is just to add according to the overlapping area. The image generation routine is also illustrated later using FIG. In step S48, the process returns to the caller step S28.

  Details of the operation of the contrast evaluation routine will be described with reference to FIG. Step S51 indicates the start of the contrast evaluation routine.

  In step S52, the size of an evaluation frame for performing contrast evaluation is set. If the size of the evaluation frame is too small, only the local characteristics of the subject are evaluated, and the evaluation value may become unstable. If it is too large, it is difficult to focus on the target subject. What is necessary is just to set according to the focal distance of an imaging optical system.

  Steps S53 to S57 form a loop. This loop is a loop corresponding to the size of the evaluation frame. In this embodiment, contrast evaluation is performed in one direction, and the other directions are added.

  Step S54 is a band pass filter (BPF), which selectively extracts signals in a frequency band suitable for performing contrast evaluation. The frequency here corresponds to a so-called spatial frequency, and corresponds to the fineness of the brightness signal of the subject image. This is different from the general frequency in the time direction.

  In step S55, peak hold is performed on the output of step S54. In step S56, the output of step S55 is integrated. By calculating from step S54 to step S56, it is possible to obtain a rough contrast value (good approximate value) of a signal in a specific frequency band included in the subject image by a simple calculation. In step S58, the integral value is set as the contrast evaluation value. In step S59, the process returns to the caller step S16.

  The developing surface control routine will be described with reference to FIG. Step S61 indicates the start of the development surface control routine.

  In step S62, the following equation (1) is calculated.

z = z0 + f (zAF-zL) (1)
Here, f is an operator representing mapping onto the developing surface in the developing routine, and z0 is a developing surface corresponding to the aerial imaging surface of the photographing optical system 3. zAF is the distance from the development surface corresponding to the aerial imaging surface obtained with the focus key frame to the development surface where a recorded image with good focus is obtained, and zL is the focus adjustment amount from the focus key frame.

  Although details will be described later with reference to FIG. 6, it is possible to select a development surface on which a recorded image with a good focus state can be obtained by Expression (1). Further, f is a mapping to the development surface in the development routine, but if the development routine can be developed on a continuous development surface, the value is returned as it is, and if development is possible on a discontinuous development surface, the closest development surface is returned. A mapping that returns the value of. In this embodiment, in order to make the explanation easy to understand, FIGS. 4, 5 and 6 are described so that development is performed on a discontinuous development surface. However, the present embodiment is not limited even if development is possible on other surfaces. There is no effect on the part. In step S63, z obtained in step S62 is set as the development surface. In step S64, the process returns to the caller step S6 or step S7.

  Next, image shift and image generation are schematically shown in FIG. 4, and the usefulness of contrast calculation by image reconstruction will be described. FIG. 4 is aligned with (a), (b), and (c) from above. 4B shows a surface on which the image sensor 6 is present and an image is acquired, and FIG. 4A shows a reconstruction surface (reconstruction surface 1) closer to the object than FIG. 4B. 4C shows a reconstruction surface (referred to as reconstruction surface 2) on the side farther from the object than FIG. 4B.

  In FIG. 4B, X1, i, X2, i, X3, i, X4, i, X5, i pass through pupil regions 1, 2, 3, 4, 5 and enter the microlens Xi. The obtained data is shown. That is, of the subscripts, the first half indicates the passing pupil region, and the second half indicates the pixel number. Also, in FIG. 4, data is described as having only a one-dimensional spread for the sake of simplicity. In relation to the physical position, X1, i represents the data obtained from the area 21 in FIG. 2C, X2, i represents the data obtained from the area 22 in FIG. 3, 4, and 5 indicate that they correspond to the regions 23, 24, and 25, respectively.

  In order to generate an image on the acquisition surface, data incident on the microlens Xi may be added as shown in FIG. Specifically, the integrated value in the angular direction of the light incident on Xi can be obtained with Si = X1, I + X2, i + X3, i + X4, i + X5, i. As a result, an image similar to that of a normal camera is generated.

  Next, a method for generating an image on the reconstruction plane 1 will be considered. As described with reference to FIG. 2, the imaging optical system of the present embodiment limits the luminous flux incident on each pixel to a specific pupil region, and thus the incident angle is known. The position of each pixel on the reconstruction plane is reconstructed along this angle. Specifically, it is assumed that a pupil region with a subscript of 1 such as X1, i is incident at an angle as shown at 41 on the right side of FIG. Hereinafter, it is assumed that the subscripts 2, 3, 4, and 5 of the pupil region correspond to 42, 43, 44, and 45, respectively. At this time, the light beam incident on the microlens Xi on the reconstruction surface 1 is scattered and incident on the acquisition surface from Xi-2 to Xi + 2. More specifically, it is distributed in X1, i-2, X2, i-1, X3, i, X4, i + 1, X5, i + 2. In order to restore the image on the reconstruction plane 1 as well as Xi, it is understood that the image may be shifted and added according to the incident angle. In order to generate an image on the reconstruction plane 1, a pupil region with a subscript of 1 shifts 2 pixels to the right, a pupil region with a subscript of 2 shifts to the right with a 1 pixel shift, and a pupil region has a subscript. Shifting according to the incident angle by shifting 3 characters without shift, shifting pupil pixels with 4 subscripts by 1 pixel to the left, and shifting pupil regions with 5 subscripts by 2 pixels to the left Can be given.

  Thereafter, the data in the reconstruction plane 1 can be obtained by adding in the vertical direction of FIG. Specifically, the integrated value in the angular direction of the light incident on Xi can be obtained on the reconstruction plane 1 with Si = X1, i-2 + X2, i-1 + X3, i + X4, i + 1 + X5, i + 2. As a result, an image on the reconstruction surface was obtained.

  Here, if Xi has a bright spot on the reconstruction plane 1, X1, i-2, X2, i-1, X3, i, X4, i + 1, X5, i + 2 on the acquisition plane. It is dispersed and in a so-called blurred state. However, when the image on the reconstruction plane 1 described above is generated, a bright spot is generated again in Xi, and an image with high contrast is obtained. That is, so-called contrast AF can be performed by reconstructing the image and calculating the contrast.

  Further, as can be seen from FIG. 4C, an image can be generated on the reconstruction plane 2 in the same manner as the reconstruction plane 1. If the direction in which the reconstruction plane is arranged is different (meaning opposite to the object), it is only necessary to reverse the direction of shifting.

  As described with reference to FIG. 3 and FIG. 4, a development surface is obtained in which a recorded image with good focus can be obtained by reconstructing the image and performing contrast evaluation without reading the image only once. Is possible.

  The operation in the focus key frame will be described with reference to FIG. The upper part of FIG. 5 indicates that 53 of the temporally continuous frames 51, 52, 53, 54, and 55 are taken as the focus key frames with the horizontal axis taken as time. Here, the focus key frame refers to a frame for performing focus detection as described in FIG.

  In the center of FIG. 5, 53a, 53b, 53c, 53d, and 53e indicate images reconstructed from the focus key frame 53, respectively. The focus key frame 53 is extracted (the frame 53 means Yes in step S4 of FIG. 3A and proceeds to step S5), and development is performed by changing the development surface to generate a plurality of images. The operation is shown. This corresponds to the operation of obtaining the image developed in step S15 by changing the development surface in loops S14 to S17 shown in FIG.

  The lower part of FIG. 5 is a graph in which the horizontal axis represents the development surface (focus position, changing the reconstructed development surface as shown in FIG. 4 is equivalent to changing the focus position), and the vertical axis represents the contrast evaluation value. Yes, the evaluation values obtained in step S16 shown in FIG. 5, 63a, 63b, 63c, 63d, and 63e are the contrast evaluation values of the reconstructed images 53a, 53b, 53c, 53d, and 53e, 64 is an approximate curve that connects the contrast evaluation values, and 65 is an approximate curve 64. Each extreme value (peak) is shown. A broken line extending from the center of FIG. 5 toward the lower stage of FIG. 5 is a line indicating that the image on the same development surface corresponds to the evaluation value.

  The example of FIG. 5 shows an example in which the contrast evaluation value is obtained by changing five focus states. The number of 5 is an example, and any number may be used as long as it is plural. However, calculation is generally performed at three or more points for focus detection based on contrast. The focus state is shown as -2, -1, 0, +1, +2 (corresponding to the reconstructed images 53a, 53b, 53c, 53d, and 53e, respectively), but this is different in focus state (development surface). It is an indicator that indicates that it has no physical meaning. That is, it does not indicate the shift amount corresponding to the development surface, but corresponds to the loop counter of loops S14 to S17 in FIG. However, this indicates that the focus states are different in the order of the numbers. In the example of FIG. 4, the reconstruction plane 1, the acquisition plane, and the reconstruction plane 2 are made to correspond to, for example, −1, 0, and +1 in this order. A value of 0 is set to the current focus position of the photographic optical system (position where the aerial imaging surface in the design of the photographic optical system is set as the development surface).

  In the example of FIG. 5, it is assumed that there is a position where the contrast is maximum at a position closer to the focus: +1 than the position of the focus: 0. This corresponds to, for example, a case where the subject approaches (or moves away) toward the camera, and indicates that the focus position is slightly shifted at the moment of acquiring the focus key frame. At this time, the contrast AF routine in FIG. 3B outputs the position 65 in FIG. 5 as the developing surface having the best focus evaluation value. This is the zAF in equation (1). Further, since the development surface having the best focus evaluation value in this frame is recalculated, zL in the equation (1) is reset to zero.

  As a result, in step S6 (development surface control routine immediately after focus detection in the focus key frame) in FIG. 3A, zAF is processed as the value obtained in step S5, zL = 0, and as a result, FIG. 65 is designated as the development surface.

  Next, the operation from the focus key frame to the next focus key frame will be described with reference to FIG. The upper part of FIG. 6 is the same as FIG. 5 and shows that 53 of the frames 52, 53, 54, 55, 56, 57, 58 that are temporally continuous with time on the horizontal axis are focus key frames. . The lower part of FIG. 6 shows the development surface (focus position) in the vertical direction, the contrast evaluation value at the focus key frame position in the horizontal direction, and the time at other positions.

  5 and FIG. 6 correspond to each other, and the development surface having the best focus evaluation value in the focus key frame 53 is indicated by 65 between 63c and 63d. Reference numeral 81 denotes a focus shift amount in the focus key frame 53, and 82, 83, 84, 85, and 86 denote focus adjustment amounts from the focus key frame 53 to the frames 54, 55, 56, 57, and 58, respectively. 82a is the remaining amount of focus adjustment in the frame 54.

  In FIG. 6, the broken lines extending vertically from the frames 53, 54, 55, 56, 57, and 58 are lines indicating the same time. Further, the solid line extending from the bottom of the focus-2, -1, 0, +1, +2 to the right indicates the position of the developing surface. The fact that the solid line inclines from the focus key frame 53 to the frame 58 indicates that the focus adjustment is performed by the lens control routine as will be described later, and the position of the developing surface in the image processing has changed.

  At this time, the lens control routine operates in a direction to correct the so-called out-of-focus state. However, as described above, since it is desired to adjust the focus slowly by driving the focus lens, as described above, the focus shift 81 is not corrected immediately, but it is corrected over time (over several frames). To do. FIG. 6 schematically shows this state.

  Since the focus shift 81 is detected by the focus key frame 53, the lens control routine can immediately determine the direction and amount to move the lens. The lens driver 13 is used to adjust the focus adjustment speed below a predetermined speed, and the focus adjustment is performed so that the focus shift is reduced. Obviously, a signal for driving the lens can be obtained by performing calculation for obtaining a recorded image in good focus (calculation for obtaining position 65).

  In the example of FIG. 6, the amount of movement from the focus key frame 53 to the frame 54 is 82, the amount of movement from the focus key frame 53 to the frame 55 is 83, and so on. The amount measured is shown at 86. That is, the focus adjustment amounts 82, 83, 84, 85, 86 correspond to zL in the equation (1). Of course, zL is a value that changes from frame to frame. In the example of FIG. 6, 86 and 81 are the same amount, indicating that the focus adjustment is completed at the frame 58. The lens control routine performs the focus adjustment as in the example of FIG. 6 and stops the adjustment of the photographing optical system 3 when the adjustment is completed.

  A description will be given using the frame 54 as an example. By this frame, the focus adjustment corresponding to 82 has been completed. Considering the equation (1) at this time, the focus adjustment remaining amount 82a in FIG. 6 is calculated by (zAF−zL). That is, since the focus adjustment by the optical system has advanced by the amount corresponding to 82 from the focus key frame 53, it is indicated that the focus adjustment by image reconstruction may be performed so as to correspond to the remaining focus adjustment amount 82a. .

  Similar calculations are repeated in the following frames. After the frame 58, (zAF−zL) = 0, and the image reconstruction may be performed using the aerial imaging surface of the photographing optical system 3 as a developing surface.

  According to the present embodiment, it is possible to provide an imaging apparatus that can obtain a recorded image with a good focus state during movie shooting and obtain a lens driving signal.

  Note that the main part of the present invention can be applied to an imaging apparatus that can adjust the focus by reconstructing an image, and can be used even when a parallax image is obtained in another form such as a multi-camera (light space information is obtained). The invention can be applied.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

Claims (5)

An image sensor for obtaining a signal including light space information ;
Image sensor control means for outputting a signal from the image sensor at a constant period;
A developing unit that sets the development surface for the obtained signal from the previous SL imaging device generates an image signal in the developing surface by performing an addition of the signals respectively corresponding to the developing surface,
A calculation unit that calculates a development surface with the best focus state using image signals on a plurality of development surfaces obtained from the development unit;
Based on the calculation result by the calculating means, at a driving speed below the speed that has been set in advance, and a lens control means for controlling the photographic lens in a direction in which the focus state becomes better,
Based on the calculation result by the calculating means, and determining means for focus state to determine the best development plane as the development face of the recorded image,
An imaging apparatus comprising: The imaging apparatus according to claim 1, wherein the developing unit generates an image of the developing surface determined by the determining unit as a recorded image. The imaging apparatus according to claim 1, wherein the determination unit further determines a development surface by the development unit for the recorded image based on a state of the lens control unit. 4. The calculation unit according to claim 1, wherein the calculation unit calculates a development surface with the best focus state by comparing contrasts of image signals on a plurality of development surfaces obtained from the development unit. The imaging apparatus according to item 1. A method of controlling an imaging device including an imaging device that obtains a signal including light space information ,
An image sensor control step of outputting a signal from the image sensor at a constant period;
A developing step of setting the developer surface for the obtained signal from the previous SL imaging device generates an image signal in the developing surface by performing an addition of the signals respectively corresponding to the developing surface,
A calculation step of calculating a development surface with the best focus state using image signals on a plurality of development surfaces obtained in the development step;
Based on the calculation result in the calculation step, a lens control step for controlling the taking lens at a speed equal to or lower than a preset driving speed and in a direction in which the focus state is improved ,
Based on the calculation result in the calculation step, a determining step of focusing state to determine the best development plane as the development face of the recorded image,
Control method for an imaging apparatus characterized by having a.

Images