JP6053329B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an image pickup apparatus and an image pickup apparatus control system for picking up, recording, and reproducing still images and moving images, and more particularly to an image pickup apparatus having a microlens array on the front surface of an image pickup element that is a component of the image pickup apparatus.

  2. Description of the Related Art Conventionally, there are many imaging devices such as electronic cameras that record and reproduce still images and moving images captured by a solid-state imaging device such as a CCD or CMOS using a memory card having a solid-state memory device as a recording medium.

  As an example of the technology related to these image pickup devices, by arranging a microlens array arranged at a ratio of one for a plurality of pixels on the front surface of a solid-state image pickup device, information on the incident direction of light incident on the image pickup device can be obtained An imaging apparatus having a simple structure has been proposed. (For example, refer nonpatent literature 1).

  In such an imaging apparatus, in addition to generating a normal captured image based on an output signal from each pixel, the captured image is focused on an arbitrary focal length by performing predetermined image processing. It is also possible to reconstruct the reconstructed image (refocus).

Ren.Ng and 7 others, “Light Field Photography with a Hand-Held Plenoptic Camera”, Stanford Tech Report CTSR 2005-02

  However, shooting in the imaging apparatus having the microlens array as described above has the following problems.

  The pixel unit in the image generated by the imaging device is one pixel per microlens. Since a normal image sensor has a configuration of one pixel per microlens, it is only necessary to read out pixel signals of approximately the same number as the number of image data to be generated. On the other hand, since the imaging device of Non-Patent Document 1 described above has a configuration having several tens of pixels per microlens, it must handle a pixel signal several tens of times that of a normal imaging device. For this reason, the file size of the image data is increased, and a very large memory area is required in the imaging apparatus as compared with an imaging apparatus such as a normal camera, and the recording medium to be stored has a larger capacity than usual. There was a need. In addition, it takes an enormous amount of time to read one piece of image data, and for example, it has been difficult to increase the frame speed during continuous shooting.

  The present invention has been made in view of the above-described problems. In an imaging apparatus having a microlens array in which one microlens is arranged for each of a plurality of photoelectric conversion units, the file size of a recorded image is suppressed and read out. The purpose is to improve speed.

In order to achieve the above object, an imaging apparatus of the present invention includes a plurality of two-dimensionally arranged photoelectric conversion units that photoelectrically convert light incident through a photographing lens and output an electrical signal, and an optical axis. A plurality of unit regions that are arranged between the photographing lens and the plurality of photoelectric conversion units, and each of the plurality of unit regions obtained by dividing the plurality of photoelectric conversion units by a predetermined number is associated with each other. There is a possibility of using it for refocusing processing for generating an image focused at an arbitrary distance for each divided region obtained by dividing an image based on an electrical signal obtained from the imaging unit into a plurality of regions. determining means for determining whether the divided region among the plurality of divided regions, the unit region included in the divided region is determined that there is no possibility of using the refocus processing by the determination unit An electric signal output from the photoelectric conversion unit to be formed is added for each unit area, and among the plurality of divided areas, the electric signal of the divided area added by the adding means is recorded, and Recording means for recording each of the divided areas determined to be possibly used for the refocus processing by the determination means without being added.

  According to the present invention, in an imaging device having a microlens array in which one microlens is arranged for each of a plurality of photoelectric conversion units, the file size of a recorded image can be suppressed and the reading speed can be improved.

1 is a block diagram showing a schematic configuration of an imaging apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a configuration around an optical system of the imaging apparatus according to the embodiment. The figure which shows the positional relationship of the pixel arrangement | sequence of the image pick-up element in embodiment, and a micro lens array. FIG. 6 is a diagram for explaining a correspondence relationship between a pupil region and divided pixels of the photographing lens according to the embodiment. FIG. 6 is a ray trajectory diagram from a subject in the imaging apparatus of the embodiment. It is a figure explaining the object distance detection operation | movement in embodiment of the imaging device concerning this invention. It is a figure explaining the image file production | generation method according to the picked-up image in embodiment of the imaging device concerning this invention.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings. However, the dimensions, shapes, relative arrangements, and the like of the components exemplified in the present embodiment should be changed as appropriate according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to.

  FIG. 1 is a block diagram illustrating a schematic configuration of the imaging apparatus according to the first embodiment. In FIG. 1, reference numeral 101 denotes an optical system such as a photographing lens and a diaphragm, and 102 denotes a mechanical shutter. The image sensor 103 includes a two-dimensionally arranged photoelectric conversion unit 104 that converts incident light into an electric signal and an amplification circuit 105 that amplifies the electric signal, and converts the incident light into an electric signal. The analog signal processing circuit 106 performs analog signal processing on the image signal output from the image sensor 103. The analog signal processing circuit 106 includes a CDS circuit 107 that performs correlated double sampling, a signal amplifier 108 that amplifies the analog signal, a clamp circuit 109 that performs horizontal OB clamping, and an A / D converter 110 that converts the analog signal into a digital signal. Including.

  The timing signal generation circuit 111 generates a signal for operating the image sensor 103 and the analog signal processing circuit 106. The drive circuit 112 drives the optical system 101 and the mechanical shutter 102. The digital signal processing circuit 113 includes an image correction circuit 114 that performs necessary correction processing on the image data, a signal amplification circuit 115 that amplifies the digital signal corrected by the image correction circuit 114, and a necessary image for the image data. An image processing circuit 116 that performs processing is included. With these configurations, the digital signal processing circuit 113 performs digital signal processing necessary for captured image data.

  117 is an image memory for storing processed image data, 118 is a recording medium removable from the imaging apparatus, and 119 is a recording circuit for recording signal processed image data on the recording medium 118. Reference numeral 120 denotes an image display device that displays signal-processed image data, and 121 denotes a display circuit that displays an image on the image display device 120.

  The system control unit 122 controls the entire imaging apparatus. The non-volatile memory (ROM) 123 stores a program describing a control method executed by the system control unit 122, control data such as parameters and tables used when executing the program, and correction data such as a scratch address. To do. A volatile memory (RAM) 124 transfers and stores the program, control data, and correction data stored in the nonvolatile memory 123, and is used when the system control unit 122 controls the imaging apparatus. In addition, the shooting mode setting unit 125 sets shooting conditions such as ISO sensitivity setting, and switches between still image shooting and live view driving.

  Here, a photographing operation in the imaging apparatus having the above configuration will be described. Prior to the shooting operation, necessary programs, control data, and correction data are transferred from the nonvolatile memory 123 to the volatile memory 124 and stored at the start of the operation of the system control unit 122 such as when the imaging apparatus is turned on. Keep it. These programs and data are used when the system control unit 122 controls the imaging apparatus. Further, as necessary, additional programs and data are transferred from the nonvolatile memory 123 to the volatile memory 124, or the system control unit 122 directly reads and uses the data in the nonvolatile memory 123. .

  First, the optical system 101 drives the optical system 101 such as a lens in accordance with a control signal from the system control unit 122 to form a subject image controlled to have appropriate brightness on the image sensor 103. . Next, during still image shooting, the mechanical shutter 102 is driven by the control signal from the system control unit 122 so as to shield the image sensor 103 in accordance with the operation of the image sensor 103 so that a necessary exposure time is obtained. Is done. At this time, when the image sensor 103 has an electronic shutter function, it may be used together with the mechanical shutter 102 to secure a necessary exposure time. The mechanical shutter 102 is maintained in an open state so that the image sensor 103 is always exposed during shooting by a control signal from the system control unit 122 during moving image shooting and live view driving.

  The image sensor 103 is driven with a drive pulse based on an operation pulse generated by the timing signal generation circuit 111 controlled by the system control unit 122. The photoelectric conversion unit 104 converts the subject image into an electrical signal by photoelectric conversion, and the amplification circuit 105 multiplies the gain of the amplification factor set according to the amount of incident light by the electrical signal, and outputs it as an analog image signal.

  From the analog image signal output from the image sensor 103, the clock synchronization noise is removed by the CDS circuit 107 of the analog signal processing circuit 106 by the operation pulse generated by the timing signal generation circuit 111 controlled by the system control unit 122. Further, the gain of the amplification factor set in accordance with the amount of incident light is applied by the signal amplifier 108, the signal output in the horizontal OB region is clamped as a reference voltage by the clamp circuit 109, and converted into a digital image signal by the A / D converter 110. Is done.

  Next, the digital image signal output from the analog signal processing circuit 106 is processed in the digital signal processing circuit 113 controlled by the system control unit 122. First, the image signal converted into the digital signal is subjected to various image correction processes such as defect correction and dark shading correction by the image correction circuit 114.

  Thereafter, the signal amplification circuit 115 multiplies the gain set in accordance with the amount of incident light, and the image processing circuit 116 performs various image processing such as color conversion, white balance, and gamma correction, resolution conversion processing, and image compression processing. Perform image processing. Further, in this image processing circuit 116, for example, using the method disclosed in Non-Patent Document 1 described above, refocus processing (based on an electrical signal output from the image sensor 103, an arbitrary distance can be adjusted. A process of generating a plurality of focused images). Further, the image processing circuit 116 also performs pixel signal addition processing and addition averaging processing, which is a feature of the present invention, performed before the refocus processing. At this time, the image memory 117 is used for temporarily storing a digital image signal during signal processing or for storing image data that is a digital image signal subjected to signal processing.

  The image data signal-processed by the digital signal processing circuit 113 and the image data stored in the image memory 117 are converted into data suitable for the recording medium 118 (for example, file system data having a hierarchical structure) by the recording circuit 119. Recorded on the recording medium 118. Alternatively, the image data subjected to resolution conversion processing by the digital signal processing circuit 113 is converted into a signal suitable for the image display device 120 (for example, an NTSC analog signal) by the display circuit 121 and displayed on the image display device 120. Or

  Here, the digital signal processing circuit 113 may output the digital image signal as it is as image data to the image memory 117 or the recording circuit 119 without performing signal processing by the control signal from the system control unit 122. Further, the digital signal processing circuit 113 outputs information on digital image signals and image data generated in the signal processing process to the system control unit 122 when requested by the system control unit 122. The image data information includes, for example, information such as the spatial frequency of the image, the average value of the designated area, the data amount of the compressed image, or information extracted from them. Further, the recording circuit 119 outputs information such as the type and free capacity of the recording medium 118 to the system control unit 122 when requested by the system control unit 122.

  Next, the reproduction operation when image data is recorded on the recording medium 118 will be described. In response to a control signal from the system control unit 122, the recording circuit 119 reads image data from the recording medium 118. Similarly, in response to a control signal from the system control unit 122, the digital signal processing circuit 113 performs an image expansion process when the read image data is a compressed image, and stores it in the image memory 117. Further, the image data stored in the image memory 117 is subjected to resolution conversion processing by the digital signal processing circuit 113, converted into a signal suitable for the image display device 120 by the display circuit 121, and displayed on the image display device 120. Is done.

  FIG. 2 is a block diagram illustrating the configuration around the optical system in the embodiment of the imaging apparatus according to the present invention.

  In FIG. 2, the taking lens 201 constitutes the optical system 101. The microlens array 202 and the sensor array 203 are components of the image sensor 103. Reference numeral 204 denotes a subject. Since the other components described in FIG. 2 are the same as those described in FIG. 1, description thereof is omitted here.

  In a state where the mechanical shutter 102 is opened by the drive circuit 112, an image of the subject 204 is formed on the image sensor 103 by the photographing lens 201. The optical signal incident on the image sensor 103 is further condensed by each microlens of the microlens array 202 and enters each pixel of the sensor array 203. The configurations of the microlens array 202 and the sensor array 203 will be described later with reference to FIG. The optical signal incident on the sensor array 203 is photoelectrically converted in each pixel and output as an electrical signal. Subsequent processing is as described with reference to FIG.

  FIG. 3 is a diagram illustrating the relative arrangement of the pixels of the sensor array 203 constituting the image sensor 103 and the microlens array 202 in this embodiment, as viewed from the subject side. In FIG. 3, reference numeral 301 denotes a recording pixel area (unit area) that is a unit pixel corresponding to one pixel of the reconstructed image, and 302 denotes a photoelectric conversion unit that divides the recording pixel area 301 into 6 rows and 6 columns. This is a divided pixel. Reference numeral 303 denotes a microlens arranged for each recording pixel region 301.

  In the description of the present embodiment, particularly with reference to FIGS. 2, 3, and 5, as shown in FIG. 3, as shown in FIG. 3, a sensor array 203 in which recording pixel regions 301 including 6 × 6 divided pixels 302 are arranged in 5 rows and 5 columns. It explains using. However, in an actual imaging device, millions to tens of millions of recording pixel areas 301 are arranged.

  FIG. 4 is a diagram for explaining the correspondence between the pupil region and the divided pixels of the photographing lens according to the present embodiment, and FIG. 4A is an enlarged view of the recording pixel region 301. For later explanation, in the present embodiment, 36 divided pixels 302 are labeled p11 to p66, respectively, as shown in FIG. FIG. 4B is a view of the aperture of the photographic lens 201 as viewed from the direction of the photographic subject. As shown in FIG. 4B, when the pupil region of the photographing lens 201 is divided into the same number of regions as the pixels under one micro lens 303, one divided pixel 302 has one pupil divided region of the photographing lens 201. The light from is imaged. However, here, it is assumed that the F-numbers of the photographing lens 201 and the microlens 303 are substantially the same.

  When the pupil division regions of the photographing lens 201 shown in FIG. 4B are a11 to a66, the correspondence relationship between the pupil division regions a11 to a66 of the photographing lens 201 and the divided pixels p11 to p66 shown in FIG. Is point-symmetric when viewed from the optical axis direction. Therefore, the light emitted from the pupil division area a11 of the photographing lens 201 forms an image on the divided pixel p11 among the divided pixels 302 behind the microlens 303. Similarly, the light emitted from the pupil division area a11 and passing through another microlens 303 forms an image on the divided pixel p11 in the divided pixel 302 behind the microlens.

  FIG. 5 is a diagram illustrating the trajectory of light rays incident on the imaging apparatus according to the present embodiment from subjects at various distances. In FIG. 5, the light emitted from each of the pupil regions a1 to a6 of the photographing lens 201 and passing through the microlens array 202 forms an image on the corresponding divided pixels p1 to p6, respectively.

  The subject 601 a is a subject placed at a position where the image is formed on the surface A including the microlens array 202 by the photographing lens 201. Among the light rays from the subject 601a, the light rays that pass through the outermost periphery of the photographing lens 201 and enter the sensor array 203 via the microlenses on the optical axis are indicated by solid lines.

  The subject 601b is a subject located farther from the subject 601a when viewed from the photographing lens 201. The image of the subject 601 b formed by the photographing lens 201 is formed on a surface B closer to the photographing lens 201 than the surface A including the microlens array 202. Of the light rays from the subject 601b, the light rays that pass through the outermost periphery of the photographing lens 201 and enter the sensor array 203 via the microlenses on the optical axis are indicated by broken lines.

  The subject 601c is a subject closer to the subject 601a when viewed from the photographing lens 201. The image of the subject 601 c formed by the photographing lens 201 is formed on a surface C farther from the photographing lens 201 than the surface A including the microlens array 202. Of the light rays from the subject 601c, the light rays that pass through the outermost periphery of the photographing lens 201 and enter the sensor array 203 via the microlenses on the optical axis are indicated by a one-dot chain line.

  As shown by the trajectory of each light ray shown in FIG. 5, the divided pixels incident in the sensor array 203 are different depending on the distance from the photographing lens 201 to the subject 601. By using this, the imaging apparatus of this configuration can generate images focused on subjects at various distances by reconstructing the image signal after shooting.

  Incidentally, the pixels p11 to p66 shown in FIG. 4A receive light that has passed through different pupil regions with respect to the photographic lens 201 as described with reference to FIG. Using this information, an operation for detecting the distance to the subject can be performed.

  FIG. 6 is a diagram for explaining the subject distance detection operation in the present embodiment. As shown in FIG. 6A, the outputs of the pixels p11 to p66 corresponding to each microlens 303 are added, and the two combined signals A and B (first output) divided in the horizontal direction as shown in the following equation. , A second synthesized signal).

The composite signal A represented by the expression (1) has passed through the right half pupil division regions a11 to a61, a12 to a62, and a13 to a63 (first pupil region) of the photographing lens 201 shown in FIG. 4B. This is the total of signals obtained by receiving the light flux. As shown in FIG. 6B, A1 to An obtained from the recording pixel regions 301 corresponding to the n microlenses 303 arranged continuously in the horizontal direction are arranged, and the one-dimensional image signal Ai (i = 1, 2, 3... N).
Similarly, the composite signals B1 to Bn calculated by Expression (2) are arranged for the pupil division areas a14 to a64, a15 to a65, and a16 to a66 (second pupil areas) in the left half of the photographing lens 201. A one-dimensional image signal Bi (i = 1, 2, 3,... N) is generated. Then, Ai and Bi are signals for viewing the right half and the left half of the exit pupil of the photographing lens 201, respectively. Therefore, the distance to the subject based on the phase difference detection method can be detected by detecting the relative position between Ai and Bi and multiplying the relative shift amount by a predetermined conversion coefficient.

  In addition, if Ai and Bi are generated at an arbitrary position in the screen, the focal position at that position can be calculated. Therefore, the focus lens included in the optical system 101 is driven automatically according to the calculation result. It is also possible to adjust the focus.

  In the example shown in FIG. 6, the case where the exit pupil of the photographing lens 201 is divided into left and right has been described, but it may be divided into upper and lower, and in this case, the distance between subjects having a luminance difference in the vertical direction is detected. It becomes possible to do.

  Next, an image file generation method corresponding to a captured image in the embodiment of the imaging apparatus according to the present invention will be described with reference to FIG.

  FIG. 7A is a diagram in which the light receiving area of the image sensor 103 of the imaging apparatus according to the present embodiment is divided into 64 areas of 8 × 8. Each divided area is referred to as a divided area 801, and a divided area corresponding to the upper left of the screen is d00, which is d00 to d77 as shown in FIG.

  FIG. 7B is an example of an image photographed by the imaging apparatus of the present embodiment. In the photographed image of FIG. 7B, the main subject exists at three types of distances, and the distances are present at distances of 2 m, 5 m, and 10 m from those close to the imaging device. Shall.

  FIG. 7C shows the photographed image shown in FIG. 7B for each of the divided areas defined in FIG. 7A, and the distance to the main subject existing in that area with reference to FIG. The result calculated using the distance detection operation is shown.

  The features of the imaging apparatus of the present embodiment are as follows. First, the divided area where the main subject is determined to be present at a distance shorter than the predetermined distance and the surrounding divided areas are subjected to a refocus process in which the subject in the area is focused at the time of subsequent refocus image generation. It is considered that this is an area that can be performed. For this reason, the signals of the divided pixels 302 in the divided area are stored for all the pixels. On the other hand, the other divided areas, which are determined to have the main subject at a distance farther than the predetermined distance, are considered to be areas where refocusing is not performed later. Therefore, it is possible to suppress the size of the generated image file by storing the signals of the divided pixels 302 of the divided area by adding or averaging the recording pixel area 301 unit (each unit area). Yes.

  In the shooting of FIG. 7B, it is assumed that a subject at a distance of 5 m or less may be subjected to a refocus process for focusing on the subject when the refocus image is generated later. Accordingly, as shown in FIG. 7C, the divided pixels included in the divided region including the main subject at a distance of 5 m or less (below a predetermined distance) and the region 802 that is a divided region adjacent to the divided region. This signal is stored as it is for all pixels. The signals of the divided pixels not included in the area 802 are added or averaged in the recording pixel area 301 in the digital signal processing circuit 113 to compress the data amount to be stored. In addition, which area is added and which area is not added is recorded in the metadata of the recorded image so that the image can be correctly generated when the refocus image is generated later.

  Note that the distance used for switching between addition and non-addition of the output signals of the divided pixels 302 is 5 m in the description using FIG. However, it is considered that the optimum distance naturally changes depending on the configuration of the imaging device such as an imaging device and an optical system such as a lens to be used, and photographing conditions such as a lens aperture value. As an example, in the case of shooting using an auxiliary light source (subject irradiating means) such as a flash, there is little possibility of focusing on a subject that is farther than the distance that the irradiation light from the auxiliary light source reaches. In this case, the addition / non-addition switching distance of the divided pixels may be set as the strobe light reach distance. Usually, in order to match the signal levels of the addition region and the non-addition region, it is appropriate to perform an addition averaging process in which the added signal is divided by the number of pixels and output. On the other hand, in night scene photography using strobe light, the background area signal that the strobe light does not reach is added as it is instead of the averaging process, so that the sensitivity of the background area is improved. There is also an advantage that an equivalent side effect can be obtained.

  In the example shown in FIG. 7C described above, the signals of the divided pixels included in the divided area including the main subject at a distance of 5 m or less and the divided areas adjacent to the divided area are stored for all pixels. explained. However, the present invention is not limited to this, and only the signals of the divided pixels included in the divided area including the subject that is closer than the predetermined distance may be stored for all pixels.

  In the imaging apparatus according to the present embodiment described with reference to FIG. 7, the recording pixels in the divided area 801 in which the subject is determined to be far away are added inside the digital signal processing circuit 113. As explained. However, the present invention is not limited to this, and a configuration in which addition is performed inside the image sensor 103 may be employed. In this case, since the amount of image signal output from the image sensor 103 can be reduced, a series of photographing operations can be further speeded up. However, in this case, the operation for calculating the distance to the subject described with reference to FIG. 6 must be performed before adding the pixel signals. Therefore, it is necessary to have a means for calculating the subject distance, a determination circuit for determining whether to perform addition, and the like inside the image sensor, not the digital signal processing circuit 113 outside the image sensor 103.

  As mentioned above, although embodiment of the imaging device concerning embodiment of this invention was described using FIGS. 1-7, this invention is not limited to this and can take various forms. .

  For example, in the pixel configuration of the present embodiment, in order to explain the structure of the pixel in an easy-to-understand manner, the divided pixels under the same microlens are divided by 6 × 6. However, the present invention is not limited to this, and various numbers and A configuration having divided pixels having a shape may be used.

  In the present embodiment described with reference to FIG. 1, the image processing such as image reconstruction is described as being performed by the digital signal processing circuit 113 which is one of the components of the imaging apparatus. The image processing need not be performed inside the imaging apparatus. Specifically, the image processing means is provided in a device different from the imaging device, such as a PC (personal computer), and the imaging data obtained by the imaging device is transferred to the PC, and image processing is performed in the PC. It is also possible to do so.

(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which a program code of software in which a procedure for realizing the functions of the above-described embodiments is described is recorded is supplied to the system or apparatus. The computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and program storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, nonvolatile memory card, ROM, or the like can also be used.

  Further, by making the program code read by the computer executable, the functions of the above-described embodiments are realized. Furthermore, when the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

Claims (14)

A plurality of two-dimensionally arranged photoelectric conversion units that photoelectrically convert light incident through the photographing lens and output an electrical signal;
On the optical axis, between the photographic lens and the plurality of photoelectric conversion units, one is associated with each of a plurality of unit areas obtained by dividing the plurality of photoelectric conversion units by a predetermined number. An imaging means having a plurality of microlenses,
Determining means for determining whether or not each of the divided areas obtained by dividing the image based on the electrical signal obtained from the imaging means is a divided area that may be used for refocus processing for generating an image focused on an arbitrary distance. When,
Among the plurality of divided regions, the electric signal output from the photoelectric conversion unit constituting the unit region included in the divided region is determined that there is no possibility of using the refocus processing by the determination unit, the unit area Adding means for adding each time;
Among the plurality of divided areas, it records the electrical signals summed divided regions by said adding means, adds the electrical signals of the determined divided regions and could be used to refocus the processing by the determination unit An image pickup apparatus comprising: a recording unit that records each of the images without recording. The determination means includes
For each divided area obtained by dividing the image based on the electrical signal obtained from the imaging means into a plurality of areas, calculating means for calculating the distance to the subject included in each divided area;
Of the plurality of divided areas, a divided area having a distance to the subject calculated by the calculating unit is equal to or less than a predetermined distance is determined as a divided area that may be used for the refocus processing, The imaging apparatus according to claim 1, wherein the other divided areas are determined as divided areas that are not likely to be used for the refocus processing. The determination means includes
For each divided area obtained by dividing the image based on the electrical signal obtained from the imaging means into a plurality of areas, calculating means for calculating the distance to the subject included in each divided area;
Among the plurality of divided areas, a divided area whose distance to the subject calculated by the calculating unit is equal to or less than a predetermined distance and a divided area adjacent to the divided area can be used for the refocus processing. The imaging apparatus according to claim 1, wherein the imaging device is determined to be a divided region having a characteristic, and other divided regions are determined to be divided regions that are not likely to be used for the refocus processing.   The calculation means calculates a first combined signal for each unit area by adding an electrical signal obtained from a photoelectric conversion unit that receives a light beam that has passed through the first pupil area of the photographic lens, and A plurality of units included in each of the divided regions are calculated by adding a signal obtained from a photoelectric conversion unit that has received a light beam that has passed through a second pupil region different from one pupil region to calculate a second combined signal. The imaging apparatus according to claim 2 or 3, wherein a distance to a subject is calculated based on a phase difference between the first synthesized signal and the second synthesized signal in a region. A subject irradiation means for irradiating the subject;
5. The method according to claim 2, wherein when photographing is performed using the subject irradiation unit, the predetermined distance is a distance that the irradiation light from the subject irradiation unit reaches. Imaging device.   The imaging apparatus according to claim 1, wherein the adding unit is configured inside the imaging unit.   6. The imaging apparatus according to claim 1, wherein the adding unit is configured outside the imaging unit.   8. The recording device according to claim 1, wherein the recording unit further records data indicating the divided area to which the electric signal is added and the divided area to which the electric signal is not added. The imaging device described in 1.   The said addition means further performs the process which averages the said added electric signal for every said unit area | region, and the said recording means records the said averaged electric signal. The imaging apparatus of Claim 1. A plurality of two-dimensionally arranged photoelectric conversion units that photoelectrically convert light incident through the photographing lens to output an electrical signal, and between the photographing lens and the plurality of photoelectric conversion units on the optical axis And a method of controlling an imaging apparatus including an imaging unit having a plurality of microlenses arranged in association with each of a plurality of unit areas obtained by dividing the plurality of photoelectric conversion units by a predetermined number. Because
Whether the determination unit is a divided region that may be used for refocus processing for generating an image focused at an arbitrary distance for each divided region obtained by dividing the image based on the electrical signal obtained from the imaging unit into a plurality of regions. A determination step for determining;
Addition means, among the plurality of divided regions, the electric signal output from the photoelectric conversion unit constituting the unit region included in the divided region is determined that there is no possibility of using the refocusing processed by the determination step Adding step for each unit region;
The recording means records the electric signal of the divided area added in the adding step among the plurality of divided areas, and the divided area determined to be used for the refocus process in the determining step. And a recording step of recording each of the electrical signals without adding them. A plurality of two-dimensionally arranged photoelectric conversion units that photoelectrically convert light incident through the photographing lens and output an electrical signal;
On the optical axis, between the photographic lens and the plurality of photoelectric conversion units, one is associated with each of a plurality of unit areas obtained by dividing the plurality of photoelectric conversion units by a predetermined number. An imaging means having a plurality of microlenses,
Calculating means for calculating a distance to a subject included in each divided area for each divided area obtained by dividing the image based on the electrical signal obtained from the imaging means into a plurality of areas;
Adding means for adding, for each unit region, an electrical signal output from the photoelectric conversion unit constituting the unit region,
The adding means does not add an electric signal of a divided area in which the distance to the subject calculated by the calculating means is within a predetermined distance range among the plurality of divided areas, and is calculated by the calculating means. imaging apparatus characterized by adding the electrical signals of the divided regions where the distance to the object is at a distance other than the distance range in which the predetermined. A plurality of two-dimensionally arranged photoelectric conversion units that photoelectrically convert light incident through the photographing lens to output an electrical signal, and between the photographing lens and the plurality of photoelectric conversion units on the optical axis And a control method for an imaging apparatus having an imaging means having a plurality of microlenses arranged in association with each of a plurality of unit areas obtained by dividing the plurality of photoelectric conversion units by a predetermined number. Because
A calculating step for calculating a distance to a subject included in each divided region for each divided region obtained by dividing the image based on the electrical signal obtained from the imaging unit into a plurality of divided regions;
And an adding step of adding an electrical signal output from the photoelectric conversion unit constituting the unit region for each unit region,
In the adding step, the electric signal of the divided region in which the distance to the subject calculated by the calculating step is within a predetermined distance range among the plurality of divided regions is not added, and is calculated by the calculating step. control method for an imaging apparatus characterized by adding the electrical signals of the divided regions where the distance to the object is at a distance other than the distance range in which the predetermined.   The program for making a computer perform each process of the control method of Claim 10 or 12.   A computer-readable storage medium storing the program according to claim 13.

Images