JP2010219741A - Imaging apparatus and image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of easily obtaining the area division information of images. <P>SOLUTION: The imaging apparatus includes an imaging element 11 for picking up an object image through a photographing optical system 1, a control means 24 for making the imaging element 11 pick up the images of a plurality of frames while changing the aperture 2 of the photographing optical system 1, and a division information generation means 24 for generating division information for dividing the picked up image into a plurality of areas on the basis of the images of the plurality of frames. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging device and an image processing device.

  Depth information (depth map) representing a position in a direction perpendicular to the screen is known for pixels constituting an image (see Patent Document 1).

JP 2008-141666 A

  In Patent Document 1, depth map information (image area division information) is acquired by performing image processing on images acquired respectively by a plurality of cameras, and thus there is a problem that a plurality of cameras are required and the calculation processing is complicated. .

An image pickup apparatus according to the present invention is based on an image pickup device that picks up a subject image via a photographing optical system, a control unit that causes the image pickup device to pick up a plurality of frames while changing a diaphragm of the photographing optical system, and a plurality of frames. And division information generating means for generating division information for dividing the captured image into a plurality of regions.
An image processing apparatus according to the present invention captures a main image with an image sensor that captures a subject image via a photographic optical system, captures a plurality of sub-images with the image sensor while changing the aperture of the photographic optical system, Division information for dividing the main image into a plurality of regions is generated based on the sub-images, and predetermined image processing is performed on the main image in units of a plurality of regions based on the division information. .

  According to the present invention, it is possible to easily obtain area division information of an image.

It is a figure explaining the principal part structure of the electronic camera by one embodiment of this invention. It is a flowchart explaining the flow of imaging | photography process. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates the acquisition image for depth maps. It is a figure which illustrates a depth map image. It is a flowchart explaining the detail of a depth map information generation process. It is a figure which illustrates a depth map information table. It is a figure which illustrates a computer apparatus. It is a flowchart explaining an example of an image processing program.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. FIG. 1 is a diagram for explaining a main configuration of an electronic camera according to an embodiment of the present invention. In FIG. 1, a lens barrel 50 configured to be detachable from the camera body 100 is attached. A display unit (not shown) is provided on the back of the camera body 100.

  Light from the subject enters the camera body 100 via the photographing optical system 1 and the diaphragm 2 of the lens barrel 50. The subject light incident on the camera body 100 is guided to the upper finder by a quick return mirror (hereinafter referred to as a main mirror) 3 positioned as indicated by a broken line before being released, and forms an image on the diffusion screen 6. A part of the subject light incident on the camera body 100 is reflected downward by the sub mirror 4 and also enters the distance measuring element 5. The distance measuring element 5 is used in a focus detection process for detecting a focus adjustment state by the lens barrel 50.

  The subject light imaged on the diffusion screen 6 further enters the pentaprism 8 via the condenser lens 7. The pentaprism 8 guides incident subject light to the eyepiece 9.

  After the release, the main mirror 3 is rotated to the position indicated by the solid line, and the subject light is guided to the image sensor 11 via the focal plane shutter 10 to form a subject image on the imaging surface. The image sensor 11 is configured by, for example, a CMOS image sensor including a plurality of charge storage photoelectric conversion elements corresponding to pixels. The image sensor 11 captures a subject image on the imaging surface and outputs a photoelectric conversion signal corresponding to the brightness of the subject image.

  The control unit 12 includes an image processing unit 21, a focus detection calculation unit 22, a buffer memory 23, a CPU 24, a lens drive control unit 25, an aperture drive control unit 26, and a flash memory 27, and controls each process performed by the electronic camera.

  The image processing unit 21 converts the photoelectric conversion signal output from the image sensor 11 into digital data, and performs predetermined image processing on the converted data. The CPU 24 inputs a signal output from each part in the camera, performs a predetermined calculation, and outputs a control signal based on the calculation result to each part.

  The focus detection calculation unit 22 performs a known phase difference type focus detection calculation using a detection signal from the distance measuring element 5. By this calculation, a focus adjustment state (defocus amount) by the photographing optical system 1 is obtained, and a moving direction and a moving amount of a focus lens (not shown) constituting the photographing optical system 1 are calculated according to the defocus amount. The movement amount information is sent to the lens driving amount control unit 25.

  The lens drive amount control unit 25 sends a drive signal to the lens drive mechanism 13 based on the movement amount information. Upon receiving the drive signal, the lens drive mechanism 13 adjusts the focus by moving the focus lens forward and backward in the optical axis direction.

  The CPU 24 obtains the subject distance to the main subject that is the focus detection target according to the defocus amount detected by the focus detection calculation unit 22. For example, by storing data indicating the relationship between the defocus amount and the subject distance in the flash memory 27 in advance, the subject distance is obtained with reference to the data.

  The buffer memory 23 temporarily stores data before the image processing by the image processing unit 21, after the image processing, and during the image processing, as well as temporarily stores data at the time of the depth map information generation processing, It is used for storing an image file before recording to (not shown) or storing an image file read from a recording medium. The recording medium is configured to be detachable from the camera body 100, and is configured by a memory card incorporating a semiconductor memory, a hard disk drive, or the like.

  The flash memory 27 stores programs executed by the CPU 24, data necessary for processing performed by the CPU 24, and the like. The contents of the program and data stored in the flash memory 27 can be added or changed by an instruction from the CPU 24.

  The aperture drive control unit 26 sends a drive signal to the aperture drive mechanism 14 based on an instruction to open or close the aperture 2 sent from the CPU 24. Upon receipt of the drive signal, the aperture drive mechanism 14 is driven to change the aperture of the aperture 2 to adjust it to a predetermined aperture value.

  The operation member 15 includes various buttons and switches of the camera body 100, and outputs an operation signal corresponding to the operation content of each operation member such as mode switching and menu operation to the control unit 12. A half-press operation signal, which will be described later, is output when the release button is pressed down to about half of the normal stroke, and the output is canceled when the half-stroke press operation is canceled. Further, a full-press operation signal described later is output when the release button is pressed down to the normal stroke, and the output is canceled when the normal stroke press-down operation is canceled.

  The electronic camera according to the present embodiment includes an image processing function that generates depth map information of a captured image. The depth map information generation process is performed by the control unit 12 described above. When an operation signal from a menu operation switch constituting the operation member 15 is input, the CPU 24 displays a menu operation screen for performing menu processing on a display unit (not shown) on the back. When an operation signal indicating an instruction to select a “depth map generation function ON” item from the menu items is input from the operation member 15, the depth map information generation function is turned on.

  When the depth map information generation function is on, the CPU 24 executes depth map information generation processing on the image data after imaging, and then records the image data and depth map information on a recording medium (not shown). When the depth map information generation function is off, the CPU 24 records the image data on a recording medium (not shown) without executing the depth map information generation process on the image data after imaging. Details of the depth map information generation processing to be executed will be described later.

<Shooting process>
When the depth map information generation function is turned on and the half-press operation signal is input from the operation member 15 in response to the half-press operation of the release button of the electronic camera, the CPU 24 activates the photographing process according to the flowchart of FIG. . In step S11 of FIG. 2, the CPU 24 performs photometry processing and proceeds to step S12. In the photometric process, the brightness of the imaging surface of the image sensor 11 is obtained based on an output from a photometric sensor (not shown).

  In step S12, the CPU 24 sets shooting conditions and proceeds to step S13. Specifically, the ISO sensitivity, the control shutter speed, and the control aperture value are obtained based on the brightness of the imaging surface and a preset shooting mode (for example, program auto, shutter speed priority auto, etc.). .

  In step S13, the CPU 24 performs AF (automatic focus adjustment) processing and proceeds to step S14. Here, the focus detection target area (focus area) is, for example, approximately the center of the screen to be imaged.

  In step S14, the CPU 24 determines whether or not a shooting instruction has been issued. When the full pressing operation signal is input from the operation member 15, the CPU 24 makes a positive determination in step S14 and proceeds to step S15. When the full pressing operation signal is not input from the operation member 15, the CPU 24 makes a negative determination in step S14 and proceeds to step S25.

  In step S25, the CPU 24 determines whether the time is up. The CPU 24 makes an affirmative determination in step S25 when the time measured in a state where no operation is performed after the half-press activation has reached a predetermined time (for example, 10 seconds), and ends the process of FIG. If the measured time is less than the predetermined time, the CPU 24 makes a negative determination in step S25, returns to step S11, and repeats the above-described processing.

  In step S15, the CPU 24 performs the main imaging process and proceeds to step S16. As a result, the main mirror 3 is driven up, the focal plane shutter 11 is driven open, and the image sensor 11 acquires image data for main imaging based on the imaging conditions set in step S12. In step S <b> 16, the CPU 24 instructs the image processing unit 21 to read image data that has been actually captured from the image sensor 11, and proceeds to step S <b> 17.

  In step S17, the CPU 24 performs pan focus setting and proceeds to step S18. Specifically, an instruction is sent to the aperture drive control unit 26 to drive the aperture 2 to the minimum aperture stop, and the depth map imaging is replaced with the control shutter time as the shutter time in the depth map imaging described later. Set the shutter time for the above (accumulation time controlled by so-called electronic shutter). In addition, the focus lens (not shown) of the photographing optical system 1 is moved to a position where the closest subject is in focus.

  In step S18, the CPU 24 performs depth map imaging processing and proceeds to step S19. The imaging for the depth map is performed while changing the aperture value while the main mirror 3 is driven up. The image sensor 11 repeats the processes up to step S21 described later, thereby acquiring image data for depth maps having different aperture values, for example, for a predetermined number of frames at a rate of 30 frames / second.

  In step S19, the CPU 24 instructs the image processing unit 21 to read out the image data captured for the depth map from the image sensor 11, and proceeds to step S20. The read image data for the depth map is temporarily stored in the buffer memory 23.

  FIG. 3 is a diagram illustrating an acquired image for depth map acquired by pan focus setting. The depth map image has a smaller data size than the main image. In the example of FIG. 3, nine subjects are captured from the object 0 closest to the electronic camera to the object 8 farthest from the electronic camera. The distance from the electronic camera to each object is d0 <d1 <d2 <d3 <d4 <d5 <d6 <d7 <d8. However, dn (n = 0, 1,..., 8) is a distance from the electronic camera to the object n. In the case of FIG. 3, almost all objects are in focus.

  In step S20, the CPU 24 determines whether or not the aperture 2 is opened to a predetermined aperture value (for example, an open aperture). If the CPU 24 is open to the predetermined aperture value, the CPU 24 makes a positive determination in step S20 and proceeds to step S22. If the CPU 24 has not opened to the predetermined aperture value, the CPU 24 makes a negative determination in step S20 and proceeds to step S21.

  In step S21, the CPU 24 enlarges the aperture of the diaphragm 2, makes settings for imaging the depth map of the next frame, and returns to step S18. Specifically, the aperture drive control unit 26 that has received an instruction from the CPU 24 drives the aperture 2 to a predetermined aperture value, and also changes the aperture value by shortening the shutter time in the imaging for the depth map of the next frame. Control is performed so that the exposure between frames is aligned by performing exposure correction correspondingly. It should be noted that the amount of change in the aperture value (corresponding to the aperture level described later) (corresponding to the aperture level difference described later) does not have to be evenly spaced, and is changed in finer steps as the aperture 2 is opened, and closer to the minimum aperture 2 side. Change in wide steps. Thereby, the depth of the depth map (corresponding to a depth level described later) can be made uniform in each frame.

  FIG. 4 is a diagram exemplifying a depth map acquisition image acquired in a state where, for example, the aperture 2 is expanded from the minimum aperture to a predetermined value. In the example of FIG. 4, the object 8 farthest from the electronic camera is out of focus.

  FIG. 5 is a diagram illustrating a depth map acquired image acquired with the aperture 2 further expanded. In the example of FIG. 5, the object 8 farthest from the electronic camera and the object 7 farthest after the object 8 are out of focus.

  Similarly, FIGS. 6 to 12 are diagrams illustrating depth map acquired images acquired at different aperture values by a predetermined aperture level difference. In FIGS. 6 to 12, as the diaphragm 2 is expanded, the object is gradually defocused from the object with the farther shooting distance.

  In step S22, the CPU 24 issues an instruction to perform predetermined image processing on the main image data to form an image file, and proceeds to step S23. In step S23, the CPU 24 generates depth map information based on the image data for the depth map, and proceeds to step S24. Details of the depth map information generation processing will be described later.

  In step S24, the CPU 24 instructs to record the image file including the main image data and the depth map information on the recording medium, and ends the process shown in FIG.

<Depth map information generation processing>
Details of the depth map information generation processing will be described with reference to the flowchart of FIG. In step S231 in FIG. 14, the CPU 24 identifies the image Pic (t) and the image Pic (t-1) among the image data for depth map stored in the buffer memory 23, and proceeds to step S232. In this example, for example, the aperture value is changed in 9 steps from the minimum aperture stop (with an aperture level of 8) to the open aperture (with an aperture level of 0), and a nine-frame depth map image is obtained. In this case, the images initially specified as the aperture level 8 (initial value) are the image Pic (8) illustrated in FIG. 3 and the image Pic (7) illustrated in FIG.

  In step S232, the CPU 24 divides the image Pic (t) and the image Pic (t-1) and proceeds to step 233. Specifically, the depth map image is divided into 10 × 10 = 100 blocks (areas) divided into 10 equal parts vertically and horizontally, and the process proceeds to step S233. The division is for convenience to perform a subtraction process to be described later in units of blocks.

  In step S233, the CPU 24 performs subtraction processing on the block basis for the image Pic (t) and the image Pic (t-1) after the division processing, and proceeds to step S234. In the subtraction process, for example, information indicating the sharpness of the outline (adjacent difference data or the like) is generated for each block based on the pixel data in the block, and the sharpness information on the image Pic (t) and the image Pic (t− The difference from the sharpness information regarding 1) is calculated for each block.

  In step S234, the CPU 24 extracts a block (region) in which the calculated difference is equal to or greater than a predetermined difference, and the process proceeds to step S235. For example, if the image Pic (8) at the aperture level 8: FIG. 3 and the image Pic (7) at the aperture level 7: FIG. 4 are specified, the difference corresponding to the block corresponding to the object 8 is more than a predetermined difference by the difference processing. A difference is detected. This is because the block corresponding to the changed object 8 is extracted as the difference because the object 8 changes from in-focus to out-of-focus between FIGS.

  In step S235, the CPU 24 determines whether or not the processing has ended up to the image Pic (0). The CPU 24 makes a positive determination in step S235 and proceeds to step S236 when the diaphragm level has been completed to 0 in the process of step S237 described later. If the CPU 24 has not finished reaching the aperture level 0 by the process of step S237 described later, the CPU 24 makes a negative determination in step S235 and proceeds to step S237.

  In step S237, the CPU 24 subtracts 1 from the value of t and returns to step S231. When returning to step S231, the same processing as the processing described above is repeated for the image Pic (t) and the image Pic (t-1) based on t after subtraction.

  4 to 12, the out-of-focus state when the object n is changed from the in-focus state to the out-of-focus state is shown in gray. Gray display indicates that a block corresponding to the object n changed to out-of-focus is extracted as a difference.

  FIG. 15 is a diagram illustrating a table that is an example of depth map information. For example, the aperture level difference “8-7” represents a subtraction process performed between an image Pic (8) with an aperture level of 8: FIG. 3 and an image Pic (7) with an aperture level of 7: FIG. The “reaction area” next to it represents a region (object 8) where a difference greater than or equal to a predetermined difference appears in the subtraction process. Further, the adjacent “depth level” represents depth = depth level (8) of the area (object 8). The deeper the depth level, the longer the subject distance (maximum 8), and the shallower the depth, the closer the subject distance (minimum 0). FIG. 15 shows that the depth level is deeper as it is closer to the minimum aperture stop (closer to the aperture level 8), and the depth level is shallower as it is closer to the open stop (closer to aperture level 0).

  In step S236, the CPU 24 temporarily stores the depth map information illustrated in FIG. 15 in the buffer memory 23, and ends the processing in FIG.

  The depth map information may be stored as a depth map image illustrated in FIG. 13 instead of or in addition to the table data illustrated in FIG. According to FIG. 15, the image is divided into regions according to the depth level. Then, combine depth levels 8 and 7 into one, combine depth levels 6 and 5 into one, combine depth levels 4 and 3 into one, and combine depth levels 2 and 1 into one. This is an example in which all are divided into five stages (area 0 to area 4). The divided areas 0-4 can be easily distinguished by changing the pixel density of the depth map image or by changing the pattern or color component. The reason why the outline of the area in FIG. 15 does not match the outline of the object in FIGS. 3 to 12 is that FIG. 13 divides the area in units of blocks after the division process (step S232).

According to the embodiment described above, the following operational effects can be obtained.
(1) The electronic camera causes the image pickup device 11 to pick up a plurality of frames while changing the aperture 2 of the shooting optical system 1 while changing the aperture 2 of the shooting optical system 1. Since the CPU 24 that generates the division information for dividing the captured image into a plurality of areas based on the image is provided, the area division information of the image obtained by the electronic camera can be easily obtained.

(2) Since the CPU 24 generates division information for dividing the captured image into a plurality of regions according to the depth corresponding to the shooting distance, the depth map information indicating the distribution state in the depth direction of the image Can be easily obtained.

(3) The CPU 24 performs a subtraction process on each image of a plurality of frames with an image of another frame having the closest aperture diameter at the time of imaging, and generates division information based on the subtraction value and the aperture diameter. The depth map information can be obtained by performing a simple subtraction process.

(4) The CPU 24 associates the image area where the subtraction value larger than the predetermined value is obtained with the aperture diameter at the time of capturing the image used for the subtraction process of the subtraction value. It can be seen at which aperture diameter the focus state has changed.

(5) The CPU 24 increases the depth as the aperture diameter is closer to the minimum aperture, and decreases the depth as the aperture diameter is closer to the open position. .

(6) Since the CPU 24 has generated the depth map image showing the image area in which the subtraction value larger than the predetermined value is obtained in different modes for each depth, the image area in which the focus state has changed can be easily expressed for each aperture diameter. .

(7) Since the CPU 24 generates the table data that associates the image area where the subtraction value larger than the predetermined value is obtained with the information indicating the depth, the CPU 24 generates the division information with a data size smaller than the data size of the depth map image. Can be saved.

(8) The CPU 24 of the electronic camera may further perform processing for recording data on a recording medium. In this case, the CPU 24 causes the image pickup device 11 to pick up a plurality of sub-images while changing the aperture of the photographing optical system, and divides the main image picked up by the image pickup device 11 into a plurality of regions based on the sub-images of the plurality of frames. Division information is generated, and main image data and division information data are recorded on a recording medium. Thereby, both the main image data and the division information can be stored in the recording medium. In addition, by using a sub-image having a data size smaller than that of the main image for generating the division information, it is possible to reduce the processing load required for the generation of the division information.

(9) Since the CPU 24 blocks the sub-image when performing the subtraction process, the CPU 24 can reduce the processing load required for generating the division information including the subtraction process as compared with the case where the sub-image is not blocked. it can.

(Modification 1)
Although a single-lens reflex electronic camera has been described as an example, the present invention may be applied to other types of electronic cameras. In addition, although an example in which the acquisition of the depth map image with different aperture values is performed after the main imaging has been described, the depth map image is acquired before the main imaging, for example, when acquiring the live view image in the case of a camera having a live view display function. You may comprise so that an image may be acquired.

(Modification 2)
In the case of acquiring depth map images with different aperture values, the example in which the aperture diameter is acquired while changing the aperture 2 in the direction of expanding from the small aperture has been described. You may comprise so that the image for depth maps may be acquired, changing.

(Modification 3)
In the above embodiment, a plurality of main images captured by the image sensor 11 based on the sub-images of the plurality of frames after the imaging processing for capturing the sub-images of the plurality of frames while changing the aperture 2 of the photographing optical system 1 are completed. The division information to divide the area is generated. Instead, when sub-images for at least two frames used for subtraction processing are acquired, subtraction processing is performed between the frames. Information may be generated.

(Modification 4)
In the above description, the process executed by the CPU 24 of the electronic camera has been described as an example. However, the depth map information generation process (FIG. 14) described above is executed by causing the computer device 200 shown in FIG. An information generation apparatus may be configured. When the program is used by being loaded into the personal computer 200, the program is loaded into the data storage device of the personal computer 200, and the program is executed to be used as a depth map information generation device.

  The loading of the program to the personal computer 200 may be performed by setting a recording medium 204 such as a CD-ROM storing the program in the personal computer 200, or to the personal computer 200 by a method via the communication line 201 such as a network. You may load. When passing through the communication line 201, the program is stored in the hard disk device 203 of the server (computer) 202 connected to the communication line 201. The program can be supplied as various types of computer program products such as provision via the recording medium 204 or the communication line 201.

(Modification 5)
An image processing apparatus that performs image processing on main image data using the main image data and depth map information recorded on a recording medium by the electronic camera described above can also be configured. In this case, the image processing apparatus is configured by causing the computer apparatus 200 shown in FIG. 16 to execute a program for performing image processing on the main image data. The electronic camera and the computer device 200 are connected (wired connection or wireless connection), and the main image data and the depth map information recorded on the recording medium by the electronic camera are transmitted to the computer device 200. May be configured to perform image processing on the main image data.

  FIG. 17 is a flowchart illustrating an example of an image processing program. In step S51 of FIG. 17, the personal computer 200 identifies the designated image file from the data recorded on the recording medium, and proceeds to step S52.

  In step S52, the CPU 24 reads main image data from the identified image file, and proceeds to step S53. In step S53, the CPU 24 reads the depth map information associated with the main image data, and proceeds to step S54.

  In step S54, the CPU 24 performs different filter processing for each region on the data of the main image, and proceeds to step S55. Specifically, the main image is divided in accordance with the divided regions illustrated in FIG. 13 (or the depth level illustrated in FIG. 15), and differs between the divided regions with respect to the main image data included in each divided region. Apply filtering. In the filter process, for example, a process of enhancing or correcting a predetermined color component included in the area 0, the area 1, and the area 2 that are close to the shooting distance.

  In step S55, the CPU 24 records the data of the main image after the filter process on the recording medium, and ends the process of FIG.

  As the above-described filtering process, a process of adding different electronic blur between the divided areas, a different contour adjustment process between the divided areas, or a different noise reduction process between the divided areas may be performed.

(Modification 6)
Instead of the filter processing described above, a synthesis process may be performed in which an image in a predetermined divided region (for example, a background with a long shooting distance) is replaced with a synthesis image prepared in advance.

(Modification 7)
Moreover, you may perform the process which represents a three-dimensional image instead of the filter process mentioned above. In this case, when the three-dimensional image is expressed, the data of the three-dimensional image is arranged in the direction of the depth level (depth direction) according to the depth map information.

  The above description is merely an example, and is not limited to the configuration of the above embodiment.

DESCRIPTION OF SYMBOLS 1 ... Shooting optical system 2 ... Aperture 11 ... Imaging element 12 ... Control part 21 ... Image processing part 22 ... Focus detection calculating part 23 ... Buffer memory 24 ... CPU
25 ... Lens drive control unit 26 ... Aperture drive control unit 27 ... Flash memory 50 ... Lens barrel 100 ... Camera body

Claims (9)

An image sensor that captures a subject image via an imaging optical system;
Control means for causing the image sensor to pick up images of a plurality of frames while changing the aperture of the photographing optical system;
An imaging apparatus comprising: division information generating means for generating division information for dividing the captured image into a plurality of regions based on the images of the plurality of frames. The imaging device according to claim 1,
The imaging apparatus according to claim 1, wherein the division information generation unit generates division information for dividing the captured image into a plurality of regions according to a depth corresponding to a shooting distance. The imaging device according to claim 2,
The division information generation means performs a subtraction process on each image of the plurality of frames with an image of another frame having the closest aperture diameter at the time of imaging, and the division information is based on the subtraction value and the aperture diameter. An imaging device characterized by generating The imaging device according to claim 3.
The imaging apparatus characterized in that the division information generation means associates an image area in which the subtraction value larger than a predetermined value is obtained with an aperture diameter at the time of imaging of an image used for the subtraction process of the subtraction value. The imaging apparatus according to claim 4,
The imaging apparatus according to claim 1, wherein the division information generation unit increases the depth as the aperture diameter is closer to a minimum aperture, and decreases the depth as the aperture diameter is closer to an open position. The imaging apparatus according to claim 5,
The imaging apparatus according to claim 1, wherein the division information generation unit generates a distribution image in which an image area where the subtraction value larger than the predetermined value is obtained is shown in a different manner for each depth. In the imaging device according to claim 5 or 6,
The imaging apparatus according to claim 1, wherein the division information generation unit generates table data that associates an image area in which the subtraction value larger than the predetermined value is obtained with information indicating the depth. In the imaging device according to any one of claims 1 to 7,
It further comprises recording means for recording data on a recording medium,
The control means causes the imaging element to capture a plurality of sub-images while changing the aperture of the photographing optical system,
The division information generation unit generates division information for dividing the main image captured by the imaging element into a plurality of regions based on the sub-images of the plurality of frames.
The image recording apparatus, wherein the recording unit records data of the main image and data of the division information on the recording medium. Capture the main image with an image sensor that captures the subject image via the photographic optical system,
Taking a sub-image of a plurality of frames with the image sensor while changing the aperture of the photographing optical system,
Generating division information for dividing the main image into a plurality of regions based on the sub-images of the plurality of frames;
An image processing apparatus comprising: an image processing unit that performs predetermined image processing on the main image in units of the plurality of regions based on the division information.

Images