JP7337566B2 - IMAGING DEVICE, IMAGE PROCESSING METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to an imaging device, an image processing method, and a program.

In recent years, in order to achieve both miniaturization and a long focal length, a photographic lens has been proposed that has an imaging optical system (catadioptric optical system) with a reflecting member (reflecting mirror) at the center on the same optical axis. (For example, Patent Document 1). In the imaging optical system of Patent Document 1, the reflecting member described above is used to internally reflect the incident light, and the optical path is turned back within the lens barrel. ing.

JP 2004-85725 A

As described above, since the reflective member is provided in the central part of the catadioptric optical system, it is difficult to provide the catadioptric optical system with a mechanical diaphragm mechanism arranged in the optical path in a normal imaging optical system. is. Without a diaphragm mechanism, the amount of light cannot be adjusted by the diaphragm. That is, the aperture value (F number) indicating the aperture cannot be changed. Therefore, it is difficult to acquire an image with an adjusted F-number in an image pickup apparatus using a catadioptric optical system without an aperture mechanism.

In view of the above circumstances, the present invention provides an image capturing apparatus, an image processing method, and a program capable of obtaining an image corresponding to a desired F-number even when an image is captured using a photographic lens having a catadioptric optical system. It is in.

In order to achieve the above object, an image pickup apparatus of the present invention is an image pickup apparatus which can take an image by a photographing lens having a catadioptric optical system with a constant photographing F-number without an aperture mechanism, and which is arranged two-dimensionally. an imaging element having a plurality of pixel blocks each including a plurality of pixels, and a plurality of microlenses respectively corresponding to the plurality of pixel blocks; and an exit pupil region of the microlenses corresponding to a desired F value and a control means for obtaining an image corresponding to the desired F-number by using pixel signals output from the pixels in the pixel block, wherein the catadioptric optical system receives a part of the incident light flux. and a light shielding portion that blocks the light to generate an unreceivable region in the exit pupil region, and the control means outputs from the pixels corresponding to the exit pupil region different from the exit pupil region corresponding to the desired F value and acquiring the image corresponding to the desired F-number using the obtained pixel signals .

According to the present invention, an image corresponding to a desired F-number can be obtained even when an imaging lens having a catadioptric optical system with a constant imaging F-number without an aperture mechanism is used.

1 is a block diagram showing the overall configuration of an imaging device according to an embodiment of the present invention; FIG. 1 is an explanatory diagram of a catadioptric optical system employed in a photographing lens according to an embodiment of the present invention; FIG. 1 is an explanatory diagram of an imaging optical system of an imaging device according to an embodiment of the present invention; FIG. FIG. 4 is an explanatory diagram of the relationship between pupil regions, pixels, and F-numbers according to the embodiment of the present invention; FIG. 4 is an explanatory diagram of pixels corresponding to a pupil region corresponding to a desired F-number in the embodiment of the present invention; FIG. 4 is an explanatory diagram of a relationship between a non-receivable region and a pupil region (pixel) in the embodiment of the present invention; FIG. 4 is an explanatory diagram of image acquisition when there is a non-receivable region in the embodiment of the present invention;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the embodiments described below are merely examples of configurations that can implement the present invention. The following embodiments can be appropriately modified or changed according to the configuration of the device to which the present invention is applied and various conditions. Therefore, the scope of the invention is not limited by the configurations described in the following embodiments.

FIG. 1 is a block diagram showing the overall configuration of an imaging device 1 according to an embodiment of the invention. The imaging apparatus 1 is a lens-interchangeable digital camera, and is capable of imaging using various photographic lenses. FIG. 1(a) shows an imaging apparatus 1 in which a catadioptric imaging lens 600 having a catadioptric system is attached to a camera body 100. FIG. FIG. 1(b) shows an imaging apparatus 1 having a configuration in which a normal taking lens 500 without a catadioptric system is attached to a camera body 100. FIG.

Various photographing lenses including photographing lenses 500 and 600 can be detachably attached to the camera body 100 . Each photographing lens is attached to the camera body 100 directly or via a mount adapter.

The camera body 100 has an image pickup device 101 , a camera CPU 104 , an operation section 105 , a recording section 106 , an in-viewfinder display section 107 , an external display section 110 , a focal plane shutter 111 , a shutter drive section 112 , and a camera side communication terminal 113 .

The light beams incident on the photographing lenses 500 and 600 pass through the lens group and are guided to the image sensor 101 . The imaging lenses 500 and 600 and the imaging element 101 constitute an imaging optical system. The imaging device 101 has a pixel portion that converts an object image corresponding to an incident light beam into an electrical signal (pixel signal) by photoelectric conversion, and a peripheral circuit that processes and outputs the electrical signal. The pixel portion has a plurality of pixels arranged two-dimensionally (for example, in a matrix (rows and columns)) and performing photoelectric conversion. The peripheral circuit processes electrical signals representing pixel information input from the pixel unit and outputs the processed electrical signals to the camera CPU 104 . A pixel signal is output for each of a plurality of pixels.

The camera CPU 104 executes various processes on electrical signals (pixel signals) from the image sensor 101 . For example, the camera CPU 104 performs correction processing for acquiring image signals and focus detection signals based on electrical signals, and converts the acquired image signals into images for various purposes (recorded images, EVF images, live view images, etc.). Execute the conversion process. The camera CPU 104 develops and executes a program stored in a memory (not shown) to execute the above-described various processes, as well as the process of this embodiment, which will be described later. A configuration in which an image processing circuit provided separately from the camera CPU 104 executes the above processing can also be adopted.

The operation unit 105 is an element that receives operation inputs from the user regarding shooting modes and shooting conditions (desired F number, ISO, exposure time, etc.) of the imaging apparatus 1, shooting instructions, and the like. The operation unit 105 includes, for example, a release button, mode dial, electronic dial, setting button, touch panel, and the like.

The recording unit 106 is a recording medium for recording captured still images and moving images, such as a flash memory. The recording unit 106 may be a medium detachable from the camera body 100, a medium built into the camera body 100, or a combination thereof.

The viewfinder display unit 107 is a so-called electronic viewfinder (EVF) and includes a display 108 and an eyepiece lens 109 . The display 108 is preferably a small, high-definition display means such as an organic EL display or a liquid crystal display. The external display unit 110 is an organic EL display or a liquid crystal display with a screen size suitable for a user to view with the naked eye. The viewfinder display unit 107 and the external display unit 110 display various information such as the setting state of the camera body 100 (desired F value, etc.), live view images, captured images, and the like.

The focal plane shutter 111 is a shutter device for entering or blocking a light flux with respect to the image sensor 101, and is arranged on the front side (object side in the z-axis direction). The shutter driving unit 112 is an element, such as a motor, that controls the exposure time when capturing a still image by driving and controlling the shutter blades.

The camera-side communication terminal 113 is a conducting element provided in the camera mount section on which the photographing lenses 500 and 600 are mounted. The camera-side communication terminal 113 is electrically connected to the lens-side communication terminal 508 provided on the lens mount portion, and relays signals transmitted and received between the camera CPU 104 and the lens CPU 507 .

A photographing lens 600 shown in FIG. 1A is a so-called mirror lens having a catadioptric system. On the other hand, the taking lens 500 shown in FIG. 1B is a conventional zoom lens with a variable focal length. First, the taking lens 500 will be described with reference to FIG.

The photographing lens 500 in FIG. 1B includes a first lens group 501, a second lens group 502, a third lens group 503, a focus driver 504, an aperture 505, an aperture driver 506, a lens CPU 507, and a lens side communication terminal. 508.

A light beam from a subject passes through the first lens group 501, the second lens group 502, and the third lens group 503 and forms an image on the imaging surface of the image sensor 101 to form an image of the subject. The second lens group 502 functions as a variator that advances and retreats in the optical axis direction (z-axis direction) to perform zooming. The third lens group 503 functions as a focus lens that advances and retreats in the optical axis direction (z-axis direction) to perform focus adjustment.

A diaphragm 505 has a plurality of diaphragm blades that adjust the amount of light incident on the imaging lens 500 . A diaphragm driving unit 506 executes diaphragm driving so as to change the size of the diaphragm opening formed by the diaphragm blades of the diaphragm 505 . That is, the aperture drive unit 506 drives the aperture 505 to adjust the amount of light and achieve a desired shooting F-number.

The lens CPU 507 transmits and receives signals to and from the camera CPU 104 via the lens side communication terminal 508 and the camera side communication terminal 113 . Also, the lens CPU 507 controls the focus driving unit 504 and the diaphragm driving unit 506 based on instruction signals from the camera CPU 104 .

The zoom range and maximum F-number of the photographing lens 500 are designed according to the intention of shooting, but in this embodiment, the maximum F-number is set to a constant value regardless of the zoom state or focus state. On the other hand, the distance between the exit pupil and the imaging plane, the so-called exit pupil distance, changes according to the focus state.

Next, the photographing lens 600 will be described with reference to FIGS. 1(a) and 2. FIG. As shown in FIG. 1A, the photographing lens 600 has a first lens group 601, a third lens group 503, a focus driving section 504, a lens CPU 507, and a lens side communication terminal 508. The photographing lens 600 does not have the second lens group 502, which is a variator, and the diaphragm 505 and diaphragm driver 506, which are diaphragm mechanisms, due to the structural difficulty of the catadioptric system.

A catadioptric optical system employed in the photographing lens 600 will be described with reference to FIG. The first lens group 601 of the photographing lens 600 has lenses 601A, B, C, and E and a light blocking portion 601D. A solid line on the left side of the drawing indicates the object 604 , and a dotted line emitted from the object 604 indicates a light ray emitted from one point on the object 604 .

Light emitted from the subject 604 is condensed through the lens 601A and reflected by the lens 601B (primary mirror). The light reflected by the lens 601B is reflected and refracted by the lens 601C (secondary mirror), passes through the lens 601E and the focus lens 503, and forms an image on the imaging surface of the imaging device 101. FIG. The lens 601B is a reflecting lens that reflects and reverses the optical path from the object side. The lens 601C is provided with a light shielding portion 601D on the side opposite to the optical path (object side).

As described above, in the photographing lens 600 having the catadioptric system, the optical path is folded back in the lens, so that the size in the optical axis direction can be kept small and the focal length can be increased.

The imaging optical system of the imaging device 1 according to the embodiment of the present invention will be described with reference to FIG. To implement the present invention, ray space information (light field information) including position information about the position of the ray and angle information about the angle of the ray is used.

In this embodiment, a microlens array (MLA) 10 in which microlenses are arranged over the entire surface of the imaging device 101 is provided in order to acquire angle information. Each of the microlenses 20 included in the microlens array 10 corresponds to a plurality of pixels (pixel blocks) arranged in a matrix.

FIG. 3A shows a front view of the microlens array 10 and the imaging device 101 viewed from the z-axis direction (optical axis direction) and a side view of the microlens array 10 and the imaging device 101 viewed from the x-axis direction (longitudinal direction of the imaging device 101). The microlens array 10 is arranged on the imaging device 101 so that the front principal point of the microlens array 10 is near the imaging plane of the imaging optical system.

When viewed from the front side of the imaging device 1 , each microlens 20 is arranged so as to cover the pixel block of the imaging device 101 . In FIG. 3A, the microlenses are drawn relatively large for clarity of illustration, but the actual size of the microlenses is about several times the pixel size.

FIG. 3B is an enlarged view of the front view (left side) shown in FIG. Grid frames (rectangular regions) in FIG. 3B are individual pixels (for example, pixels 21a, 22a, 23a, 24a, and 25a) of the image sensor 101 . Note that each pixel has one photoelectric conversion unit.

Circles in FIG. 3B are individual microlenses 20 (for example, microlenses 20a, 20b, 20c, and 20d) included in the microlens array 10. As shown in FIG. As shown, one microlens 20 is provided to correspond to a pixel block including 25 pixels arranged in 5 rows and 5 columns. Therefore, the size of the microlens 20 corresponds to the size of the pixel size multiplied by 5 with respect to the x-axis direction and the y-axis direction.

Referring to FIG. 3C, the microlens array 10 associates the pixels under the microlens array 10 with a specific exit pupil area. FIG. 3(c) is a front view (upper side) and a top view of one microlens 20 and a part of a plurality of corresponding pixels as seen from the z-axis direction (optical axis direction) and from the y-axis direction. (lower side).

The top view of FIG. 3(c) is a view of the microlens 20 and part of the imaging element 101 cut along the line CC shown in FIG. 2(b). contains. Pixels 21a, 22a, 23a, 24a, and 25a shown in the top view (lower side) of FIG. 3(c) correspond to the pixels 21a, 22a, 23a, 24a, and 25a shown in FIG. 3(b). The front view (upper side) of FIG. 3C shows the exit pupil plane (exit pupil area 605) of the imaging optical system.

For the sake of explanation, the projection direction is changed between the top view and the front view of FIG. 3(c). Actually, the exit pupil plane in the top view of FIG. xy plane). In addition, one-dimensional projection processing and signal processing will be described below for ease of explanation, but it should be understood that the following description can be easily extended to two-dimensional projection processing and signal processing. be.

As shown in FIG. 3C, each pixel of the image sensor 101 is designed to be conjugate with a specific area on the exit pupil plane (exit pupil area 605) of the imaging optical system by the microlens 20. . The pixel 21a and the exit pupil region 21, the pixel 22a and the exit pupil region 22, the pixel 23a and the exit pupil region 23, the pixel 24a and the exit pupil region 24, the pixel 25a and the exit pupil region 25, They correspond to each other and are mutually beneficial. That is, only the light flux that has passed through the corresponding area (eg, the exit pupil area 21) on the exit pupil plane of the imaging optical system is incident on a certain pixel (eg, the pixel 21a). Therefore, based on the relationship between the passage area of the light flux on the exit pupil plane and the position on the imaging device 101, the angle information of the light flux (ray) can be obtained.

Processing for acquiring an image corresponding to a desired F-number based on pixel signals output from the image sensor 101 will be described with reference to FIGS. 4 and 5. FIG. The processing described below is executed by control means configured by the camera CPU 14 executing a program stored in a memory (not shown). Note that the control means may be configured by the image processing circuit described above. A desired F value may be input by the user by operating the operation unit 105, and the input desired F value may be displayed on the display means (inside viewfinder display unit 107, external display unit 110). . According to the above configuration, it becomes clear to the user that the F value can be adjusted.

FIG. 4 is an explanatory diagram of the relationship between the exit pupil area, the pixels, and the F-number in the embodiment of the present invention. An exit pupil area 605 of the imaging optical system indicated by a circular solid line is an exit pupil area at the maximum F value (F2.0) of the imaging lenses 500 and 600, and corresponds to one microlens 20. area.

FIG. 4 shows exit pupil areas for F-numbers of F2.0, F2.8, F4.0, F5.6, and F8.0. As shown, when the aperture 505 of the taking lens 500 is closed (that is, when the F-number increases), the size (diameter) of the exit pupil area decreases. As can be seen from FIG. 4, the exit pupil area for F8.0 corresponds to the area of one pixel 23, and the exit pupil area for F2.8 is nine pixels 17 to 19, 22 to 24, 27-29 region.

Therefore, even if the diaphragm is not set by the diaphragm mechanism, the control means selectively acquires the pixel signals from the pixels corresponding to the exit pupil region corresponding to the desired F number, thereby obtaining the desired F number. You can get the image you want. More specifically, for example, as shown in FIG. 5A, using pixel signals obtained from one pixel 23a, 23b, 23c, 23d, . . . in each pixel block corresponding to F8.0, , the control means can generate an image corresponding to the desired F number of F8.0. Further, as shown in FIG. 5B, signals obtained from nine pixels 17a to 29a, 17b to 29b, 17c to 29c, 17d to 29d, . . . in each pixel block corresponding to F2.8 are used. Thus, the control means can generate an image corresponding to the desired F value of F2.8.

As described above, even if the diaphragm is not set by the diaphragm mechanism, by performing pupil division on the image sensor 101 and performing image processing on the read pixel signals for each pixel block, a desired F value can be obtained. image can be generated. That is, according to the above configuration, the control means can generate an image corresponding to a desired F number larger than the F number based on an image acquired at a certain F number (for example, an open F number). can. The above configuration is particularly useful in an imaging apparatus having an optical system that does not have a diaphragm mechanism (for example, the imaging apparatus 1 having the photographing lens 600 of the catadioptric optical system described above). It can also be applied to an imaging device having

Note that depending on the desired F-number (for example, F4.0 and F5.6 in the example of FIG. 4), the exit pupil area corresponding to the F-number and the pixel area corresponding to the exit pupil area are significantly different. There are cases (for example, when the area ratio of the exit pupil area to the pixel area exceeds a predetermined threshold range). In the above case, it is preferable that the control means executes an interpolation calculation to generate an image corresponding to the desired F-number. The above interpolation calculation is executed based on, for example, the ratio between the area of the exit pupil area and the area of the pixel area corresponding to the exit pupil area, the area of the exit pupil area and the signal ratio of the imaging device 101, and the like. can be done.

The control means may selectively read out the pixels in the image sensor 101 corresponding to the exit pupil region corresponding to the desired F-number to obtain the above pixel signals and generate an image of the desired F-number. Further, after recording all pixel signals read out from all pixels in the image sensor 101 in the recording unit 106, the control means extracts pixel signals from pixels corresponding to the exit pupil region corresponding to the desired F value. An image of any desired F number may be generated. Furthermore, an external device such as a PC other than the imaging device 1 may generate a desired F-number image by executing similar processing using the pixel signals recorded in the recording unit 106 .

In addition, when the normal photographing lens 500 is used in the image pickup apparatus 1, the diaphragm drive unit 506 drives the diaphragm 505, and the control unit acquires an image with a desired F number. In the above case, since the diaphragm 505 is narrowed down so as to correspond to the desired F-number, the control means should always read pixel signals from the pixels corresponding to all the exit pupil regions 11 to 35 in each pixel block. . Alternatively, the control unit may control the diaphragm drive unit 506 so as to fix the diaphragm 505 in an open state, and then acquire the pixel signal corresponding to the desired F-number as described above.

By the way, as described above, the catadioptric optical system of the imaging lens 600 of this embodiment is provided with the light shielding portion (secondary mirror) 601D in the light receiving area (incidence area to the lens) of the luminous flux. Therefore, as shown in FIG. 6, there is an unreceivable area UR on the exit pupil plane, which is caused by the light shielding portion 601D blocking part of the light flux. The size of the unreceivable region UR is determined according to the size of the light blocking portion 601D in the optical path. As shown in the figure, the unreceivable region UR is located near the center of the luminous flux. Therefore, when the subject image is not formed on the imaging surface of the image sensor 101 (when the luminous flux does not converge), a ring-shaped bokeh (ring blur) occurs.

In the example of FIG. 6, the unreceivable area UR corresponds to the exit pupil area (area of the pixel 23) corresponding to F8.0. Therefore, the central pixel 23 (pixels 23a, 23b, 23c, 23d, . . . shown in FIG. 5A) in the pixel block is included in the non-light receiving area UR. As a result, even if the pixels 23a, 23b, 23c, 23d, . Therefore, in the present embodiment, an image corresponding to a desired F-number is acquired by a method described below with reference to FIG. 7 when there is a non-receivable region UR.

FIG. 7(a) shows a case of acquiring an image of F8.0 when there is a non-receivable area UR. The control means acquires pixel signals from pixels 22a, 22b, 22c, 22d, . .0 equivalent image is generated. The exit pupil area 22 is an exit pupil area at least part of which is not the unreceivable area UR. In place of the exit pupil region 22, pixels corresponding to any one of the exit pupil regions 17, 18, 19, 24, 27, 28, and 29 adjacent to the exit pupil region 23 are instructed to the control means as the pixel signal acquisition target. may be selected. As described above, the control means shifts the positions of pixels from which pixel signals are to be obtained when generating an image, thereby avoiding the unreceivable region UR and obtaining an image having a shooting depth equivalent to the desired F-number. can be obtained.

As described above, when an image is generated by selecting pixels corresponding to an exit pupil area different from the exit pupil area corresponding to the desired F-number, the center of gravity of the exit pupil area deviates from the center of the optical axis. Therefore, parallax occurs in the generated image compared to the case where image generation is performed based on the pixels 23a, 23b, 23c, 23d, . However, the amount of parallax in the generated image can be reduced by the control means selecting an exit pupil region adjacent to the exit pupil region corresponding to the desired F-number.

FIGS. 7(b) and 7(c) show a case of acquiring an image of F2.8 when there is an unreceivable area UR. When the desired F-number is F2.8, only the exit pupil region 23 is the unreceivable region UR among the exit pupil regions 17 to 19, 22 to 24, and 27 to 29 corresponding to F2.8. Therefore, as shown in FIG. 5B, based on signals obtained from nine pixels 17a-29a, 17b-29b, 17c-29c, 17d-29d, . . . in each pixel block corresponding to F2.8, and the control means may acquire the image.

However, in the image generated as shown in FIG. 7(b), the above-described ring blur occurs remarkably. Therefore, as shown in FIG. 7(c), it is preferable for the control means to shift the position of the pixel from which the pixel signal is to be obtained to an adjacent pixel, thereby generating an image with a desired F value of F2.8. be. In the image generated as shown in FIG. 7(c), the occurrence of ring blur with missing center portion is avoided, and an inverted C-shaped blur with missing horizontal end portion occurs.

According to the above configuration, in the imaging apparatus 1 to which the imaging lens 600 having the catadioptric optical system requiring the light shielding portion 601D can be mounted, the desired F value can be obtained while suppressing the influence of the light unreceivable region UR caused by the light shielding portion 601D. can acquire an image corresponding to More specifically, by shifting the position of the pixel from which the pixel signal is to be obtained according to the position and size of the unreceivable area UR generated on the exit pupil plane by the light shielding portion 601D, the pixel It is possible to suppress the influence of pixels from which signals cannot be obtained.

In the embodiment of the present invention described above, the imaging device 1 is a lens-interchangeable digital camera. However, the imaging device 1 may be formed integrally with a photographing lens (for example, a photographing lens 600 having a catadioptric system).

Although preferred embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications and changes are possible within the scope of the gist thereof. For example, the present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors of a computer of the system or device executes the program. It is also possible to implement the process of reading and executing the The invention can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

REFERENCE SIGNS LIST 1 imaging device 10 microlens array 20 microlens 100 camera body 101 imaging device 500 photographing lens 600 photographing lens 601D light shielding portion 605 exit pupil region UR non-receivable region

Claims (10)

An imaging device capable of imaging with a photographic lens having a catadioptric optical system with a constant photographic F-number without an aperture mechanism,
an imaging device having a plurality of pixel blocks each including a plurality of pixels arranged two-dimensionally, and a plurality of microlenses respectively corresponding to the plurality of pixel blocks;
a control means for acquiring an image corresponding to the desired F-number using pixel signals output from the pixels in the pixel block corresponding to the exit pupil area of the microlens corresponding to the desired F-number; with
The catadioptric optical system has a light blocking part that blocks a part of the incident light beam to create a non-receivable region in the exit pupil region,
The control means uses the pixel signals output from the pixels corresponding to an exit pupil area different from the exit pupil area corresponding to the desired F number to reproduce the image corresponding to the desired F number. An imaging device characterized by : 2. The imaging apparatus according to claim 1 , wherein said control means deviates the center of gravity of said exit pupil area corresponding to said desired F number from the optical axis of said catadioptric system. The control means obtains the image corresponding to the desired F value by interpolating the pixel signals based on the area of the exit pupil region and the area of the pixel region corresponding to the exit pupil region. 3. The image pickup apparatus according to claim 1 , characterized by: 4. The imaging apparatus according to claim 1 , wherein said control means obtains said image by selectively reading said pixels corresponding to said exit pupil area. The control means acquires the image by extracting the pixel signals output by the pixels corresponding to the exit pupil region from all pixel signals read out from all pixels in the imaging device. The imaging device according to any one of claims 1 to 3 . an operation unit used to input the desired F value;
6. The imaging apparatus according to any one of claims 1 to 5 , further comprising a display section for displaying the desired F-number received by the operation section. 7. The imaging apparatus according to any one of claims 1 to 6 , wherein the photographing lens having the catadioptric system is detachable. 7. The imaging apparatus according to any one of claims 1 to 6 , wherein the imaging apparatus is configured integrally with the photographing lens having the catadioptric optical system. a photographing lens having a catadioptric optical system with no diaphragm mechanism and a constant photographing F-number;
Acquired by imaging using an imaging device having an imaging element having a plurality of pixel blocks each including a plurality of pixels arranged two-dimensionally and a plurality of microlenses respectively corresponding to the plurality of pixel blocks An image processing method for processing a pixel signal obtained by
obtaining an image corresponding to the desired F-number using the pixel signals output from the pixels in the pixel block corresponding to the exit pupil region of the microlens corresponding to the desired F-number ;
The catadioptric optical system has a light blocking part that blocks a part of the incident light beam to create a non-receivable region in the exit pupil region,
obtaining the image corresponding to the desired F-number using the pixel signals output from the pixels corresponding to an exit pupil area different from the exit pupil area corresponding to the desired F-number; An image processing method characterized by: A program that causes a computer to execute the image processing method according to claim 9 .

Images