JP2021082911A - Image processing device, image processing method, and program - Google Patents

Abstract

To provide an image processing device, an image processing method, and a program that can achieve correction of color shading that occurs when the position of an image pickup device is moved relative to an imaging optical system without storing a large amount of correction data.SOLUTION: An image processing unit 110 in an image processing device obtains an angle of incidence of a light ray from an imaging optical system 104 to each pixel of an image pickup device 103 when the image pickup device 103 is moved in a direction orthogonal to the optical axis of the imaging optical system 104 and shake correction of an image is performed. The image processing unit 110 corrects the color shading of each pixel of the image pickup device 103 based on a relationship between the angle of incidence of the light ray to the image pickup device 103 and the color ratio of the image pickup device 103.SELECTED DRAWING: Figure 1

Description

The present invention relates to a technique for processing a captured image.

In the image pickup apparatus, color unevenness of the captured image called color shading may occur due to the relationship between the angle of incidence of light rays from the imaging optical system and the shrinkage of the microlens of the image pickup element. In the imaging device, when the optical axis of the photographing optical system and the center of the imaging element coincide with each other, the angle of incidence of light rays from the imaging optical system on the imaging element is rotationally symmetric with respect to the shrink of the microlens of the imaging element. .. Therefore, for color shading when the optical axis of the photographing optical system and the center of the image sensor are aligned, it is sufficient to perform color shading correction with one-dimensional correction data from the optical axis in the image height direction.

However, in an imaging device, when the position of the image sensor is moved relative to the imaging optical system to correct image blur, the angle of light incident from the imaging optical system is asymmetric with respect to the shrink of the microlens. Become. Therefore, when the position of the image sensor is moved relative to the photographing optical system, it is not possible to perform color shading correction with one-dimensional correction data in the image height direction from the optical axis. Note that Patent Document 1 provides a technique for suppressing image deterioration by correcting asymmetric color shading that occurs when the image sensor is shifted to correct blur by switching a plurality of color shading correction data. It is disclosed.

Japanese Unexamined Patent Publication No. 2006-1216113

However, in the case of the technique disclosed in Patent Document 1 above, it is necessary to store a large number of color shading correction data corresponding to each of a plurality of image sensor positions when the image sensor is shifted with respect to the optical axis. Must be. Therefore, the memory capacity for storing the color shading correction data becomes large. Further, since the color shading correction is performed by switching a plurality of color shading correction data, the color change of the image may be conspicuous when the color shading correction data is switched at the time of moving image shooting or the like.

Therefore, an object of the present invention is to make it possible to realize color shading correction that does not require a large amount of correction data and does not cause a change in the color of an image.

In the image processing apparatus of the present invention, when the image sensor is moved in a direction orthogonal to the optical axis of the image sensor to correct image blurring, each pixel of the image sensor is subjected to the image sensor from the image sensor. From the acquisition means for acquiring the incident angle of light rays, the relationship between the incident angle of light rays with respect to the image sensor and the color ratio of the image sensor, and the image pickup optical system for each pixel of the image sensor acquired by the acquisition means. A generation means for generating correction data for correcting the color shading of each pixel of the image sensor based on the incident angle of the light beam, and moving the image sensor in a direction orthogonal to the optical axis of the image sensor. It is characterized by having a correction means for performing color shading correction based on the correction data on the captured image when the blur correction of the image is performed.

According to the present invention, it is possible to realize color shading correction that does not require a large amount of correction data and does not cause a change in the color of an image.

It is a block diagram which shows the functional structure example of the camera which concerns on embodiment. It is explanatory drawing of the light ray incident angle change to an image sensor at the time of blur correction. It is a figure which shows the relationship between the light ray incident angle to an image sensor, and a color ratio. It is a flowchart related to color shading correction data creation.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
In the present embodiment, an image processing device (digital camera) with an interchangeable lens will be described as an application example of the image processing device. The digital camera according to the present embodiment is not limited to the digital camera with interchangeable lenses, but also includes a digital camera with an integrated lens, a video camera, an information terminal having a camera function, a surveillance camera, an in-vehicle camera, and the like. Applicable. In the following description, the digital camera of this embodiment is simply referred to as a camera. The camera of the present embodiment has a function of moving the image pickup element in a direction orthogonal to the optical axis of the image pickup optical system to correct blurring and the like during shooting.

FIG. 1 is a block diagram showing an example of a functional configuration of a camera according to the present embodiment.
The operation unit 101 is composed of switches, buttons, a touch panel, and the like that the operator operates to input various instructions to the camera. The operation unit 101 also includes a shutter switch, and when the shutter switch is operated in a so-called half-pressed state, the signal SW1 is notified from the operation unit 101 to the control unit 102. Further, when the shutter switch is operated in the so-called fully pressed state, the signal SW2 is notified from the operation unit 101 to the control unit 102.

The imaging optical system 104 includes an imaging lens including a focusing lens, a zoom lens, and the like, and an exposure mechanism 105 or the like using an aperture or the like. Further, the imaging optical system 104 is attached to and detached from the housing of the camera by a mount portion (not shown). When mounted on the mount of the camera, the image pickup optical system 104 is arranged in front of the image pickup element 103, which will be described later. The imaging optical system 104 is capable of transmitting information such as the focal length of the imaging lens, the aperture value of the exposure mechanism 105, and the focus position of the focusing lens to the control unit 102, which will be described later.

The data holding unit 107 holds various data related to the imaging optical system 104 including the light incident angle data incident on the imaging optical system 104 from the imaging optical system 104. In addition to the light incident angle data, the various data related to the imaging optical system 104 held in the data holding unit 107 are focused by the focal length, aperture value, and focusing lens of the imaging optical system 104. Information indicating the focus position and the like are also included. When the center of the image pickup optical system 104 and the center of the image pickup surface of the image sensor unit 103 coincide with each other, the light incident angle data held by the data holding unit 107 receives light from the image pickup optical system 104. It is data which shows the incident angle for each image height at the time of incident on 103. The data holding unit 107 holds the light ray incident angle data corresponding to the focal length, the aperture value, and the focus position in the imaging optical system 104. When the imaging optical system 104 is mounted on the mount unit of the camera, the data holding unit 107 outputs various data related to the imaging optical system to the control unit 102. The data holding unit 107 may be provided in the imaging optical system 104 in the case of the interchangeable lens type, or may be provided in the camera body. When the camera body includes the data holding unit 107, the data holding unit 107 holds various data related to the imaging optical system for each photographing lens that can be attached to and detached from the camera.

The image sensor unit 103 receives a light image incident on the image pickup surface via the image pickup optical system 104 and outputs an electric charge according to the amount of the light. Further, in the present embodiment, the image sensor unit 103 can be optically corrected for camera shake by being moved in a direction orthogonal to the optical axis of the image pickup optical system 104 by the image sensor drive unit 108 described later. It also includes a blur correction mechanism.

The blur detection unit 106 detects the movement of the camera due to blur and the like, and outputs the blur detection result to the control unit 102.

The control unit 102 controls the operation of each unit of the camera of the present embodiment. The control unit 102 controls each unit by executing a program stored in a memory (not shown) or the like, and also controls each unit in response to an instruction from the operation unit 101.
Further, the control unit 102 is for correcting the blur of the camera based on the blur detection result of the camera detected by the blur detection unit 106 and the data related to the imaging optical system sent from the data holding unit 107. Calculate the blur correction signal. The blur correction signal is sent to the image sensor driving unit 108.

The image sensor driving unit 108 drives the blur correction mechanism of the image sensor unit 103 based on the blur correction signal from the control unit 102, so that the image sensor unit 103 is orthogonal to the optical axis of the image pickup optical system 104. Move in the direction you want. Therefore, the blur correction signal output by the control unit 102 includes at least a drive direction and a drive amount with respect to the blur correction mechanism when the image sensor unit 103 is moved in a direction orthogonal to the optical axis of the image pickup optical system 104. It is a signal. In this way, the image sensor driving unit 108 moves the image sensor unit 103 in the direction orthogonal to the optical axis of the image pickup optical system 104 based on the blur correction signal generated by the control unit 102, so that the image sensor unit 103 Will output an image sensor with the camera shake corrected.

The A / D conversion unit 109 performs sampling, gain adjustment, A / D conversion, and the like on the analog image pickup signal output from the image pickup element unit 103, and outputs the analog image pickup signal as a digital image signal.

The image processing unit 110 performs various image processing on the digital image signal output from the A / D conversion unit 109, and outputs the digital image signal after the image processing. For example, the image processing unit 110 performs image processing such as white balance on the digital image signal received from the A / D conversion unit 109 and outputs the digital image signal.

Further, the image processing unit 110 holds data showing the relationship between the angle of light incident on the image sensor unit 103 and the color ratio of each pixel of the image sensor unit 103. Hereinafter, the data showing the relationship between the incident angle of the light beam on the image sensor unit and the color ratio of each pixel of the image sensor unit, which is held in the image processing unit 110, will be referred to as incident angle color ratio correspondence data.
Here, when blur correction is performed in which the image sensor unit 103 is moved in a direction orthogonal to the optical axis of the image pickup optical system 104, the control unit 102 obtains data on the drive direction and drive amount of the blur correction mechanism in the blur correction signal. It is sent to the image processing unit 110. Further, the control unit 102 acquires the light ray incident angle data from the data holding unit 107 and sends the light ray incident angle data to the image processing unit 110.

The image processing unit 110 is based on the data of the driving direction and the driving amount of the blur correction signal, the data corresponding to the incident angle color ratio held, and the light beam incident angle data sent from the control unit 102. Correction data for correcting the color shading of each pixel of 103 is generated. A detailed description of the color shading that occurs when the camera shake correction is performed and the correction thereof will be described later.

At this time, the image processing unit 110 first determines the image sensor unit at the time of blur correction based on the light ray incident angle data sent from the control unit 102 and the drive direction and drive amount of the blur correction mechanism in the blur correction signal. The light beam incident angle for each pixel of 103 is calculated. Hereinafter, the calculated light beam incident angle will be referred to as a calculated incident angle.

Subsequently, the image processing unit 110 calculates the color ratio of each pixel of the image sensor unit 103 with respect to the calculated incident angle based on the calculated incident angle and the held incident angle color ratio corresponding data.
Further, the image processing unit 110 calculates correction data for color shading correction for each pixel of the image sensor unit 103 based on the calculated color ratio.

Then, the image processing unit 110 refers to the correction data for the captured image when the image sensor driving unit 108 moves the image sensor unit 103 in the orthogonal direction with respect to the optical axis of the image pickup optical system 104 to perform blur correction. Performs color shading correction based on.

The EVF display unit 111 is composed of a small liquid crystal screen or the like, and displays an image according to the image data processed by the image processing unit 110.
The format conversion unit 112 converts the format of the digital image signal (image data) output from the image processing unit 110 into a format such as JPEG and outputs it to the image recording unit 113.
The image recording unit 113 performs a process of recording the image data after the format conversion by the format conversion unit 112 in a memory (not shown) in the camera, an external memory inserted in the camera, or the like.
The external connection unit 114 functions as an interface for connecting the camera to an external device such as a PC (personal computer) or a printer.

Hereinafter, the operation of the camera when imaging is performed using the camera of the present embodiment will be described.
First, when the operator turns on the power switch included in the operation unit 101, the control unit 102 detects this and supplies power to each unit of the camera. Since the shutter opens when power is supplied to each part constituting the camera, light enters the image sensor unit 103 via the image pickup optical system 104 and the exposure mechanism 105. Then, the image sensor unit 103 reads out the accumulated electric charge and outputs it to the A / D conversion unit 109 as an analog image signal.

Further, the control unit 102 uses the image sensor driving unit 108 for blur correction based on the blur detection result of the camera detected by the blur detection unit 106 and the data related to the imaging optical system acquired from the data holding unit 107. The blur correction signal of is calculated.
Further, the control unit 102 sends the blur correction signal and the data related to the imaging optical system acquired from the data holding unit 107 to the image processing unit 110, and controls the image processing unit 110 so that image processing such as color shading correction can be performed. ..

At this time, the image processing unit 110 acquires the calculated incident angle from the information on the driving direction and the driving amount included in the blur correction signal and the light ray incident angle data included in the data related to the image pickup optical system, that is, the image pickup element unit 103. The angle of incidence of light rays on each pixel is calculated. Further, the image processing unit 110 generates correction data for correcting the color shading of each pixel based on the calculated incident angle and the held incident angle color ratio corresponding data.

Then, the image processing unit 110 performs various image processing such as white balance and color shading correction on the digital image signal output from the A / D conversion unit 109, and outputs the processed digital image signal. In the color shading correction, the color shading correction is performed using the latest drive amount and drive direction information in the blur correction signal to the image sensor drive unit 108. Further, since the image processing unit 110 outputs the processed image data to the EVF display unit 111, the image according to the processed image data is displayed on the EVF display unit 111.

The control unit 102 repeats the above-described processing unless the shutter switch notifies the signal SW1 (that is, the notification of half-pressing of the shutter switch). On the other hand, when the control unit 102 receives the notification of the signal SW1 from the shutter switch, the control unit 102 performs AE (automatic exposure) processing using the image at that time to acquire the optimum exposure setting condition for shooting.

Further, the control unit 102 repeats the above-described processing after the notification of the signal SW1 unless the notification of the signal SW2 (that is, the notification of the full press of the shutter switch) is received from the shutter switch. After that, when the control unit 102 receives the notification of the signal SW2 from the shutter switch, the control unit 102 controls each unit so that the main shooting is performed by the camera. In the case of the main shooting, the light from the outside world enters the image sensor unit 103 via the image pickup optical system 104 and the exposure mechanism 105. Therefore, the photoelectric conversion element constituting the image pickup element unit 103 has an amount of the incoming light. Corresponding charge is accumulated.

Then, the electric charge accumulated in the image sensor unit 103 is read out and sent to the A / D conversion unit 109 as an analog image signal. The A / D conversion unit 109 performs sampling, gain adjustment, A / D conversion, and the like on the analog image signal output from the image sensor unit 103, and outputs the analog image signal as a digital image signal.

The image processing unit 110 performs various image processing such as white balance and color shading correction on the digital image signal output from the A / D conversion unit 109 according to the shooting conditions, and produces the processed digital image signal. Output.
The format conversion unit 112 converts the format of the digital image signal (image data) output from the image processing unit 110 into a format such as JPEG, and outputs the format to the image recording unit 113. The image recording unit 113 performs a process of recording the format-converted image data in a predetermined memory.

By the operation described above, according to the present embodiment, a digital camera capable of acquiring image data in which color shading correction corresponding to a change in the incident angle of light rays on the image sensor unit 103 due to blur correction has been performed is realized. To.

Hereinafter, the cause of color shading occurring asymmetrically with respect to the screen due to camera shake correction and the color shading correction will be described.

2 (a) to 2 (e) show the case where the pixel structure of the image sensor unit 103 and the image sensor unit 103 are moved by driving the image sensor drive unit 108 to correct the camera shake. It is a figure which showed the incident angle of the light beam to a pixel.
FIG. 2A is a diagram schematically showing the structure of one pixel 207 of the image sensor unit 103. As shown in FIG. 2A, a microlens 201 that collects the incident light on the pixel 207 is arranged above the pixel 207 (on the imaging optical system side). Further, a color filter 202 that determines the color characteristics of the pixel 207 is arranged below the microlens 201, and a photodiode 203 that photoelectrically converts incident light is arranged below the color filter 202. The electrical signal photoelectrically converted by the photodiode 203 is sent to the A / D conversion unit 109 by a wiring (not shown) arranged below the photodiode 203.

Here, the center 204 of the pixel 207 and the center 205 of the microlens 201 coincide with each other at the center of the image sensor 103, but move away as a shrink 206 toward the periphery of the image sensor 103. In this way, the light incident angle with respect to the pixels on the image sensor 103 differs depending on the arrangement location (that is, the image height) of the pixels on the image sensor 103, so that a color shift called color shading depending on the light incident angle occurs. ..

2 (b) and 2 (c) show the positional relationship between the image circle 210 of the image pickup optical system 104 and the image sensor unit 103 when the camera shake correction is not performed, and the pixels of FIG. 2 (a). It is a figure which showed the light ray incident angle to a structure. FIG. 2B is a view of the image circle 210 of the image pickup optical system 104 and the image sensor unit 103 as viewed from the image pickup optical system 104 side. FIG. 2C shows the light incident angle indicated by the arrow with respect to the center a of the image sensor unit 103, and the light incident incident with the arrows at the pixel b portion and the pixel c portion equidistant from the center a of the image sensor unit 103. It is a figure showing each corner. When the blur correction is not performed, since the center of the image circle 210 and the center a of the image sensor unit 103 coincide with each other, the light beam is vertically incident at the center a of the image sensor unit 103. Further, the light incident angles with respect to the pixel b portion and the pixel c portion of the peripheral portions equidistant from the center a of the image sensor unit 103 are equal to each other. When the blur correction is not performed, the light incident angles with respect to the pixel b portion and the pixel c portion equidistant from the center a are equal, so that the color shading correction data goes from the center a of the image sensor unit 103 to the peripheral portion. Therefore, the correction data that changes one-dimensionally may be applied.

2 (d) and 2 (e) show the image circle of the image pickup optical system 104, the positional relationship of the image sensor unit 103, and the pixels of FIG. 2 (a) when blur correction is performed by the camera. It is a figure which showed the light ray incident angle to a structure. FIG. 2D is a view of the image circle 210 of the image pickup optical system 104 and the image sensor unit 103 as viewed from the image pickup optical system 104 side. FIG. 2E shows the light ray incident angle indicated by the arrow with respect to the center a of the image sensor unit 103 and the light ray incident angle indicated by the arrow with respect to the pixel b portion and the pixel c portion equidistant from the center a, respectively. It is a figure that was made. When blur correction is performed, the center of the image circle 210 and the center a of the image sensor unit 103 may not match. In this way, when the center of the image circle 210 and the center a of the image sensor unit 103 do not coincide with each other, the light beam does not enter vertically at the center a of the image sensor unit 103 and is equidistant from the center a. The light incident angles with respect to the pixel b portion and the pixel c portion are not equal. Therefore, when blur correction is performed, in order to perform color shading correction, the color ratio of the image sensor unit 103 to the light ray incident angle of each pixel of the image sensor unit 103 and the light ray incident angle of each pixel is set. It is necessary to obtain and calculate the color shading correction data for each pixel.

3 (a) to 3 (c) are views showing the color ratio of the image sensor unit 103 between the central portion, the outermost peripheral portion, and the intermediate portion of the image sensor unit 103, and FIG. 3 (a) shows the color ratio of the image sensor unit 103. Is the central portion, FIG. 3 (b) is the outermost portion, and FIG. 3 (c) is the intermediate portion. The color ratio is the ratio of the R (red) signal and the B (blue) signal to the G (green) signal of the image sensor unit 103.

The image processing unit 110 has at least the relationship between the light incident angle and the color ratio of the central portion of the image sensor unit 103 shown in FIG. 3 (a) and the outermost periphery of the image sensor unit 103 shown in FIG. 3 (b). The data showing the relationship between the light ray incident angle and the color ratio of the part is held as the incident angle color ratio corresponding data. That is, it is assumed that the image processing unit 110 does not hold the data corresponding to the incident angle color ratio in the intermediate portion of the image sensor unit 103 shown in FIG. 3C. In this case, the image processing unit 110 interpolates the relationship between the light ray incident angle and the color ratio in the intermediate portion of the image sensor unit 103 shown in FIG. 3C based on the incident angle color ratio corresponding data held. Obtained by. That is, the image processing unit 110 is the distance from the central portion to the intermediate portion and from the outermost peripheral portion to the intermediate portion of the image sensor unit 103 based on the incident angle color ratio correspondence data between the central portion and the outermost peripheral portion of the image sensor unit 103. The relationship between the light incident angle and the color ratio of the intermediate portion of the image sensor unit 103 is obtained by interpolation according to the above. Then, the image processing unit 110 sends the image processing unit 110 to each pixel of the image sensor unit 103 so that the output of the G signal and the output of the R signal and the B signal are the same when the same amount of light is incident on each pixel of the image sensor unit 103. The gain is calculated to generate correction data that corrects the color shading.

FIG. 4 is a flowchart in which the image processing unit 110 generates correction data for correcting color shading.
Prior to the processing of FIG. 4, the control unit 102 determines the blur correction signal (blurring correction) of the image sensor driving unit 108 from the camera blur detection result by the blur detection unit 106 and the data related to the imaging optical system from the data holding unit 107. A signal indicating the drive amount and drive direction of the mechanism) is calculated.

Then, in step S401, the image processing unit 110 determines the distance of each pixel of the image sensor unit 103 from the center of the image circle of the image pickup optical system 104 based on the drive amount and drive direction of the blur correction mechanism in the blur correction signal. calculate.
Further, in step S402, the image processing unit 110 is based on the distance from the center of the image circle and the light incident angle data of the image pickup optical system 104 output from the data holding unit 107 and sent from the control unit 102. The angle of incidence of light rays on each pixel of 103 is calculated.

Further, in step S403, the image processing unit 110 calculates the distance from the center of the image sensor for each pixel of the image sensor unit 103.
Further, in step S404, the image processing unit 110 interpolates the color ratio corresponding to the light ray incident angle at each pixel position of the image sensor unit 103 from the incident angle color ratio corresponding data corresponding to each pixel position held. Calculated by

Next, in step S405, the image processing unit 110 determines each pixel position (that is, image height) of the image sensor unit 103 from the light ray incident angle (calculated incident angle) calculated in step S402 and the color ratio calculated in step S404. Calculate the color ratio to the angle of incidence of light rays on.

Next, in step S406, the image processing unit 110 outputs a G signal and an R signal and a B signal when the same amount of light is incident on each pixel from the color ratio with respect to the incident angle of light rays at each pixel position calculated in step S405. Calculate the gain for each pixel so that the output of is the same. Then, the image processing unit 110 generates color shading correction data based on the gain for each of these pixels.

After that, in step S407, the image processing unit 110 performs color shading correction processing on the captured image using the color shading correction data generated in step S406. According to the present embodiment, by performing these processes for each captured image, it is possible to obtain an image with little change in color shading even if blur correction is performed.

In the present embodiment, in a camera having a mechanism for correcting blur during shooting by moving the image sensor unit 103 in a direction orthogonal to the optical axis of the image pickup optical system 104, light rays are incident on the image sensor unit 103 during blur correction. Color shading correction data corresponding to angle changes can be created at any time. Therefore, according to the present embodiment, it is possible to always obtain an image in which color shading is suppressed. That is, according to the present embodiment, since a large amount of color shading correction data is not required and the color shading correction data is not switched at the time of moving image shooting or the like, the color of the image does not change.

The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.
The above-described embodiments are merely examples of embodiment in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

102: Control unit, 103: Image sensor unit, 104: Image pickup optical system, 106: Blur detection unit, 107: Data holding unit, 108: Image sensor drive unit, 110: Image processing unit

Claims (10)

Acquires the angle of incidence of light rays from the image pickup optical system on each pixel of the image pickup element when the image sensor is moved in a direction orthogonal to the optical axis of the image pickup optical system to correct image blurring. Acquisition method and
Based on the relationship between the light ray incident angle on the image sensor and the color ratio of the image sensor, and the incident angle of the light ray from the image pickup optical system on each pixel of the image sensor acquired by the acquisition means. A generation means for generating correction data for correcting the color shading of each pixel of the image sensor, and
A correction means for performing color shading correction based on the correction data on the captured image when the image blur correction is performed by moving the image sensor in a direction orthogonal to the optical axis of the image pickup optical system. ,
An image processing device characterized by having. It has a holding means for holding information indicating the relationship between the light ray incident angle with respect to the image pickup element and the color ratio of the image pickup element.
The image processing apparatus according to claim 1, wherein the generation means generates the correction data by using the information held by the holding means. The holding means holds information indicating the relationship between the light ray incident angle and the color ratio at at least two or more image heights of the central portion and the outermost peripheral portion of the image pickup device. The image processing apparatus according to claim 2. The generation means provides information indicating the relationship between the light ray incident angle not held by the holding means and the color ratio, and the relationship between the light ray incident angle held by the holding means and the color ratio. The image processing apparatus according to claim 2 or 3, wherein the image processing apparatus is calculated by interpolation using. The acquisition means includes light incident angle data from the image pickup optical system on each pixel of the image pickup element when the optical axis of the image pickup optical system coincides with the center of the image pickup element, and light from the image pickup optical system. The imaging optics for each pixel of the image sensor based on the direction and amount of movement of the image sensor when the image sensor is moved in a direction orthogonal to the axis to correct image blur. The image processing apparatus according to any one of claims 1 to 4, wherein the angle of incidence of the light beam from the system is calculated. The acquisition means acquires the incident angle of the light beam from the image pickup optical system to the image pickup element according to the focal length of the image pickup optical system, the aperture value of the image pickup optical system, and the focus position of the image pickup optical system. The image processing apparatus according to any one of claims 1 to 5, wherein the image processing apparatus is used. The generation means generates the correction data of the color shading for each pixel of the image sensor, and generates the correction data.
The image processing apparatus according to any one of claims 1 to 6, wherein the correction means performs color shading correction of each pixel of the image pickup device for each captured image. Imaging optics and
An image sensor that captures an image formed by the imaging optical system,
A blur correction mechanism that corrects the blur of the image by moving the image sensor in a direction orthogonal to the optical axis of the imaging optical system.
The image processing apparatus according to any one of claims 1 to 7.
An imaging device characterized by having. An image processing method executed by an image processing device.
Acquires the angle of incidence of light rays from the image pickup optical system on each pixel of the image pickup element when the image sensor is moved in a direction orthogonal to the optical axis of the image pickup optical system to correct image blurring. Acquisition process and
Based on the relationship between the light ray incident angle on the image sensor and the color ratio of the image sensor, and the incident angle of the light ray from the image pickup optical system on each pixel of the image sensor acquired by the acquisition step. A generation step of generating correction data for correcting the color shading of each pixel of the image sensor, and
A correction step of performing color shading correction based on the correction data on the captured image when the image blur correction is performed by moving the image sensor in a direction orthogonal to the optical axis of the image pickup optical system. ,
An image processing method characterized by having. A program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 7.

Images