JP2023004680A - Image processing device and control method for image processing device - Google Patents

Abstract

To address a probability that, when the state of a lens, a barrel, or a sensor is changed due to a temperature change or posture change of an imaging device, optical characteristics such as a field curvature and a distortion aberration are changed to generate a ranging error in an overall screen.SOLUTION: An image processing device includes generation means for generating distance information using a plurality of image signals obtained from different view points, detection means for detecting a flat region in the image signals, and estimation means for estimating a correction map for correcting the distance information. The estimation means expresses the correction map by using a function in which position coordinates of an image are variables, and estimates the correction map on the basis of the distance information corresponding to the flat region.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing apparatus and a control method thereof for an imaging apparatus such as a digital camera, a digital video camera, an in-vehicle sensor device, a robot vision sensor device, etc., having a distance measurement function using an image.

An imaging device has been proposed that has a distance measurement function capable of detecting distance information such as the defocus state of a subject and the distance from the imaging device to the subject based on image signals captured from different viewpoints (Patent Document 1).

WO2010/010707

In imaging devices with such a range finding function, optical characteristics such as curvature of field and distortion change due to changes in the state of lenses, barrels, and sensors due to changes in the temperature and posture of the imaging device. Distance measurement errors may occur across the screen.

In view of the above problems, an image processing apparatus according to the present invention provides a generation means for generating distance information using a plurality of image signals captured from different viewpoints, a detection means for detecting a plane area in the image signals, and the estimating means for estimating a correction map for correcting the distance information, wherein the estimating means expresses the correction map as a function having the position coordinates of the image as variables, and calculates the correction map based on the distance information corresponding to the plane area. and estimating the correction map.

According to the present invention, it is possible to reduce distance measurement errors that occur in the entire screen, and to perform highly accurate distance measurement even if there is a change in the posture of the device or a change in temperature.

1 is a block diagram showing the configuration of an imaging device according to a first embodiment; FIG. FIG. 2 is a diagram explaining an optical system and an imaging unit according to the first embodiment; 1 is a block diagram showing the configuration of an image processing unit according to the first embodiment; FIG. FIG. 4 is a flow diagram for explaining the correction map estimation processing operation according to the first embodiment; An example of an input image according to the first embodiment Example of graph when distance information is generated according to the first embodiment An example of a graph showing a correction amount of curvature of field according to the first embodiment FIG. 4 is a diagram explaining an image used for estimating a correction map according to the first embodiment; A diagram showing the configuration of an imaging device according to a second embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings, but the present invention is not limited to the contents described in each example. The present invention can be combined with the configuration shown in each embodiment, and various modifications and changes can be made.

In the first embodiment, a use case of photographing a person in a room using an imaging apparatus having an image plane phase-difference ranging function will be described as an example. This embodiment can also be applied to an image processing apparatus capable of obtaining an image captured by an external imaging device and performing distance measurement calculation. The image processing device may include mobile phones including digital cameras and smartphones, game machines, tablet terminals, watch-type and eyeglass-type information terminals, medical equipment, equipment for monitoring systems and in-vehicle systems, and the like.

FIG. 1 is a block diagram showing a configuration diagram when an image processing apparatus to which the present invention can be applied is applied to an imaging apparatus 100. As shown in FIG. The imaging device 100 includes an optical system 101 , an imaging unit 102 , an A/D conversion unit 103 , a control unit 104 , an image processing unit 105 , a recording unit 106 , a volatile memory 107 and a nonvolatile memory 108 .

The optical system 101 includes a lens group including a zoom lens and a focus lens, an aperture adjustment device, and a shutter device. This optical system 101 adjusts the magnification, focus position, or amount of light of the subject image reaching the imaging unit 102 .

The imaging unit 102 is a photoelectric conversion element such as a CCD or CMOS sensor that photoelectrically converts the luminous flux of an object that has passed through the optical system 101 and converts the converted light into an electric signal.

A drive control unit may be provided for each of the optical system 101 and the imaging unit 102 . The drive control unit can drive the optical system and the imaging unit according to instructions from the control unit 104, which will be described later, and automatically adjust the focus position.

The A/D converter 103 converts the input analog image signal into digital image signal data.

Further, the imaging unit 102 may be an imaging surface phase difference sensor that includes an A/D conversion unit and outputs phase difference information or defocus information converted from the phase difference information to the control unit 104 described later.

The image processing unit 105 performs parallax calculation processing according to the present invention in addition to normal signal processing on data from the A/D conversion unit 103 and the like. Here, the normal processed signal refers to processing such as noise reduction processing, development processing, and gradation compression processing by gamma conversion to a predetermined output range.

A control unit 104 is a control unit including at least one processor or circuit, and controls the imaging apparatus 100 as a whole. For example, it includes a processor such as a CPU and an MPU, develops a program recorded in the nonvolatile memory 108 in the volatile memory 107, and executes it to control the operation of each block included in the imaging apparatus 100. FIG. For example, the amount of exposure at the time of shooting to obtain an input image with appropriate brightness is calculated, and in order to achieve this, the optical system 101 and imaging unit 102 are controlled, and the aperture, shutter speed, and analog gain of the sensor are calculated. to control.

The recording unit 106 has a function of recording an image, and may include, for example, an information recording medium using a package containing a rotating recording medium such as a memory card equipped with a semiconductor memory or a magneto-optical disk.

FIG. 2A shows the positional relationship between the optical system 101 and the imaging unit 102 in the imaging apparatus 100 of FIG. A dashed line 101a is the optical axis of the optical system 101, and the imaging unit 102 is arranged so as to be substantially perpendicular to the optical axis 101a.

FIG. 2B shows the pixel array configuration of the imaging unit 102 in FIG. Here, the optical axis 101a coincides with the z-axis direction. Each pixel 200 is composed of a microlens 201, a color filter 202, and photoelectric conversion units 203A and 203B, as shown in the cross-sectional view of FIG. 2(C). The imaging unit 102 is provided with red, green, and blue spectral characteristics according to the wavelength band detected by the color filter 202 for each pixel, and the color filters are arranged according to a known coloration pattern. Photoelectric conversion units 203A and 203B having sensitivities corresponding to wavelength bands are formed on the substrate 204 .

FIG. 2D is a view of the exit pupil viewed from the intersection of the optical axis 101a of the optical system 101 and the imaging unit 102. FIG. A light flux that has passed through the first pupil region 210 and a light flux that has mainly passed through the second pupil region 220, which are different regions of the exit pupil, enter the photoelectric conversion units 203A and 203B. By photoelectrically converting the light beams incident on the photoelectric conversion units 203A and 203B, an A image and a B image captured from different viewpoints are generated. The detected A image and B image are transmitted to the image processing unit 105 , distance information is calculated by distance measurement calculation processing, and stored in the recording unit 106 . Also, an image obtained by adding the A image and the B image can be used as image information.

Reference numeral 211 in FIG. 2D denotes the center-of-gravity position of the pupil region 210 , and reference numeral 221 denotes the center-of-gravity position of the pupil region 220 . In this embodiment, the center of gravity 211 of the pupil region 210 is decentered (moved) from the center of the exit pupil in the x-axis direction, and the center of gravity 221 of the pupil region 220 is decentered (moved) in the opposite direction. . The distance between the centers of gravity is the baseline length 222, and the line segment direction is called the pupil division direction. The positions of the A image and the B image change in the same direction as the pupil division direction. The amount of relative positional change between the images, that is, the parallax between the A image and the B image has a value corresponding to the defocus amount. This parallax can be detected and converted by a known method, and converted into distance information such as the defocus amount and the distance from the imaging device to the subject.

Next, the configuration of the image processing unit 105 according to this embodiment will be described with reference to FIG. Note that each block of the image processing unit 105 may be realized by a combination of software and hardware. Also, multiple functional blocks may be integrated or one functional block may be separated.

A signal processing unit 301 performs the above-described normal signal processing such as noise reduction processing and development processing. Note that the signal processing unit 301 may synthesize the signals of the A image and the B image and handle them as one image signal.

The shooting information acquisition unit 302 acquires various information such as the shooting mode set by the user at the time of shooting, the focal length, the aperture value, and the exposure time from the nonvolatile memory 108 or the volatile memory 107 via the control unit 104, for example. and provides it to the distance information generation unit 305 .

A correction information acquisition unit 303 acquires a variety of regular information that is not included in shooting settings, such as field curvature correction information and distortion aberration correction coefficients, in the nonvolatile memory 108 or volatile memory via the control unit 104, for example. 107 and provided to the distance information generation unit 305 .

The plane detection unit 304 detects the plane area of the subject based on the image signal output from the signal processing unit 301 .

A distance information generation unit 305 acquires the image signals of the A image and the B image, and generates distance information such as the defocus amount and the distance from the imaging device to the subject based on the parallax, shooting information, and correction information.

A correction map estimation unit 306 outputs a correction map for correcting distortion of distance information using the outputs of the plane detection unit 304 and the distance information generation unit 305 . The correction map is stored, for example, in the non-volatile memory 108 or volatile memory 107 via the control unit 104, and used to generate the next distance information.

A series of operations of the correction map estimation process according to the present embodiment will be described below with reference to FIGS. 4 and 5. FIG. FIG. 5 is a diagram of an example of an input image 501. FIG. An input image 501 is an image of a scene in which three persons 502, 503, and 504 are standing in a room.

In S<b>401 , the distance information generation unit 305 acquires the A image and the B image of the input image 501 .

In S402, the distance information generation unit 305 generates distance information based on the A image and the B image. Specifically, after calculating the parallax between the A image and the B image and converting the parallax into a defocus amount, the lens formula is used to convert the defocus amount into a distance to the object plane. Parallax calculation and defocus amount conversion may be performed by generating a defocus amount distribution for each pixel using, for example, a known method disclosed in Japanese Patent Application Laid-Open No. 2016-9062.

Here, since the distance conversion is performed in consideration of the defocus amount caused by the field curvature, the field curvature correction map W is used to correct the defocus amount. Assuming that the defocus amount before correction is L and the defocus amount after correction is L', L'=L+W is calculated. If the states of the optical system 101 and the imaging unit 102 do not change, the curvature of field does not change. Therefore, it is also possible to obtain the field curvature correction map W in advance by design data or measurement, and record it in the nonvolatile memory 107 or the like.

As described above, the distance from the imaging device to the subject can be calculated from the lens formula in geometric optics.

Ａ：物面から光学系１０１の主点までの距離
Ｂ：光学系１０１の主点から結像位置までの距離
Ｆ：光学系１０１の焦点距離

A: Distance from the object surface to the principal point of the optical system 101 B: Distance from the principal point of the optical system 101 to the imaging position F: Focal length of the optical system 101

In equation (1), the value of B is obtained from the distance from the principal point to the image sensor and the amount of defocus, and F is the design data or the value obtained by measurement, so that the distance A to the object plane is can be calculated for each Hereinafter, the distance from the imaging device to the object calculated in this manner is referred to as a distance measurement value. If the value of A in equation (1) is used, the reference of the distance measurement value will be the principal point of the optical system. good too.

In S403, the plane detection unit 304 detects a plane area from the subject appearing in the input image 501 based on the image signal. An existing technique may be used as the detection method. For example, image segmentation using a neural network may be used to make the areas estimated to be the walls, floor, and ceiling of the room as planes. If a plane is detected, the process proceeds to S404, and if not detected, the correction map estimation process ends.

In S404, the correction map estimation unit 306 determines whether or not to estimate a correction map based on the distance information corresponding to the plane detected by the plane detection unit 304 (hereinafter referred to as a plane).

FIG. 6 is a graph showing distance measurements on dashed line 510 in FIG. A dashed line 510 straddles indoor walls 506, 507, and 509, which are planar subjects, and persons 502, 503, and 504, which are non-planar subjects. When the distance information generated by the distance information generation unit 305 is correct, the amount of change in the distance measurement value in the plane area is constant. FIG. 6A is a graph when the distance information is correctly generated, and the amount of change in the distance measurement value corresponding to the plane is constant. On the other hand, FIG. 6B shows a case where the distance information is not correctly generated, and the distance information corresponding to the plane is a curve (curved surface). Thus, when the amount of change in the distance measurement value corresponding to the plane is not constant, the correction map estimating unit 306 starts estimating the correction map.

As a specific determination method, for example, a difference in distance measurement values between adjacent pixels on a certain plane may be obtained, and correction may be started when the amount of change in the difference between adjacent pixels exceeds a certain threshold. . Alternatively, correction may be started when the amount of change in the planar region exceeds a certain threshold. Alternatively, the distance measurement value of each pixel on the same plane may be extracted, fitting may be performed so as to form a plane, and the obtained distance value may be compared with the distance measurement value of each pixel for determination. The error between the distance value on the plane of each pixel and the measured distance value is obtained, and estimation of the correction map is started when the average error exceeds the threshold value. Alternatively, estimation of the correction map may be started when the error variation within a certain area of each plane exceeds a specific threshold.

In S405, the correction map estimating unit 306 estimates a correction map based on the distance information corresponding to the plane so that the distance measurement value approaches the plane. The correction map is expressed as a function with image position coordinates as variables. That is, the correction map can be treated as having a correction value for each pixel. In this embodiment, the amount of change ΔW of the curvature of field correction map W is estimated as the correction map. In this embodiment, ΔW is a quadratic surface whose variables are the x and y coordinates of the image, as in equation (2). or minimum) can be used. Since .DELTA.W is the amount of change in the field curvature correction map W, the defocus amount L' after correction is expressed as L'=L+W+.DELTA.W.
_a1x2 ⁺ _{a2y2 + a3z2} ⁺ _a4xy + _a5yz + _a6zx +a7=0 ( ² )
a _{1, 2, . . . 7} : Parameter of quadratic surface (x, y): Position coordinates of image z: Correction value

A correction map estimation process will be described. A defocus amount L' corresponding to the plane is extracted, and the parameter of the quadratic surface is estimated based on that value. According to the lens formula, the measured distance value A of each pixel considering .DELTA.W is calculated by equation (3) (assuming that B=L').

The quadratic surface parameters are estimated so that the distance measurement value A for each pixel calculated according to Equation (3) is as close to a plane as possible in each plane region. A known method may be used for estimating parameters. For example, like the correction start determination process described above, an optimization problem that minimizes evaluation values such as the amount of change in the difference in the distance measurement value between adjacent pixels, the difference from the distance value obtained by fitting, and the variation in the difference. can be solved. However, since it is necessary to consider the distance measurement error of the entire screen, it is necessary to obtain evaluation values for all planes present in the image and minimize the sum total.

In the present embodiment, since the correction is made in consideration of the curvature of field, a quadratic Determine the offset component of the surface.

Since ΔW uses the position coordinate of the image as a variable, it is possible to perform correction even in a non-flat area.

The graph in FIG. 7 shows the correction amount of the curvature of field on the dashed line 510 . A dotted line 701 is the original field curvature correction map W. FIG. The solid line 702 is W+ΔW in planar regions and the dashed line 703 is W+ΔW in non-planar regions. As described above, the correction map estimation unit 306 can estimate the correction map of the entire image based on the distance measurement value of the plane.

In S406, the distance information generation unit 305 regenerates the distance information using the correction map estimated by the correction map estimation unit 306. FIG. As a result, corrected distance information obtained by correcting the distance information of the entire screen is generated.

Through the processing described above, the correction of the distance information of the entire screen is performed. Since an appropriate correction map can be estimated based on the distance measurement value of the plane and the corrected distance information can be generated, the distortion component of the distance measurement value is reduced, and highly accurate distance measurement is possible.

If the curvature of field changes due to changes in the temperature or posture of the imaging apparatus, the curvature of field correction map W at the time of shipment from the factory cannot be used to correct the curvature of field. Since the curvature of field can be represented by a predetermined function according to the positional coordinates of the image, it is possible to estimate a correction map for the entire screen using information on a partial plane area of the screen.

In this embodiment, plane detection is automatically performed by image processing, but it is also possible to allow the user to directly specify a plane by using an imaging device equipped with a touch panel having image display and contact input functions. good. In other words, the user can specify a plane that is difficult to determine in image processing as a detection target, or the user can specify a falsely detected plane and exclude it from the detection target, thereby estimating the correction map with higher accuracy. can.

In this embodiment, the correction map is estimated using all plane distance information. may be used to estimate the correction map. By doing so, the correction map can be estimated with higher accuracy.

In this example, one set of A and B images is used to estimate the correction map, but two or more sets of A and B images are used to correct based on the range measurements of each set of planes. Maps may be estimated. That is, image signals obtained by imaging multiple times are used. For example, in addition to the image 501, an image 801 as shown in FIG. 8A may be used for estimation. Image 801 has a slightly different scene from image 501 . The positions of persons 502 and 503 have moved, 504 has disappeared from the imaging range, and the posture of the imaging device is also different from that of image 501 . FIG. 8B is a diagram in which the image 501 is superimposed on the image 801 by a dotted line. The ceiling 508 and the right wall 507 are plane regions in each image, and the distance information obtained from each image can be used. Also, the vertical line area 802 has a planar area at a position different from that of the image 501, and distance information of a wider planar area than the image 501 alone can be used. In any case, since the amount of distance information in the planar area increases, the correction map can be estimated more robustly and with high accuracy.

A notification function may be added to the image processing apparatus to display on the screen an area where there is no plane distance information. By doing so, it becomes easier for the user to newly capture an image necessary for more accurate correction.

In this embodiment, the correction map is generated when it is determined that the distance measurement value of the plane is not correctly generated, but the correction map may be estimated based on another determination criterion. For example, a temperature sensor or an orientation sensor may be attached to the imaging device, and the correction map may be estimated when the amount of change in them exceeds a certain threshold. Alternatively, sensor values and correction map values may be stored in the non-volatile memory 108, and correction may be performed using a previously estimated correction map according to current sensor values.

Also, an area detection means for detecting an area estimated to be insufficiently corrected by the correction map may be added to the image processing apparatus. In this case, by also providing area notification means for notifying the detected area, it becomes easier for the user to newly capture an image necessary for more accurate correction. A function such as a switch for controlling whether or not to perform correction map estimation may be provided in the image processing apparatus. Power consumption of the image processing apparatus can be suppressed by not performing unnecessary plane detection processing.

Alternatively, the control unit 104 may drive the drive control unit based on the distance information calculated by the image processing device. This makes it possible to drive both or one of the optical system and the imaging unit and automatically adjust the focus position to an appropriate one, so that the user can easily take an image in which the subject is in focus.

In a second embodiment, an image processing apparatus to which the present invention can be applied will be described using an image pickup apparatus that picks up stereo images as an example. In addition, in the case of the same configuration and processing as in the first embodiment, the description will be omitted.

This embodiment can also be applied to an image processing apparatus capable of obtaining an image captured by an external imaging device and performing distance measurement calculation. The image processing device may include mobile phones including digital cameras and smartphones, game machines, tablet terminals, watch-type and eyeglass-type information terminals, medical equipment, equipment for monitoring systems and in-vehicle systems, and the like.

FIG. 9A is a block diagram showing the configuration of an imaging device 900 in which an image processing device to which the present invention can be applied is applied. The imaging device 900 includes optical systems 9011 and 9012, imaging units 9021 and 9022, A/D conversion units 9031 and 9032, a control unit 904, an image processing unit 905, a recording unit 906, a volatile memory 907, and a nonvolatile memory 908. .

Except for optical systems 9011 and 9012 and imaging units 9021 and 9022, the constituent elements of the imaging apparatus 900 have the same functions as those of the constituent elements of the first embodiment, so description thereof will be omitted.

The optical systems 9011 and 9012 are the same type of optical system including a lens group including a zoom lens and a focus lens, an aperture adjustment device, and a shutter device. These optical systems 9011 and 9012 adjust the magnification, focus position, or light amount of the subject images reaching the imaging units 9021 and 9022, respectively, and these parameters are adjusted to be the same for each optical system. .

The imaging units 9021 and 9022 are photoelectric conversion elements such as CCDs and CMOS sensors that photoelectrically convert the subject light beams that have passed through the optical systems 9011 and 9012, respectively, and convert them into electric signals.

A/D converters 9031 and 9032 convert the analog image signals input from the imaging units 9021 and 9022, respectively, into digital image signal data.

Also, the imaging units 9021 and 9022 may be imaging surface phase difference sensors that include an A/D conversion unit and output defocus amount information to the control unit 904, which will be described later.

FIG. 9B shows the positional relationship between optical systems 9011 and 9012 and imaging units 9021 and 9022 of the imaging device 900 . The optical system 9011 and the optical system 9012 are arranged so that the optical axis 9011a and the optical axis 9012a are substantially parallel. The distance between the principal points of the optical system 9011 and the optical system 9012 is the baseline length 921 . The imaging units 9021 and 9022 are arranged substantially on the same plane, and generate an A image and a B image, respectively. A relative positional change in the baseline length direction of the same subject in the A image and the B image is called parallax. By detecting and converting this parallax by a known method, the distance from the imaging device to the subject can be calculated.

A series of operations of the correction map estimation process according to the present embodiment will be described below. Only the map estimation process will be described.

A distance information generation unit 305 of the image processing unit 905 generates distance information from the imaging device to the subject based on the A image and the B image by a known technique. The process of generating distance information is divided into three steps: image preprocessing, parallax calculation, and distance conversion.

First, image preprocessing will be described. The A image and the B image each include distortion aberration, and if left as it is, a distance measurement error corresponding to the area of the image will occur. Use to correct distortion. Distortion generally occurs in the circumferential and tangential directions about the optical axis. Distortion can therefore be modeled, for example, as in Equation (4).

（ｘ_ｕ，ｙ_ｕ）：歪曲収差のないレンズの画像座標
（ｘ_ｄ，ｙ_ｄ）：歪曲収差のあるレンズの画像座標
ｋ_{１，２，・・・５}：歪曲補正パラメータ

(x _u , _yu ): Image coordinates of lens without distortion (x _d , y _d ): Image coordinates of lens with distortion k _{1, 2, . . . 5} : Distortion correction parameters

Since the distortion correction parameter k does not change unless the states of the optical systems 8011 and 8012 change, basically the one recorded in the nonvolatile memory 807 at the time of shipment from the factory is used.

Next, the distance information generation unit 305 calculates parallax. A known method may be used as the calculation method. For example, the parallax is calculated by block matching, phase-only correlation method, or the like.

Finally, the distance information generation unit 305 converts the parallax calculated at each pixel into distance information from the imaging device to the subject according to Equation (5).

Ａ：物面から光学系の主点までの距離
Ｂ：基線長の距離
Ｆ：光学系の焦点距離
Ｚ：視差

A: Distance from object surface to principal point of optical system B: Distance of baseline length F: Focal length of optical system Z: Parallax

The distance information from the imaging device to the object generated as described above is called a distance measurement value.

The correction map estimation unit 306 starts estimation processing of the correction map when the amount of change in the distance measurement value corresponding to the plane is not constant. When the distortion coefficient of the lens changes due to changes in the temperature of the imaging device, etc., the imaging position of the light flux changes depending on the area, and thus a distorted distance measurement error appears on the entire screen. In this embodiment, the amount of change in the imaging position due to distortion is estimated as the correction map. That is, the correction map is a model similar to Equation (4), and correction is performed by obtaining the amount of change Δk of the distortion correction parameter k for each coefficient. As in the first embodiment, the correction method is the same as in the first embodiment. should be considered and minimized.

By the above-described processing, an appropriate correction map can be estimated based on the distance measurement value of the plane, so the distortion component of the distance measurement value is reduced, and highly accurate distance measurement is possible.

If the distortion changes due to a change in temperature or a change in the posture of the apparatus, the distortion correction parameter k at the factory shipment cannot correct the distortion, resulting in a distance measurement error such that the entire screen is distorted. Since distortion aberration can be represented by a predetermined function according to the positional coordinates of the image, it is possible to estimate a correction map for the entire screen using information on a partial plane area of the screen.

[Other embodiments]
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

100 imaging device 101 optical system 102 imaging unit 103 A/D conversion unit 104 sentencing unit 105 image processing unit 106 recording unit 107 volatile memory 108 nonvolatile memory

Claims (21)

generating means for generating distance information using a plurality of image signals captured at different viewpoints;
a detecting means for detecting a plane area in the image signal;
estimating means for estimating a correction map for correcting the distance information;
The estimating means expresses the correction map as a function having the position coordinates of the image as variables,
An image processing apparatus, wherein the correction map is estimated based on distance information corresponding to the plane area. The generating means generates the distance information using a field curvature correction map that corrects aberration based on field curvature,
2. The image processing apparatus according to claim 1, wherein said estimating means estimates said correction map for correcting an error of said distance information due to said curvature of field correction map. 3. The image processing apparatus according to claim 2, wherein the correction map is a function represented by a quadratic surface. The generating means generates the distance information based on the image signal corrected for distortion,
2. An image processing apparatus according to claim 1, wherein said function in said estimation means is expressed based on a model of distortion aberration. The estimating means calculates a plane based on distance information corresponding to the plane area, and estimates a correction map so that a difference between the obtained distance value from the plane and the distance information is small. 5. The image processing apparatus according to any one of claims 1 to 4. 6. The method according to any one of claims 1 to 5, wherein said estimating means estimates the correction map such that a change amount of distance information corresponding to said planar area is constant in said planar area. Image processing device. 7. The image processing apparatus according to any one of claims 1 to 6, wherein said generating means generates corrected distance information by correcting said distance information using said correction map. The generation means acquires image signals captured multiple times from different viewpoints, generates a plurality of distance information based on the signals of each pixel, and the correction means generates a correction map based on the plurality of distance information. 8. The image processing apparatus according to any one of claims 1 to 7, wherein: 9. The image processing apparatus according to claim 8, wherein said estimating means estimates said correction map based on said plurality of pieces of distance information at the same position coordinates of the image. 9. The image processing apparatus according to claim 8, wherein the image signal is an image signal acquired by imaging a plurality of times so that the positions of the planar regions are different. 11. The image processing apparatus according to any one of claims 1 to 10, wherein said detection means has a function of determining whether a plane is included in said image signal. 12. The image processing according to claim 11, further comprising notifying means for notifying a user of information prompting the user to photograph a plane when the detecting means determines that the plane is not included in the image. Device. 13. The apparatus according to any one of claims 1 to 12, further comprising area detection means for detecting an area estimated to be insufficiently corrected by said correction means, and area notification means for notifying the area detected by said area detection means. The image processing device according to any one of items 1 and 2. 14. The estimating means according to any one of claims 1 to 13, wherein the estimating means estimates the correction map when there is a change greater than a predetermined threshold in the distance information corresponding to the plane. Image processing device. 15. The image processing apparatus according to any one of claims 1 to 14, wherein said correction means estimates a correction map based on distance information in said plane area and reliability of said distance information. An imaging apparatus comprising: the image processing apparatus according to any one of claims 1 to 15; and imaging means for imaging a subject image formed via an optical system. The imaging means is configured such that light beams passing through different pupil regions of an optical system are incident on a plurality of photoelectric conversion units, and the plurality of images are generated based on signals generated by the plurality of photoelectric conversion units. 17. An imaging device according to claim 16. A change detection means for detecting a change in the temperature or posture of the imaging device is provided, and the generation means generates the correction map when the change detection means detects a change amount larger than a predetermined threshold value. 18. The imaging device according to claim 16 or 17. drive control means for driving and controlling both or one of the optical system and the imaging section, the drive control means driving the optical system and the imaging section using the distance information obtained by the image processing apparatus, and focusing; 19. The imaging apparatus according to any one of claims 16 to 18, wherein the position is automatically adjusted. a generation step of generating distance information using a plurality of image signals captured at different viewpoints;
a detection step of detecting a planar area in the image signal;
an estimation step of estimating a correction map for correcting the distance information;
The image processing method, wherein the estimating step expresses the correction map by a function having positional coordinates of the image as variables, and estimates the correction map based on distance information corresponding to the plane area. A program that causes a computer to execute each step of the control method according to claim 20.

Images