JP2009232348A - Imaging apparatus, distance information acquiring method, image processing method, and drive control method of optical system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To acquire the information of distances in an image from the single image without carrying out special photographing such as high-speed continuous shooting. <P>SOLUTION: The imaging apparatus has an image generating means, a region extracting means, and a distance information acquiring means. The image generating means generates a plurality of processed images by subjecting the single image to smoothing processing of different degrees. The region extracting means calculates the differences between the single image and the processed images and extracts the regions where differences are generated. The distance information acquiring means acquires the information of the distances in the regions, which are extracted by the region extracting means, by using the processing information of the smoothing processing and the imaging information in the process of taking the single image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that reflects a measured distance to a subject at the time of image acquisition, a distance information acquisition method, an image processing method, and an optical system drive control method.

  As focus adjustment at the time of shooting in an imaging apparatus such as a digital camera, a focus evaluation value indicating the sharpness of the image is calculated from an image signal captured in the imaging apparatus, and the focus evaluation value is maximized. It is common to move a focus adjustment lens (so-called focus lens). In addition to this, distance measurement sensors are discretely arranged on the image pickup surface of an image sensor such as a CCD or CMOS, and the focus lens is moved based on the output value from these distance measurement sensors. Adjustments are also generally made.

As a method for acquiring distance information in an image in such an imaging apparatus, a method for determining a distance from the size (number of pixels) occupied by a main subject in the image, or a plurality of optical systems in different aperture states are used. A method has been proposed in which photographing is performed at the same time, the degree of blur is detected based on the result of comparing the contrast ratio of each pixel in the acquired image, and distance information in the image is obtained (see Patent Document 1).
JP 2002-10126 A

  However, although the above-described Patent Document 1 can detect the degree of blur according to the distance of the captured image, a plurality of imaging optical systems are required for simultaneously capturing images with different aperture values. Even if a single single-lens optical imaging device can perform high-speed continuous shooting while changing the aperture value, it is necessary to perform high-speed continuous shooting that does not change the position of the subject. In addition, it is necessary to mount a special imaging function for performing such high-speed continuous shooting in the imaging apparatus. For this reason, it is difficult to acquire distance information in an image from one image acquired by a single single-lens optical system imaging device.

  The present invention has been invented to solve the above-described problem, and can accurately acquire distance information in an image from a single image without performing special shooting such as high-speed continuous shooting. An object of the present invention is to provide an imaging apparatus, a distance information acquisition method, an image processing method, and an optical system drive control method that can be performed.

  An imaging apparatus according to a first aspect of the present invention includes an image generation unit that generates a plurality of processed images obtained by performing smoothing processing with different degrees on a single image, and the single image and the processed image. A region extraction unit that calculates a difference and extracts a region where the difference is generated, and processing information at the time of the smoothing process and imaging information when the single image is captured are extracted by the region extraction unit. Distance information acquisition means for acquiring distance information in the area.

  In a second aspect based on the first aspect, the image generating means performs the smoothing process for each evaluation area obtained by dividing the single image into a plurality of parts, and the area extracting means By calculating the difference between the evaluation area before the smoothing process and the evaluation area after the smoothing process for each evaluation area, an area where the difference occurs is extracted.

  According to a third invention, in the first or second invention, the image processing apparatus further comprises filter processing means for executing filter processing on the single image, and the filter processing means is configured to perform filter processing based on the distance information. Is executed for each corresponding region.

  According to a fourth invention, in the first or second invention, an imaging optical system that captures subject light, and a drive control unit that drives and controls the imaging optical system along an optical axis direction of the imaging optical system. The drive control means performs drive control of the imaging optical system based on the distance information.

  A distance information acquisition method according to a fifth aspect of the present invention includes an image generation step of generating a plurality of processed images obtained by performing smoothing processing with different degrees on a single image, the single image, and the processed image. Are extracted by the region extraction step using the region extraction step for extracting the region where the difference occurs and the processing information at the time of the smoothing process and the imaging information when the single image is captured. A distance information acquisition step of acquiring distance information in the area.

  In a sixth aspect based on the fifth aspect, the image generation step performs the frequency processing for each evaluation region obtained by dividing the single image into a plurality of segments, and the region extraction step includes the smoothing A difference between the evaluation area before the process and the evaluation area after the smoothing process is calculated for each evaluation area, thereby extracting an area where the difference occurs.

  An image processing method according to a seventh aspect is characterized in that different filtering processes are executed for each extracted region based on the distance information acquired by the distance information acquisition method according to the fifth or sixth aspect.

  An optical system drive control method according to an eighth aspect of the invention is characterized in that the imaging optical system that captures subject light is driven and controlled based on the distance information acquired by the distance information acquisition method of the fifth or sixth invention. To do.

  According to the present invention, it is possible to easily acquire distance information in an image only by using a single image and imaging information. Further, by using the distance information in the acquired image, the adjustment of the imaging optical system at the time of imaging can be adjusted with high accuracy. Moreover, a natural image based on the distance information can be acquired by performing image processing combined with the distance information.

  FIG. 1 is a functional block diagram showing an electrical configuration of a digital camera using the present invention.

  FIG. 1 is a functional block diagram illustrating an example of a digital camera 10. As is well known, the digital camera 10 photoelectrically converts the subject light transmitted through the imaging optical system 15 by the imaging element 16 and acquires an electrical signal after the photoelectric conversion as image data.

  The imaging optical system 15 includes an imaging lens 20 and a lens group 21 including a zoom lens and a focus lens. The lens group 21 is composed of a plurality of lenses, but is a single lens in FIG. The zoom lens included in the lens group 21 moves along the optical axis L so as to achieve the selected imaging magnification. Further, the focus lens slightly moves along the optical axis L when adjusting the focus of the subject image. The lens group 21 is driven and controlled by a lens driving mechanism 22.

  For example, a CCD (Charge Coupled Device), a CMOS (Complementary Metal-Oxide Semiconductor), or the like is used as the imaging device 16, and the signal charge corresponding to the amount of received light is accumulated, and the accumulated signal charge is AFE (Analog Front End). Output to 30. The drive control of the image sensor 16 is controlled by a driver 31. In the following description, the signal charge accumulated by the image sensor 16 is referred to as an analog image signal.

  The AFE circuit 30 includes an AGC circuit and a CDS circuit (not shown). The AFE circuit 30 performs analog processing such as gain control and noise removal on the input analog image signal.

  A DFE (Digital Front End) circuit 32 converts an analog image signal subjected to analog processing by the AFE circuit 30 into a digital image signal. The converted digital image signals are collected frame by frame and stored in the buffer memory 33 as digital image data (hereinafter referred to as image data). The AFE circuit 30, the driver 31, and the DFE circuit 32 are controlled based on the operation timing in the timing generator (TG) 34, respectively. Reference numeral 35 denotes a bus, and each part of the digital camera 10 is electrically connected via the bus 35.

  The image processing unit 40 performs image processing such as contour compensation, gamma correction, and white balance correction on the image data stored in the buffer memory 33. The image processing unit 40 includes a filter processing unit 41, and the filter processing in the filter processing unit 41 is also performed during the above-described image processing. The filtering process 41 is executed based on distance information attached to the image data.

  The image data subjected to the image processing is subjected to format processing for compression into a storage method such as a JPEG method, and then stored in the buffer memory 33 again. Thereafter, the image data stored in the buffer memory 33 is compressed using a compression rate corresponding to a predetermined compression method. For example, when a through image is displayed on the LCD 42 provided in the digital camera 10, compression processing is performed on the image data after the image processing using a compression rate that matches the resolution of the LCD 42, and the display control unit 43. Output to.

  On the other hand, when storing image data, the image data is stored in the built-in memory 45 after being subjected to compression processing using a compression rate corresponding to the storage method described above. The image data stored in the built-in memory 45 is written to a storage medium 47 such as a memory card or an optical disk via the media controller 46. The compression rate for image storage is set to the resolution set by the user operating the operation unit 54 in addition to the compression rate set for the image storage resolution set in advance in the digital camera 10. Compression rate.

  The distance measuring sensor 50 operates, for example, when the release button 55 is operated, and detects the distance to the subject. The photometric sensor 51 operates, for example, when the release button 55 is operated, and detects the brightness (luminance) of the subject.

  The strobe device 52 irradiates the subject with strobe light when the subject brightness obtained from the output signal of the photometric sensor 51 is a predetermined value or less. The strobe device 52 is controlled by a strobe control unit 53.

  The operation unit 54 is operated, for example, when selecting a mode during imaging or changing settings during imaging. The release button 55 is a button operated when taking an image. When the release button 55 is operated, a distance measuring process, an AE process, an AF process, and the like to the subject are executed, and then an imaging process is executed.

  The CPU 60 comprehensively controls each unit of the digital camera 10. The CPU 60 functions as an area extraction unit 63 and a distance information acquisition unit 64 in addition to the AE processing unit 61 and the AF processing unit 62 by executing the control program 66 stored in the built-in memory 45. .

  Based on the subject brightness obtained from the output signal of the photometric sensor 51, the AE processing unit 61 controls the aperture value, shutter speed (in the case of an electronic shutter, the charge accumulation time in the image sensor 16), and control sensitivity of the image sensor 16. Find exposure conditions such as values. Since the method for obtaining the aperture value, shutter speed, and control sensitivity value of the image sensor 16 is well known, the details thereof are omitted here.

  The AF processing unit 62 calculates a physical quantity necessary for AF control from the input image signal. For example, a focus evaluation value indicating the sharpness of the image is calculated from the input image signal. The movement of the focus lens is controlled via the lens driving mechanism 22 so that the calculated focus evaluation value is maximized.

  The region extraction unit 63 extracts a region having frequency components included in each of a plurality of different frequency component bands from the single image data for each band. Hereinafter, a method for extracting a region including a specific frequency component will be described with reference to FIGS. 2 and 3. Below, the image used for the process which extracts an area | region is called the original image P0, and is demonstrated.

  First, for example, blur processing using two different blur amounts is performed on the original image P0 to generate two images. Hereinafter, an image generated by the blur processing with a small blur amount will be referred to as a first image, and an image generated by the blur processing with a large blur amount will be referred to as a second image. In the present embodiment, a case where the original image P0 is subjected to the blur processing using two different blur amounts will be described. However, the blur amount used for the blur processing is not limited to two types, and three or more types of blur amounts are used. The amount of blur may be used.

  FIG. 3B shows the pixel intensity when attention is paid to the pixel between A and A ′ in the image in FIG. In FIG. 3B, the pixel intensity of the original image P0 is indicated by a solid line, the pixel intensity of the first image P1 is indicated by a dotted line, and the pixel intensity of the second image P2 is indicated by a two-dot chain line. When an image from which points B and C are extracted as edges when edge extraction processing is performed is the original image P0, the first image P1 is the vicinity of the edges B and C in the original image P0 (DD ′ And the image intensity between EE ′ changes in a curved line. Similarly, also in the second image P2, the image intensity in the vicinity of the edges B and C (between F-F 'and between G-G') changes in a curved line. In the second image P2, the region of the pixel whose image intensity changes in a curved line is wider than that of the first image P1.

  FIG. 3C shows the difference between the pixel intensity in the original image P0 and the pixel intensity in the first image P1 (solid line in the figure), and the difference between the pixel intensity in the original image P0 and the pixel intensity in the second image P2 (see FIG. 3). The middle dotted line) is shown. In FIG. 3C, the calculated difference is shown as an absolute value. As shown in FIG. 3C, the difference between the pixel intensity in the original image P0 and the pixel intensity in the first image P1 occurs in the vicinity of the edges B and C of the original image P0. Is the maximum. Similarly, in the difference between the pixel intensity in the original image P0 and the pixel intensity in the second image P2, a difference in pixel intensity occurs in the vicinity of the edges B and C of the original image, and the difference is maximum at the edges B and C. It becomes. As described above, in the second image P2, since the range in which the pixel intensity changes in a curved line is wider than the range of the first image P1, the pixel intensity in the original image P0 and the pixel intensity in the second image P2 The range in which the difference occurs is wider than the range in which the difference between the pixel intensity in the original image P0 and the pixel intensity in the first image P1 occurs.

  As described above, since the first image P1 is an image that has been subjected to a blurring process with a small blur amount with respect to the original image P0, the difference between the pixel intensity of the original image P0 and the pixel intensity of the first image P1. The region where the occurrence of the noise can be determined as a region having a frequency component in the high frequency band (region 71 shown in FIG. 2). That is, edge information (hereinafter referred to as first edge information) can be acquired by extracting a region where a difference between the pixel intensity of the original image P0 and the pixel intensity of the first image P1 occurs.

  In addition, since the second image P2 is an image that has been subjected to blurring processing with a large blur amount with respect to the original image P0, a region in which a difference between the pixel intensity of the original image P0 and the pixel intensity of the second image P1 occurs. Is determined to be a region (region 72 shown in FIG. 2) having a frequency component of a high frequency band and a predetermined frequency band less than the high frequency band. Accordingly, by extracting a region where a difference between the original image P0 and the second image P2 occurs, edge information different from the first edge information (hereinafter, second edge information) can be acquired. It should be noted that the amount of blur in the blur processing performed on the original image P0 and the frequency component band included in the region extracted by the difference are associated in advance based on the result of an experiment or the like.

  The distance information acquisition unit 64 acquires distance information in the image using the first edge information and the second edge information acquired from the region extraction unit 63. The first edge information is information indicating a region having a frequency component of a high frequency band, and the second edge information is a region having a high frequency band and a frequency component of a predetermined frequency band less than the high frequency band. It is information which shows.

  As shown in FIG. 2, a region having a frequency component of a high frequency band from the first edge information becomes a region 71, and a frequency component of a high frequency band and a predetermined frequency band less than the high frequency band from the second edge information. When the area to be included is the area 72, an area (area indicated by hatching in FIG. 2) obtained by dividing the area 72 from the area 71 has an area having a frequency component of a predetermined frequency band less than the frequency component of the high frequency band; Become. The region excluding the region 71 and the region 73 is a region having frequency components in other frequency bands. Thereby, an image can be divided | segmented for every zone | band of a frequency component by using 1st edge information and 2nd edge information.

  First, the distance information acquisition unit 64 calculates the defocus amount Δ using the aperture value F at the time of imaging. The defocus amount Δ is calculated using Δ = σ × F (σ: the diameter of the scattering circle, F: aperture value).

  Further, the distance information acquisition unit 64 calculates a distance (subject distance) L to the main subject from the feed amount a of the imaging lens 20 and the focal length f at the time of imaging using the equation (1).

  The distance information acquisition unit 64 refers to the distance table data 67 stored in the built-in memory 45, and reads the subject depth x associated with the calculated defocus amount Δ and subject distance L. Since the subject distance L is calculated by the above-described equation (1), it is estimated that the target region is within ± x with respect to the subject distance L. That is, the sum of the subject distance L and the subject depth ± x is the distance of the target region. In this way, the distance to the target region is acquired. The calculated distance is attached to the image data as distance information associated with each target region.

  The distance table data 67 described above is composed of table data in which the subject distance and the defocus amount are associated with the subject depth x. As shown in FIG. 4, when the subject distance is L, the focal length is f, and the defocus amount is Δ, the subject depth x at a position where the focus is not in focus (blurred) is expressed by equation (2).

  Using this equation (2), the subject depth x when the subject distance L and the defocus amount Δ are changed is calculated, and the calculated subject depth x is used as the subject distance L and the defocus amount Δ. The distance table data 67 is generated by associating with each other. The distance table data 67 is created in advance and stored in the built-in memory 45.

  Next, the flow of processing for acquiring distance information using image data will be described based on the flowchart of FIG. Step S101 is a process for performing imaging. Since the processing at the time of imaging is well known, the details are omitted here. Image data acquired by this imaging is accompanied by imaging conditions such as exposure conditions and focal length as imaging information.

  Step S102 is processing for acquiring first edge information from the acquired image data. The processing in step S102 is executed by the region extraction unit 63. The area extraction unit 63 performs a blurring process with a small amount of blur using the image data as the original image data, and then extracts an area in which a difference between the obtained image data and the original image data is generated, whereby the first edge is extracted. Get information.

  Step S103 is processing for acquiring second edge information from the acquired image data. The processing in step S103 is executed by the region extraction unit 63, similarly to the processing in step S102. The region extraction unit 63 performs blur processing using a blur amount larger than the blur amount used in step S102, and then extracts a region where a difference between the obtained image data and the original image data is generated. 2 Edge information is acquired.

  Step S104 is processing for acquiring distance information in the image using the imaging information and the acquired first edge information and second edge information. The processing in step S104 is executed by the distance information acquisition unit 64. As described above, the first edge information indicates a region having a frequency component of a high frequency band, and the second edge information is a frequency component of a high frequency band and a predetermined frequency band less than the high frequency band. The area | region which has is shown. Using these edge information, a region having a frequency component in a high frequency band in the original image data (reference numeral 71 in FIG. 2), a region having a frequency component in a predetermined frequency band that is less than the high frequency band (in FIG. 2) 73) and a region having frequency components in other frequency bands (reference numeral 74 in FIG. 2).

  After dividing the region in the image for each frequency component band, the distance information acquisition unit 64 reads the aperture value F from the imaging information attached to the image data that is the source of the image, and calculates the defocus amount Δ. . In addition, the distance information acquisition unit 64 reads the feeding amount a and the focal distance f of the imaging lens 20 from the imaging information, and calculates the subject distance L.

  When the defocus amount Δ and the subject distance L are respectively calculated, the distance information acquisition unit 64 reads the distance table data 67 and subjects associated with the calculated defocus amount Δ and the subject distance L, respectively. Read depth x. By obtaining the sum of the read subject depth x and subject distance L, the distance of the target area is acquired, and distance information is generated by associating the target area with the determined distance. In this way, distance information for each region is generated and attached to the image data. It should be noted that distance information cannot be generated for the region 74 because there is no edge information. For such a region, for example, if distance information is generated using a distance set for each scene or mode. Good. Thereby, the distance information in the image can be easily obtained without using a plurality of images.

  The distance information attached to the image data is used, for example, at the time of filter processing in the filter processing unit 41. That is, since the distance information includes information in which a region and a distance in the region are associated with each other, a filtering process corresponding to the distance is performed for each corresponding region. Note that examples of the filter processing include edge enhancement processing in addition to blur processing. As a result, for example, it is possible to perform different filtering processes for the main subject area and the background area, and to acquire an image with natural blur reflecting the depth direction of the actual object scene. Can do. It is also possible to easily generate an image in which the main subject is emphasized by reducing the blur amount for the main subject and increasing the blur amount in the background region.

  In this embodiment, edge information is acquired after performing blur processing using two different blur amounts on the entire image, but the present invention is not limited to this, and the image is divided into a plurality of regions (hereinafter referred to as a plurality of regions). , Evaluation area), and edge information may be acquired for each evaluation area. By dividing the image into a plurality of evaluation regions, it is possible to shorten the processing time of the above-described blur processing and processing for acquiring edge information. In addition, when acquiring distance information, since an area | region is acquired for every zone | band of a different frequency component, the evaluation area | region should be small.

  In this embodiment, although it is set as embodiment which performs a filter process based on the acquired distance information, it is not limited to this, For example, the acquired distance information is used for drive control of an imaging optical system. Is also possible. First, the CPU 60 acquires distance information from image data obtained when the release button 55 is pressed. Thereafter, from the acquired distance information and the imaging conditions, the amount of extension of the imaging lens 20 when the subject desired by the user is focused is calculated using the Gauss formula. The difference between the calculated amount of extension of the imaging lens 20 and the current amount of extension of the imaging lens 20 is obtained as a correction amount of the imaging lens 20, and the position of the imaging lens 20 is adjusted based on this correction amount and imaging is performed again. I do. Thereby, even in a digital camera in which AF sensors are not discretely arranged, an image focused on a main subject desired by a user can be easily and highly accurately acquired.

  In this embodiment, the filter processing is executed by the filter processing unit provided in the digital camera. However, the present invention is not limited to this. For example, image data with distance information in the image attached by the digital camera is converted into an image. It is also possible to perform filter processing in the image processing apparatus after being taken into the processing apparatus or the like.

  In the present embodiment, by dividing the region extracted based on the difference between the original image and the first image from the region extracted based on the difference between the original image and the second image, the image is separated for each frequency band. It is not necessary to limit to this. For example, it is possible to add an area obtained from the original image and the first image and an area obtained from the original image and the second image. In this case, as shown in FIG. 6, for example, a value “1” is assigned to a region 75 where a difference between the original image and the first image occurs, and a value “0” is assigned to a region 76 where no difference occurs. Similarly, a value “1” is assigned to an area 77 where a difference between the original image and the second image occurs, and a value “0” is assigned to an area 78 where no difference occurs. When these are added, the region 75 where the difference between the original image and the first image occurs is partly overlapped with the region 77 where the difference between the original image and the second image overlaps. The value is “2”. Of the regions 77, the region 79 that does not overlap with the region 75 overlaps with the region 76, but since the value assigned to the region 76 is “0”, the value attached to the region 79 remains “1”. It is. Further, although the area 78 overlaps with a part of the area 76, the values assigned to the area 78 and the area 76 are “0”, respectively. Therefore, the value assigned to the value of the area 78 is “0”. "". Thus, the distance information in the image can be acquired by obtaining the distances corresponding to the values assigned to the respective areas and combining the values assigned to the areas and the distances.

  In this embodiment, a digital camera has been described. However, the present invention is not limited to this, and can be applied to, for example, an image processing apparatus. As shown in FIG. 7, the image processing apparatus 90 includes an area extraction unit 91, a distance information acquisition unit 92, and an image processing unit 93. The functions of the region extraction unit 91 and the distance information acquisition unit 92 are the same as the functions of the region extraction unit 63 and the distance information acquisition unit 64 described above, and are therefore omitted here. When the image data 95 is input to the image processing apparatus 90, edge information is acquired by the area extraction unit 91, and then distance information in the image is acquired using the distance table data 96. Note that the image processing unit 93 executes blur processing, edge enhancement processing, and the like on the image data based on the acquired distance information. As a result, it is possible to obtain image data 95 'that has undergone image processing.

It is a functional block diagram which shows the electric constitution of the digital camera using this invention. It is a schematic diagram which shows the flow at the time of acquiring the distance information in an image. (A) the pixel intensity measurement position in the image, (b) the pixel intensity of the original image, the first image and the second image at the measurement position, and (c) the pixel intensity of the original image and the pixel intensity of the first image. It is a figure which shows the difference and the difference of the pixel intensity of an original image, and the pixel intensity of a 2nd image. It is a schematic diagram which shows the positional relationship at the time of imaging. It is a flowchart which shows the flow which acquires distance information. It is a schematic diagram in the case of adding the areas obtained from the first edge information and the second edge information. It is the schematic of the structure of the image processing apparatus which acquires distance information from image data.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Digital camera, 60 ... CPU, 63, 91 ... Area extraction part, 67, 96 ... Distance table data, 64, 92 ... Distance information acquisition part, 41 ... Filter processing part, 90 ... Image processing apparatus

Claims (8)

Image generating means for generating a plurality of processed images obtained by performing smoothing processing with different degrees on a single image;
A region extracting means for calculating a difference between the single image and the processed image and extracting a region where the difference occurs;
Distance information acquisition means for acquiring distance information in the region extracted by the region extraction means, using processing information at the time of the smoothing processing and imaging information when the single image is captured;
An imaging apparatus comprising: The imaging device according to claim 1,
The image generating means performs the smoothing process for each evaluation region obtained by dividing the single image into a plurality of parts,
The area extraction unit is configured to extract an area in which the difference occurs by calculating a difference between the evaluation area before the smoothing process and the evaluation area after the smoothing process for each evaluation area. apparatus. The imaging apparatus according to claim 1 or 2,
A filter processing means for performing filter processing on the single image;
The image pickup apparatus, wherein the filter processing unit executes a filter process based on the distance information for each corresponding region. The imaging apparatus according to claim 1 or 2,
An imaging optical system that captures subject light;
Drive control means for driving and controlling the imaging optical system along the optical axis direction of the imaging optical system,
The image pickup apparatus, wherein the drive control unit performs drive control of the image pickup optical system based on the distance information. An image generation step for generating a plurality of processed images obtained by performing smoothing processing with different degrees on a single image;
A region extraction step of calculating a difference between the single image and the processed image and extracting a region where the difference occurs;
A distance information acquisition step of acquiring distance information in the region extracted by the region extraction step using the processing information at the time of the smoothing processing and the imaging information when the single image is captured;
A distance information acquisition method characterized by comprising: The distance information acquisition method according to claim 5,
The image generation step performs the frequency processing for each evaluation region obtained by dividing the single image into a plurality of parts,
The region extracting step extracts a region in which the difference occurs by calculating a difference between the evaluation region before the smoothing process and the evaluation region after the smoothing process for each evaluation region. Information acquisition method.   7. An image processing method, wherein, based on the distance information acquired by the distance information acquisition method according to claim 5 or 6, different filter processing for each extracted region is executed on the single image. .   7. An optical system drive control method, comprising: driving-controlling an imaging optical system that captures subject light based on distance information acquired by the distance information acquisition method according to claim 5.

Images