JP2006067521A - Image processor, image processing method, image pickup device, and image pickup method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image pickup device, image processor and image processing method capable of creating an image with a three-dimensional appearance from a flat image with simple operations. <P>SOLUTION: A predetermined area of a screen of picked up image data 1 is divided into multiple portions. Front-rear positional relations of objects in the portions are classified into groups. Each group is applied with a different filter with a different frequency characteristic. Thus, some image parts are highlighted by applying HPF in units of portions, while the other image parts are blurred by applying LPF. The image with the three-dimensional appearance is thereby created. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

The present invention relates to an image capturing apparatus such as an electronic camera apparatus, an image processing apparatus, and a method thereof.
In particular, the present invention relates to an image capturing apparatus that provides a two-dimensional image captured by an electronic camera apparatus using a small lens as a stereoscopic image, and an image processing method and apparatus for performing the processing.

2. Description of the Related Art Electronic camera devices that use a CCD (Charge Coupled Device) as an image pickup device, incorporate image processing means, and can immediately display image data picked up by a CCD on a display device are widely used.
By the way, in general, in an electronic camera device using an image pickup device such as a CCD having a deep focal depth, in order to produce a stereoscopic effect like a silver salt camera device, only an image at a specific distance is focused, and the other distance portion It was difficult to perform the process of blurring the image.

Here, if a lens with a large aperture and an image sensor with a large light receiving area are used, the depth of focus can be reduced, and a blurred image like an analog silver halide film camera can be taken, but a lens with a large aperture In addition, an image pickup device having a large light receiving area must be used, and the electronic camera device becomes large and the price of the electronic camera device is high.
Therefore, it is desired that the electronic camera apparatus can perform image processing such as enhancing or blurring arbitrary image data by electrical or software processing.
Such prior art will be described.

  Japanese Patent Application Laid-Open No. 2004-228561 sequentially captures images while shifting the focal length, and sequentially overwrites only the focused image portion to finally obtain an in-focus image over the entire screen area. At this time, distance information of each point of the image is also acquired. Thereafter, a technique is disclosed in which a blurring process is performed on pixels other than the designated focal length so as to obtain an image in which only the designated focal length is in focus.

  In Patent Document 2, each point of a captured image is measured by a three-time measurement data input unit, a data table of each point of the image and a distance is created, and when the user specifies a position to be blurred, the distance of the specified position is data A technique is disclosed in which image points of the same distance are obtained from a table, and background blur processing is performed for all image points of the same distance.

Similar to the technique disclosed in Patent Document 2, Patent Document 3 measures each point of a captured image, creates a data table of each image point and distance, and designates a specified position when a user specifies a position to be blurred. Is obtained from a data table, image points with the same distance are extracted, and background blur processing is performed for all image points with the same distance. Patent Document 3 further discloses a technique for combining a blurred background image and a non-blurred image.
JP-A-8-163423 JP 2000-207549 A JP 2000-259823 A

  Since the technique disclosed in Patent Document 1 sequentially captures images while shifting the focal length, it takes time to obtain a desired image. Furthermore, since distance information for each point is also used, it is necessary to store a large amount of distance information.

  The technique disclosed in Patent Literature 2 is complicated in processing and takes a long time. Furthermore, since a data table of each point and distance of the image is created, it is necessary to store a large amount of data.

  The technique disclosed in Patent Document 3 is also complicated in processing and takes processing time. Furthermore, since a data table of each point and distance of the image is created, it is necessary to store a large amount of data.

  The present invention is intended to solve the above-described problems and the like, and an object of the present invention is to provide an image pickup apparatus, an image processing apparatus, and an image processing method capable of performing preferable image processing with simple processing.

  The present invention divides a captured image data screen into a plurality of areas, divides the anteroposterior relationship of subjects in each area into groups, and applies filters having different frequency characteristics according to the groups, thereby specifying a specified range. This is based on the concept of emphasizing or blurring an image of a subject at a distance of each region.

In order to achieve such an object, the image processing apparatus according to the present invention virtually divides a subject space into small spaces, measures distances to subject portions belonging to the spaces, and stores them as distance data. The image data after imaging is divided into groups based on the distance from the imaging device based on the distance data, and filters having different characteristics are applied to each group.
As a result, image data belonging to a desired virtual space in the subject space can be emphasized or blurred, restrictions on lens focus characteristics are relaxed, and image processing that is preferable regardless of which lens is used Can do.

  According to the present invention, image dividing means for dividing one screen of image data (image data of one screen) into a plurality of regions, and different spatial frequency filters are applied to the plurality of regions according to the distance to the subject. And an image processing apparatus including a filter unit.

  In addition, according to the present invention, an imaging unit that captures an image of the imaged space, an image dividing unit that divides the captured image data of one screen into a plurality of regions, and the divided image data. Provided is an image pickup apparatus comprising distance detection means for detecting a distance to a subject in image data of each region and filter means for applying different spatial frequencies to the plurality of regions according to the distance to the subject. .

With the above-described means, even in an image pickup apparatus using a CCD or the like having a deep focal depth, the subject space can be three-dimensionally measured by measuring the distance in front and back of each image in the subject space (captured space) in advance or by measuring each time. It is possible to create an image with suitable image processing (for example, arbitrary blurring) by dividing the image into regions and applying (applying) filters having different frequency characteristics to each region.
As a result, it is possible to create an image with arbitrary blurring or emphasis desired by the photographer.

  The present invention can provide an image processing apparatus, an image capturing apparatus, and an image processing method which are not conventionally provided.

First Embodiment A first embodiment of the image processing apparatus of the present invention will be described.
FIG. 1 is an explanatory diagram showing an image pickup apparatus 1 which is a first embodiment of an image processing apparatus of the present invention and an image pickup method using the same.
In FIG. 1, the image capturing apparatus 1 includes a distance measuring device 3.
The configuration of the image pickup apparatus 1 will be described with reference to FIG.
The image capturing apparatus 1 in the present embodiment is an image capturing apparatus capable of capturing a still image or a moving image, and is, for example, an electronic camera apparatus, a still camera apparatus, a video camera apparatus, or the like.

In the present embodiment, the imaged (subject) space S imaged by the image capturing apparatus 1 is divided into a plurality of groups according to the distance from the image capturing apparatus 1. In this example, the image pickup apparatus 1 is divided into three groups according to the short distance D1, the medium distance D2, and the long distance D3. In the example illustrated in FIG. 1, the distance to the subject is divided into three groups in the example illustrated in FIG. 1, but the distance to the subject can be further finely divided, or conversely, two distance groups It can also be divided into
A distance measuring device 3 is used to measure the distance from the image capturing device 1 to the subject. As will be described later, the distance measuring device 3 can have various device configurations such as an infrared sensor and an ultrasonic sensor. However, on the other hand, the processing function of the distance measuring device 3 is a signal processing by a computer system in the image pickup device 1. Can also be realized. Therefore, the distance measuring device 3 such as an infrared sensor or an ultrasonic sensor is not essential, but is called the distance measuring device 3 including signal processing by a computer system or has a distance measuring function.

Also, by using an AF sensor using a two-dimensional area sensor without using an infrared sensor, an ultrasonic sensor, etc., numerical data corresponding to the distance from the AF data for each area to the subject for each area is detected. Then, it may be divided into the same distance groups.
Further, the in-focus position of the image pickup apparatus 1 is sequentially moved by an AF motor (not shown) to acquire image data of the image pickup CCD 23 from the nearest to infinity ∞, and which in-focus position for each area (region) of the image pickup CCD 23 It is also possible to detect whether the distance is within the range and classify the same distance group.

  In the present embodiment, the imaged space from the image capturing apparatus 1 to the distance D1 is the first group space (G1 space), the imaged space belonging to the distances D1 to D2 is the second group space (G2 space), and the distance D2. An imaging space belonging to the far distance D3 is named a third group space (G3 space). The distance D3 may be set to infinity.

FIG. 2 is a configuration diagram of the image pickup apparatus 1 according to the first embodiment of the present invention.
The image capturing apparatus 1 includes a computer system 10 as an example of an image processing apparatus of the present invention, a lens 21, a diaphragm 22, a CCD 23 as an example of an image sensor, a shutter 24, an AF mechanism 25, and an AE mechanism. 26, the distance measuring device 3, and the like.

  The lens 21 is a lens barrel including, for example, a zoom lens and a focus lens. Since this embodiment is not a silver halide camera but an imaging device using a CCD or the like, it goes without saying that the depth of field of the optical system is deep, and in the prior art, a stereoscopic image is obtained like a silver halide camera. It is difficult.

The computer system 10 includes an arithmetic processing unit (CPU) 11 such as a processor, an image memory 12 that stores image data, a memory 13 that temporarily stores processing results of the CPU 11, and a program that stores a program that operates on the CPU 11. A memory 14, a table memory 15 for storing filter coefficients and groups to which each block belongs, an external storage device 16 for storing image data and the like for a long time, and a display such as a liquid crystal display 17 , An interface (I / F) 18 for taking image data from the CCD 23 or taking distance data from the distance measuring device 3, and a bus 19 for connecting the above-described components.
The image memory 12, the memory 13, and the table memory 15 are composed of, for example, a RAM. On the other hand, the memory 13 is composed of a ROM, for example. The external storage device 16 is composed of, for example, a replaceable memory card.

  The CPU 11 of the computer system 10 performs image signal processing, which will be described later, and various control processes of the electronic camera device. Therefore, the CPU 11 cooperates with various means of the present invention, for example, a distance dividing means, a screen dividing means, a filter processing means, an image synthesizing means, or other parts in the image capturing apparatus 1, It functions as part of these means.

The AF mechanism 25 adjusts the focal position by controlling the position of the focus lens in the lens 21 under the control of the CPU 11.
Under the control of the CPU 11, the AE mechanism 26 controls the iris, the opening degree of the diaphragm 22, the speed of the shutter 24, and the like to adjust the illuminance.
The zoom lens in the lens 21 is also controlled under the control of the CPU 11.
The distance measuring device 3 measures the distance to the subject in cooperation with the imaging operation of the CPU 11. The measurement result is input to the computer system 10 via the I / F 17 and stored in the memory 13 or the table memory 15 under the control of the CPU 11.

An outline of normal operation of the image pickup apparatus 1 will be described.
When the user directs the small lens 21 to the imaged space S, the image that has passed through the lens 21 and the aperture 22 with the AF mechanism 25 and the AF mechanism 25 operating under the control of the CPU 11 is input to the CCD 23. It is converted into an electrical signal. The CPU 11 inputs imaging data of the CCD 23 via the I / F 17 and stores it in the image memory 12 and displays it on the liquid crystal display 17. As a result, the user can confirm the image of the imaged space S facing the lens 21 of the image capturing apparatus 1 on the liquid crystal display 17.
When the user presses the shutter 21, the image data detected by the CCD 23 at that time is temporarily stored in the image memory 12, displayed on the liquid crystal display 17, and image data stored in the image memory 12. The CPU 11 performs an image compression operation and the like and stores it in the external storage device 16.
The operation relating to the distance measuring device 3 will be described later.

The entire imaged space S is obtained as a single piece of image data P illustrated in FIG. 3A as a result of imaging with the image imaging device 1 of FIG.
The image illustrated in FIG. 3A is a person image M at a short distance D1, an image H of a house at a medium distance D2, an image T of a tree and an image M of a landscape such as a mountain at a long distance D3. Illustrated.
One piece of image data P corresponds to an image displayed on the liquid crystal display 17 or one piece of image data stored in the image memory 12.

The CPU 11 processes one piece of image data P obtained by the image pickup device 1 by dividing it into a plurality of small blocks (small areas) as shown in FIGS.
In the present embodiment, for example, a very large 12M (mega) pixel (pixel) is used as one piece of image data P, and the entire screen is divided into a plurality of blocks of 1 block (small area) = 64 × 64 pixels. Consider the case.
As illustrated in FIGS. 3A and 3B, the size of one block may be the same size (pixels) in the vertical and horizontal directions, or may be in the vertical and horizontal blocks. The size of one block is determined in consideration of the pixel size of the CCD 23. For example, when the size of the CCD 13 is vertical 512 pixels × horizontal 512 pixels and one block is divided into 64 pixels × 64 pixels, it is divided into 8 vertical blocks × 8 horizontal blocks = 64 small blocks. On the other hand, assuming a very large value of 12 megapixels = 4000 × 3000 pixels as the pixel size, horizontal = 4000 ÷ 64 = 62.5 blocks and vertical = 3000 ÷ 64 = 46.8 blocks. , Horizontal 63 blocks x vertical 47 blocks, and the total number = 63 × 47 = 2961 blocks.

In the present embodiment, a case in which one piece of image data P is divided into a plurality of blocks will be described. Only the predetermined range can be processed as described below.
In this case, the processing described below is not performed on the region other than the target of image enhancement or blurring described below. Alternatively, the image data of a region other than the target for image enhancement or blurring is temporarily set to 0 and the following processing is performed, and after all the processing is completed, the region other than the target for image enhancement or blurring is selected. The original image data can also be synthesized.
In the following description, a case where the entire image data P is divided into a plurality of blocks will be described.

In the plurality of blocks (small regions), image data of the subject in the imaging space S exists. Since the distance from the image pickup apparatus 1 to the position where the subject exists is divided into three groups as described above, the CPU 11 performs group division according to the distance for a plurality of blocks.
As described above, in the present embodiment, the distance between pixel points is not handled for image data, but the distance divided by a group for each block is handled. Therefore, for example, when one block is 64 × 64 pixels, the distance data is reduced to 1/64 × 64 = 1/4096 compared to the case where the distance data is associated with each pixel. There are advantages that the burden is small and the processing speed can be shortened.

  For example, when dividing the distance into four groups, if 2 bits are assigned to one group, 2961 blocks × 2 bits = 5992 bits = 741 bytes. Alternatively, if 3 bits are assigned to one group, 2961 blocks × 3 bits = 8883 bits = 1111 bytes (about 1.12 kbytes). Even for a large image of 12 megapixels, the memory capacity required to divide the distance into groups is about 1.12 kbytes, and the amount of data stored in the table memory 15 of FIG. .

Further, the absolute distance may be stored in the table memory 15 or the like. In this case, assuming that the absolute distance is expressed by about 6 bits (64 levels), the imaging data of 12M (mega) pixels is 2961 × 6 bits = about 2.22 kbytes.
Since it is only necessary to accurately represent the short distance, it is sufficient that the bit depth (bit length) is about 8 bits at the maximum. In that case, 2961 × 8 bits = 2.961 kbytes. Such a memory capacity does not pose any problem as the amount of data stored in the table memory 15.

4A to 4D are diagrams for explaining an image processing method to which the image processing apparatus 1 according to the first embodiment of the present invention is applied, in particular, an image processing method performed by the CPU 11 of the computer system 10.
Assume that the image obtained by the CCD 23 of the image pickup apparatus 1 is illustrated in FIG.
The image illustrated in FIG. 4A illustrates a landscape image F such as a person image M, a house image H, a tree image T, and a mountain, as in the image illustrated in FIG. 3A.
As described with reference to FIGS. 3A and 3B, the screen of one image data is divided into virtual small blocks (small areas), for example, small areas having 64 × 64 pixels as one block. The distance to the subject portion in each block is previously measured by the distance measuring device 3 under the control of the CPU 11. The distance measured by the distance measuring device 3 is stored in the table memory 15. The distance measuring device 3 will be described later.

The CPU 11 classifies such small blocks into a plurality of groups according to preset distances, and marks which distance group each block of the captured image data belongs to.
In the example of FIG. 4A, the small block belonging to the distance D1 group whose person image M is the subject is illustrated in FIG. 4B, and the small block belonging to the distance D2 group whose house image H is the subject is shown in FIG. A small block belonging to the distance D3 group illustrated in FIG. 4C and having a tree image T and a landscape image F such as a mountain as a subject is illustrated in FIG. 4D. Thus, in the present embodiment, the image data can be divided into three image data groups in units of blocks as shown in FIGS. 4B, 4C, and 4D according to the distance.
There is a space inside the portion illustrated in FIG. 4C of the distance D2 group and the portion illustrated in FIG. 4D of the long distance D3 group, and 0 is inserted into the pixel value of these portions. As a result, since there is no data (data = 0) in the image processing in the CPU 11, there is no problem.

  Here, in the above embodiment, an example in which different image data is created corresponding to a plurality of groups and combined last is shown, but the present invention is not limited to this, and there is, for example, one screen. An aspect in which image processing is performed on data belonging to a group, and different image processing is performed on data belonging to another group for the same one screen while image data itself remains 1, and different image processing is performed for each region. It may be. According to this aspect, desired image processing can be performed without performing image composition processing.

  In the first embodiment of the present invention, as described above, one screen of image data is divided into a plurality of blocks (small regions) and separable images, for example, a person image M, a house image H, a tree, etc. The image T and the landscape image F such as a mountain are separated into blocks and separated. The distance measuring device 3 measures the distance for each block in cooperation with the CPU 11. Then, the image data in the separated blocks, for example, the image data of the human image M is divided into groups according to the distance. Thus, in this embodiment, since the distance for each block grouped according to the distance from the image capturing device 1 to the subject is handled, compared to the case where the distance data is associated with each pixel, The distance data is very small.

In this embodiment, the three groups are divided according to the distance. However, the present invention is not limited to this, and the group may be divided into two groups (focus group and background group or focus group and foreground group). Such).
Further, instead of the three types, it may be divided into a large number of groups, or different processing may be performed for each region according to the distance. In this case, the processing becomes very complicated, but more preferable image data can be obtained.

First filter processing (two-dimensional spatial frequency filter) processing)
5A to 5C show filter characteristics applied to the image data of each group divided by the CPU 11 of the computer system 10.
Each of the filters illustrated in FIGS. 5A, 5B, and 5C is referred to as first to third two-dimensional filters (spatial frequency filters) Fl, F2, and F3. In this embodiment, as an example, the first to third two-dimensional filters are used as a two-dimensional filter for a short-distance subject, a two-dimensional filter for a medium-distance subject, and a two-dimensional filter for a long-distance subject. Although Fl, F2, and F3 are illustrated, a number of two-dimensional filters can be prepared by further subdividing the distance, a two-dimensional filter for a subject at a short distance, a two-dimensional filter for a subject at a medium distance or more It is also possible to use only two two-dimensional filters. Hereinafter, a case where the first to third two-dimensional filters Fl, F2, and F3 are used will be described.
5A, 5B, and 5C, the horizontal axis indicates the spatial frequency normalized by 0 to 0.5, and the vertical axis indicates the degree of attenuation of the image data level normalized by 0 to 1. Show. The filter is stored in advance as a filter coefficient in the table memory 15 of the computer system 10 of the image capturing apparatus 1.
For ease of illustration, the filter characteristics are shown in one dimension in FIGS. 5A, 5B, and 5C, but the image data has a two-dimensional spread. A two-dimensional convolution integral or a one-dimensional convolution integral is applied twice vertically and horizontally. Note that the filter processing by the CPU 11 can be easily implemented as a convolution integral in real space.
Although the filter size depends on the image size of the CCD 23 of the image pickup apparatus 1, if the size of the CCD 23 is assumed to be 12M (mega) pixels, the filter size can be realized with about 9 × 9 pixels.
In the present embodiment, these filter characteristics have the following characteristics.

  The first two-dimensional (spatial frequency) filter F1 illustrated in FIG. 5A is a filter applied to the image of the group having the distance D1 illustrated in FIG. That is, it is a filter when applied to a subject located at a short distance from the image pickup device 1, for example, the person image M in FIG. 3A, and has a flat characteristic in a low range, and a little in a mid range to a high range. It is used to raise the image level to enhance the edge of the image and to attenuate the image level slightly to reduce noise in the very high frequency range.

The second two-dimensional (spatial frequency) filter F2 illustrated in FIG. 5B is a filter applied to the image of the group having the distance D2 illustrated in FIG. That is, it is a filter when applied to a subject located at an intermediate distance from the image pickup apparatus 1, for example, the image H of the house in FIG. 3A, and the level is gradually attenuated from the low range to the high range. In particular, it is used to reduce the image level in the high frequency range by blurring the image level in the high frequency range.
As illustrated in the upper part of FIG. 5B, the image data in a portion where no image data exists is set to zero. Therefore, even if the CPU 11 performs the filtering process, there is no relationship and no other influence is exerted.

The third two-dimensional (spatial frequency) filter F3 illustrated in FIG. 5C is a filter applied to the image of the group having the distance D3 illustrated in FIG. That is, it is a filter when applied to a subject located at a long distance from the image pickup device 1, for example, a landscape image F such as a mountain in FIG. 3 (A). It is used to further reduce the image level over the range and blur the image level over the low to mid range.
As illustrated in FIG. 5C, the image data in a portion where no image data exists is set to zero. Therefore, even if the filter processing in the CPU 11 is performed, there is no relationship and no other influence is exerted.

An example of a method of applying filter processing to the images of each divided distance group will be described with reference to FIGS.
FIG. 6 is a diagram for explaining the filter processing for the divided image data. FIG. 7 is a flowchart showing the contents of filter processing of the CPU 11.
Image data stored in the image memory 12 from the CCD 23 via the I / F 18 in the computer system 10 illustrated in FIG. 2 is virtually divided into small blocks (areas) in the image memory 12 as shown in FIG. It is divided. The CPU 11 takes in the distance division group for each block as table data together with the distance relationship of these small blocks, and stores them in the table memory 15. The CPU 11 also stores the filter coefficients of the three types of two-dimensional filters F1 to F3 in the table memory 15 in advance.
Based on such table data, the perspective group to which each small block of the image data stored in the image memory 12 belongs is labeled.

First filter processing (two-dimensional spatial frequency filter processing)
A process in which the CPU 11 applies a two-dimensional (spatial frequency) filter to image data of individual blocks virtually divided with reference to FIG.
The first filter process performs a filter process corresponding to the distance based on the filter characteristics illustrated in FIGS.
The small blocks shown in FIG. 6 are numbered by 1 at the upper left and by increasing the number in the horizontal direction.
In step 1, the CPU 11 first selects the small block number 1, and in step 2, the CPU 11 determines the group to which the block belongs based on the data in the table memory 15. In step 3, if the data is, for example, 3, the CPU 11 selects the coefficient of the third two-dimensional filter F3 and retrieves the coefficient of the third two-dimensional filter F3 from the table memory 15. In step 4, The CPU 11 applies the coefficient of the third two-dimensional filter F3 to the image data of the small block 1 (filter processing). Specifically, for example, if the small block is a 32 × 32 pixel and the third two-dimensional filter F3 is a 5 × 5 pixel (convolution) filter, the CPU 11 can perform filter processing by these convolution operations. The CPU 11 writes the filtered result in the image memory 12.
In step 5, the CPU 11 next selects the second small block. In step 6, the CPU 11 determines whether or not processing has been performed for all the blocks. If selection for all the blocks is not completed, the table to which the block belongs as described above with reference to steps 2 to 4 is displayed. With reference to the memory 15, the coefficient of the corresponding two-dimensional filter is extracted and applied to the image data of the corresponding block.
The CPU 11 calculates these one after another and performs two-dimensional filter processing on the image data of all small blocks.
The intermediate processing result is created in the image memory 12, and the CPU 11 stores the final result subjected to the two-dimensional filter processing in the external storage device 16.

FIG. 8 is a diagram illustrating processing in which the CPU 11 again synthesizes the filter processing results obtained by applying (applying) the first to third two-dimensional (spatial frequency) filters F1 to F3 to form one image. .
In the image composition, the CPU 11 simply adds the three image processing results obtained by applying the first to third two-dimensional filters F1 to F3 corresponding to the image data of the corresponding block.
FIG. 9A shows an image obtained as a result of the synthesis process by the CPU 11.
Here, it is needless to say that a similar image can be obtained by performing different filter processing for each region of one image instead of the synthesis processing.

The second filter processing CPU 11 further performs the first filter processing exemplified below. The second filter processing is processing for image enhancement or image blurring.

Second filter processing (first example)
The CPU 11 performs the following second filter process on the image of FIG.
For example, a high pass filter (HPF) is applied to the image data of the short distance D1 group, and low pass filters (LPF) 2 and LPF 3 are applied to the images of the medium distance D2 and the long distance D3, respectively. The purpose is to further enhance the sense of perspective by applying an LPF having stronger low-pass characteristics to the image at the distance D3. As a result, the person image M at the short distance D1 is highlighted more clearly, and the landscape image F such as a mountain is blurred and a more blurred image is obtained.
9B performs the second filtering process on the composite image illustrated in FIG. 9A, and the group of the distance D1 in which the person image M is captured is closest to the image H of the house. The other image in which the image T of the tree and the landscape image F located in the background of a person such as a mountain are captured is an example determined to be behind (far) from the distance D1.
Thus, the second filter process is a process for enhancing or blurring an image.
In this way, a three-dimensional image is obtained.

Second filter processing (second example)
On the other hand, the processing result illustrated in FIG. 10B will be described for the composite image of the imaging result illustrated in FIG. In this example, a landscape in which the distance D1 group for the tree image T is the forefront, the person image M is located at a distance D2 farther than the distance D1, and is located in the background of a person such as a mountain at a further distance D3. The image F is located. Here, a processing example is shown in which the blur is enhanced more by emphasizing the person image M at the distance D2 and blurring the others.
In the CPU 11, HPF is applied to the image (person image M) of the D2 group, and LPF2 and LPF3 are applied to the image T of the tree of the D1 group and the landscape image F such as a mountain of the D3 group, respectively. The LPF 3 stronger than the LPF 2 is applied to the landscape image F such as a mountain of the group (D3) with a long distance. This makes you feel farther away. Further, the person image M is further emphasized by blurring the tree image T in front.
For the distance D1 group of the tree image T, LPF4 having a lower frequency characteristic than LPF3 applied to the image data of the D3 group can be applied. As a result, the near-field image (house image H) is further blurred, and the subject of the person image M appears.
In this way, a three-dimensional image is obtained.

Second filter processing (third example)
In the example described above, the processing of the image capturing device 1 (CPU 11) when the distance is divided for each small block by the distance measuring device in the image processing device has been described.
A method for easily blurring an image by the method shown in FIGS. 11A and 11B will be described.
FIGS. 11A and 11B show a third example of the second filter processing in which an operator (user) of the image capturing apparatus 1 roughly specifies a region on the image capturing screen, and within and outside that region. It is a figure which shows the method of changing a filter.
As shown in FIG. 11A, for example, with respect to an image displayed on the liquid crystal display 17, for example, a photographer (user) operates a manipulator (not shown) to easily form a rectangular area (block) in the field of view. A is displayed, and the photographer operates the manipulator to move the position so as to surround the subject. Then, the CPU 11 of the image capturing apparatus 1 causes two types of two-dimensional filters having different filter characteristics to act on the designated area (block) and other areas, thereby producing blur.
In this method, the depth (distance) is discriminated only in the two groups of the designated area and the other areas. However, when the person image M is designated as the area, the person image M is emphasized and the image in the area not designated is designated. A blur processing operation with blurred data is possible.
In this way, a three-dimensional image is obtained.
In the method shown in FIG. 11A, it is not necessary to divide the space into small blocks and measure the distance of each block using the distance measuring device 3, and it is specified from one piece of image data that is arbitrarily captured. Since the blur processing can be applied to the two regions separately from the other regions, the image capturing apparatus 1 is very simple.

Second filter processing (fourth example)
Similar to the method illustrated in FIGS. 11A and 11B, a method for easily blurring an image by the method shown in FIGS. 12A and 12B as the second filter processing (fourth example). State.
FIGS. 12A and 12B show, as the second filter processing (fourth example), the person (user) of the image capturing apparatus 1 roughly designates an area on the imaging screen, and within and outside the area. It is a figure which shows the method of changing a filter.
In FIG. 12A, the captured image is divided into blocks as indicated by broken lines using a manipulator (not shown) in the liquid crystal display 17, and the area B designated by the CPU 11 based on the color histogram is displayed. An example will be shown in which the edge area relationship is determined in more detail.
In FIG. 12A, the photographer (user) operates the manipulator on the liquid crystal display 17 to divide the entire image including the region B surrounding the person image M into small blocks. The CPU 11 calculates a histogram of the first color difference Cb and the second color difference Cr for each small block, and compares it with adjacent small blocks. When the distribution of the histogram is similar, the CPU 11 determines the edge in more detail by determining that the block is also a part of the image data area of the human image M.

A processing example of the method is shown in FIG. In FIG. 13, the color difference histograms of the first color difference Cb and the second color difference Cr are H1 and H2, and the CPU 11 obtains peak values in the graphs of H1 and H2, respectively.
If the peak positions in the first and second color difference histograms H1 and H2 are values close to each other, the CPU 11 determines that the small blocks H1 and H2 include the same image data. At this time, it is not always necessary for the CPU 11 of the image pickup apparatus 1 to scan the entire image stored in the image memory 12, and only the area including the area designated by the user and including the outside is divided into small blocks. A histogram may be calculated.

  12 and 13, similarly to the method shown in FIG. 11, a mechanism for dividing the image data 1 screen into small blocks and measuring the distance of each block, for example, the distance measuring device 3 is unnecessary. Since blur processing can be applied to a single region of image data that is arbitrarily captured and separated into two regions, a region designated by the user and other regions, the processing and configuration of the image capturing device 1 become very simple. The advantage is shown.

Second Embodiment A case where a distance measuring device 3 is used will be described as a second embodiment of the image pickup apparatus of the present invention.

Second embodiment (first example)
FIG. 14 is a diagram showing the processing contents of the first example of the second embodiment of the present invention, in which an infrared (IR) transceiver is used for the distance measuring device 3 of the image pickup device 1. The receiver can use a small CCD line sensor. The IR transmitter is small and can be scanned at high speed.
In this example, the distance D of the imaged space S is measured at high speed by an IR transmitter and a sensor that detects infrared rays, and the CPU 11 performs grouping according to the distance for each block.
The distance D from the image capturing apparatus 1 to each block is calculated by the CPU 11 using the formula shown in FIG. 14 by the triangulation method.

For example, the IR transmitter may be operated at a scanning frequency of about 10 kHz. Assuming that scanning is performed at 10 kHz (100 μs per block), the time for scanning the entire image of the imaged space S is 2961 blocks × 100 μs = 0.1 seconds even in the case of 12 M (mega) pixels. Time delay due to distance measurement is not a problem.
In this way, the distance to the subject in the imaging space S can be measured at high speed using the distance measuring device 3. The distance measurement result is stored in the table memory 15 by the CPU 11 and used for the above-described processing. As described above, since the distance data by the distance measuring device 3 is for each block, the memory capacity of the table memory 15 may be small.

Second embodiment (second example)
A second example of the second embodiment of the image pickup apparatus of the present invention will be described with reference to FIG.
FIG. 15 shows another IR scanning method. In this method, as a distance measuring device 3, a vertical bar-like (linear) IR is radiated to the subject space S illustrated in FIG. 1, and each block as shown in FIG. Each distance D is calculated.
In this method, the distance measurement time is faster because only one horizontal scan is required. In this method, a small surface sensor may be used as the sensor.
Note that the CCD 23 used for imaging may be used without using the surface sensor, but in this case, a mechanism (not shown) that moves so as not to use the IR filter only during distance measurement is used.

Second embodiment (third example)
A third example of the second embodiment of the image pickup apparatus of the present invention will be described with reference to FIG.
FIG. 16 is a diagram illustrating an example in which an ultrasonic transceiver is used as the distance measuring device 3. The ultrasonic transmitter sends out an ultrasonic pulse. In this example, since the distance is measured, a large output is not necessary, and the transmitter can be downsized and can be scanned at high speed.
In the third example of the second embodiment, the distance D of the imaging space S is measured at high speed by an ultrasonic pulse transceiver, and the distances are grouped for each block.
Since the sound speed is about 330 m / second, the scanning frequency is about several tens of Hz. Assuming that the sound velocity is 330 m / second and the pulse propagation time (reciprocation) from the image capturing device 1 to the object to be imaged in the image capturing space S is dt, the distance D from the image image capturing device 1 to the subject in the image capturing space S is It can be obtained as follows. The propagation distance delta per unit time 1 ms and the distance D from the image capturing apparatus 1 to the subject are as follows.

delta = 330 m / 1000 (ms) = 33 cm / 1 ms
D = 330 mx (dt / 2)

  For example, when dt = 10 ms, the measured time is a round trip time, so the one-way propagation time is dt / 2 = 5 ms. The distance traveled by the sound wave in 5 ms is as follows.

D = (330 m / s) × 5 ms
= 330mx (5/1000) sec
= 1.65m

Depending on how much the effective distance (maximum measurement distance) D of the subject is to be maximized, it is determined by the measurement time required per block. However, if the maximum measurement distance D is about 5 m at maximum, the above value is sufficient. It is considered that the blur effect will not be necessary if the maximum measurement distance is longer.
If the maximum measurement distance D is 5 m, the sound wave propagation time is 5 m / 330 m = 15 ms. Assuming an extra time before photographing of 1 second, it can be divided into 1 second ÷ 0.015 seconds = 66 blocks, and has sufficient accuracy.
When the ultrasonic wave is diffused as compared with the infrared ray, in this case, the number of pixels of the small block that divides the entire screen may be set larger than that when the distance is measured using the infrared ray.

Third Embodiment The distance information can be obtained by performing the calculation by the CPU 11 using not only the single shooting but also the image picked up twice by slightly changing the shooting position without using the distance measuring device 3.
In this case, imaging is performed twice with different imaging conditions to obtain two pieces of image data.
As a specific method, for example, the user sets the first mode and images the lens 21 in the first direction, and the user sets the second mode and changes the direction of the lens 21 to perform imaging. . Alternatively, when the user sets the first mode, the CPU 11 images the lens 21 at the first zoom ratio, and when the user sets the second mode, the CPU 11 captures the lens 21 at the second zoom ratio. To do.
The CPU 11 calculates distance information using the two pieces of image data obtained in this way.
Further, by capturing an appropriate number of times according to the focal length instead of twice, finer distance information can be acquired, and images can be divided into a plurality of groups according to the distance.
This embodiment has an advantage that it is not necessary to provide the distance measuring device 3.

Fourth Embodiment As a fourth embodiment of the present invention, referring to FIGS. 17 to 21, a three-dimensional information of the imaged space S is obtained by an autofocus (AF) device using a bandpass filter, and a blur filter is obtained. Describes how to make the function work.
FIG. 17 shows an example of image space data obtained from an image sensor such as the CCD 23. The size of the imaging data from the CCD 23 that has imaged the imaging space S is 2400 pixels horizontally and 1600 pixels vertically, and this one screen is virtually shown in FIG. 18, 1 block = 16 pixels vertically × 16 pixels horizontally. Divide into multiple blocks.
The CPU 11 illustrated in FIG. 2 has a large attenuation between the first frequency f1 and the second frequency f2 in the horizontal direction (spatial frequency direction) shown in FIG. 19 with respect to the pixel data of this small region (block). Apply (apply) a bandpass filter. The characteristics of this band pass filter are stored in the table memory 15, for example.
The CPU 11 adds the results of applying the bandpass filter. Then, when the image in the small block comes into focus, the high frequency component of the spatial frequency of the image increases. Therefore, the addition result increases as the focus position is approached as shown in FIG. The amplitude is maximized at the position just in focus. On the other hand, when the focal position shifts again, the output (amplitude) decreases.

As shown in FIG. 17, one screen of image data is virtually divided into small blocks and numbered from the upper left to the lower right. In the case of FIG. 17, 240 virtual small blocks are formed.
With reference to FIG. 21, how the CPU 11 groups each small block in the space will be described. In this example, consider a case where the distance R from the image capturing apparatus 1 is roughly divided into three groups.
In step 11, the CPU 11 sets the position of the lens 21 to the farthest focal position. In step 12, the CPU 11 calculates autofocus output data from the AF mechanism 25 for all small blocks of one screen, and stores 240 values in the memory 13. The autofocus output data indicates the distance to the subject. This value is AF data at the first distance R1.

  In step 13, the CPU 11 moves the position of the lens 21 to the focal length R2. At that position, autofocus output data from the AF mechanism 25 is calculated again for all small blocks on the screen, and 240 values are stored in the memory 13. This value is AF data at the second distance R2.

  In step 14, the CPU 11 performs the above-described processing up to the distance R3. That is, the CPU 11 next moves the position of the lens 21 to the focal length R3. At that position, the autofocus output data from the AF mechanism 25 is calculated again for all small blocks on the screen, and 240 values are stored in the memory. This value is AF data at the distance R3.

In step 15, after completing the calculation for the distances R = R1 to R3, the CPU 11 calculates the magnitude relation of the distances R1 to R3 for each small block, and divides each block into a group that gives the largest value. To do. For example, if Rl>R2> R3 for the small block 1, the CPU 11 classifies the small block 1 as belonging to the group 1. If the small block 2 is R2>Rl> R3, the CPU 11 classifies the small block 2 as belonging to the group 2.
Thus, by calculating the magnitude relationship for all 240 small blocks, the CPU 11 divides each small block into groups 1 to 3.
Of course, if R1 to R3 are further refined to R1 to R5, the CPU 11 can classify the distances more finely.

On the other hand, such small block AF calculation does not need to be directly shown (recognized) by the operator (user) of the image pickup apparatus 1, for example, an electronic camera device. 17, an AF window (200) used by the user as shown in FIG. The user brings this window to the place where the subject is most focused. In the image pickup device 1, for example, the CPU 11 of the electronic camera device, distance calculation is performed using a window that is finer than such an AF window. Therefore, it is not necessary to separately calculate AF window data used by the user. The CPU 11 can obtain the AF value of the small block by adding the AF values of the small blocks belonging to the AF window (200).
However, in order to obtain the in-focus position more finely, the CPU 11 obtains the in-focus position a little more finely based on the approximate position data of R1 to R3 in the vicinity of the AF window (200) obtained by the AF calculation of the small block. What is necessary is just to adjust a lens position.

Here, in the first to fourth embodiments, the image data is divided into a plurality of groups, and after image processing is individually performed, a desired image is obtained by combining them. The present invention is not limited to this, and image processing may be performed simultaneously or sequentially for each group of image data to obtain a desired image without being combined.
Needless to say, it is applicable not only to still images but also to moving images.
In the present embodiment, the three groups are divided according to the distance. However, the present invention is not limited to this, and the group may be divided into two groups (focus group and background group or focus group and foreground group). Such).
Further, instead of the three types, it may be divided into a large number of groups, or different processing may be performed for each region according to the distance. In this case, the processing becomes very complicated, but more preferable image data can be obtained.

  In carrying out the present invention, the various embodiments described above can be combined as appropriate.

  As described above, according to the present embodiment, even in an imaging apparatus using a lens with a large depth of focus and difficult to blur, by measuring the front-rear distance of each part of the subject in the subject space, By dividing the subject space into three-dimensional regions and applying filters having different frequency characteristics to each region, an image with an arbitrary blur can be created.

  Further, according to the present embodiment, for example, in an image pickup apparatus using a high-pixel CCD, an image with a stereoscopic effect can be provided even when the obtained image has a poor stereoscopic effect.

  As described above, according to the present invention, it is possible to provide an unprecedented image processing apparatus, image capturing apparatus, and image processing method.

FIG. 1 is an explanatory diagram showing an image pickup apparatus which is a first embodiment of an image processing apparatus of the present invention and an image pickup method using the image pickup apparatus. FIG. 2 is a configuration diagram of the image pickup apparatus 1 according to the first embodiment of the present invention. FIGS. 3A and 3B are diagrams illustrating an example of image data captured by the image capturing apparatus according to the present embodiment and the screen being divided into a plurality of blocks. 4A to 4D illustrate an image processing method to which the image processing apparatus 1 according to the first embodiment of the present invention is applied, particularly an image processing method performed by the CPU of the computer system illustrated in FIG. FIG. FIGS. 5A to 5C are diagrams showing filter characteristics applied to each distance group of filter processing performed in the CPU of the computer system illustrated in FIG. FIG. 6 is a diagram for explaining the filter processing for the image data divided in FIG. FIG. 7 is a flowchart showing the processing contents of the CPU illustrated in FIG. FIG. 8 is a diagram illustrating processing in which the CPU 11 again combines the filter processing results obtained by applying (applying) the first to third filters F1 to F3 in the first embodiment to form one image. FIG. 9A is a diagram showing an image of the result of the synthesis process by the CPU, and FIG. 9B is an example in which the first example of the second filter process is performed on the synthesized image illustrated in FIG. 9A. FIG. 10A and 10B are diagrams illustrating an example in which the second example of the second filter processing is performed. FIGS. 11A and 11B are diagrams illustrating an example in which a third example of the second filter processing is performed. 12A and 12B are diagrams illustrating an example in which a third example of the second filter processing is performed. FIG. 13 is a flowchart showing a third example of the second filter process. FIG. 14 is a diagram showing the processing contents of the first example of the second embodiment of the present invention, which is an example in which an infrared (IR) transceiver is used for the distance measuring device of the image pickup apparatus. FIG. 15 is a diagram showing an example in which a vertical bar-shaped IR as a distance measuring device is emitted to the subject space S illustrated in FIG. FIG. 16 is a diagram illustrating an example in which an ultrasonic transceiver is used as the distance measuring device 3. FIG. 17 shows image space data obtained from an image sensor when a blur filter function is activated by obtaining spatial three-dimensional information with an autofocus (AF) apparatus using a bandpass filter as a fourth embodiment of the present invention. It is a figure which shows the example of. FIG. 18 is a diagram for explaining that AF window processing is performed on the image data illustrated in FIG. FIG. 19 is a characteristic diagram of the AF bandpass filter. FIG. 20 is a diagram illustrating the relationship between lens movement and AF processing results. FIG. 21 is a flowchart showing the process of the fourth embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image pick-up device 10 ... Computer system
11 ... CPU, 12 ... image memory, 13 ... memory
14 ... Program memory, 15 ... Table memory
16 ... External storage device, 17 ... Liquid crystal display, 18 ... I / F
DESCRIPTION OF SYMBOLS 19 ... Bus 21 ... Lens, 22 ... Aperture, 23 ... CCD, 24 ... Shutter 25 ... AF mechanism, 26 ... AE mechanism 3 ... Distance measuring device

Claims (16)

Image dividing means for dividing one screen of image data into a plurality of areas (image data of one screen);
Filter means for applying different spatial frequency filters to the plurality of regions according to the distance to the subject;
An image processing apparatus comprising: Imaging means for capturing an image of the imaged space;
Image dividing means for dividing the captured image data of one screen into a plurality of regions (image data of one screen);
Distance detecting means for detecting the distance to the subject in the image data of each divided area;
Filter means for applying different spatial frequencies to the plurality of regions according to the distance to the subject;
An image pickup apparatus comprising: Image dividing means for dividing one screen of image data into a plurality of areas;
Distance dividing means for dividing the plurality of divided areas into at least three distance groups;
A first two-dimensional spatial frequency filter that emphasizes edges of image data;
A second two-dimensional spatial frequency filter for reducing the high frequency of the image data;
A third two-dimensional spatial frequency filter that further reduces the high frequency of the image data from the characteristics of the second spatial frequency filter;
The first two-dimensional spatial frequency filter is applied to the image data of each region of the divided first distance group, and the second two-dimensional spatial frequency filter is applied to the image data of each region of the divided second distance group. Filter processing means for applying the third two-dimensional spatial frequency to each region of the divided third group of distances,
An image processing apparatus comprising: Imaging means for imaging a subject that has passed through the optical system;
Image dividing means for dividing one screen of image data obtained by the imaging means into a plurality of regions;
Distance dividing means for dividing the plurality of divided areas into at least two distance groups;
A first two-dimensional spatial frequency filter that emphasizes edges of image data;
A second two-dimensional spatial frequency filter for reducing the high frequency of the image data;
The first two-dimensional spatial frequency filter is applied to the image data of each area of the divided first group, and the second two-dimensional spatial frequency filter is applied to the image data of each area of the divided second group. Filtering means for
An image pickup apparatus comprising: A third spatial frequency filter for further reducing the high frequency of the image data from the characteristics of the second spatial frequency,
The distance dividing means divides the plurality of divided areas into three distance groups,
The filter processing means applies the first two-dimensional spatial frequency filter to the image data of each area of the divided first distance group, and applies the second to the image data of each area of the divided second distance group. Applying the second two-dimensional spatial frequency filter and applying the third two-dimensional spatial frequency to each region of the divided third distance group.
The image imaging device according to claim 4, wherein Imaging means for imaging a subject that has passed through the optical system;
Image dividing means for dividing one screen of image data obtained by the imaging means into a plurality of regions;
Distance dividing means for dividing the plurality of divided areas into at least two distance groups;
A first two-dimensional spatial frequency filter that emphasizes edges of image data;
A second two-dimensional spatial frequency filter for reducing the high frequency of the image data;
The second two-dimensional spatial frequency filter is applied to the image data of each area of the divided first group, and the first two-dimensional spatial frequency filter is applied to the image data of each area of the divided second group. Filter processing means for applying and separating the second two-dimensional spatial frequency to each region of the divided third group of distances;
An image pickup apparatus comprising: A third two-dimensional spatial frequency filter that further lowers a high frequency than the characteristics of the second spatial frequency filter;
The filter processing means applies the third two-dimensional spatial frequency filter to the image data of each area of the divided first group, and applies the first 2 to the image data of each area of the divided second group. Apply a dimensional spatial frequency filter,
It is characterized by
The image pickup device according to claim 6. Further comprising distance information generating means for generating distance information corresponding to the focal length of each of the plurality of regions;
The distance classification means includes
Based on the distance information generated by the distance information generating means, the image data area of the close subject is divided into a first group,
The area of the image data of the intermediate subject is divided into the second group,
Divide the image data area of the distant subject into a third group,
It is characterized by
The image capturing device according to claim 4. Distance information generating means for generating distance information corresponding to the focal length of each of the plurality of regions obtained by dividing the one screen;
Indicating means for indicating at least one of the plurality of areas;
In-focus distance information obtaining means for obtaining distance information of the area instructed by the instructing means as in-focus distance information from the distance information generating means, and
The distance classifying unit includes an area of image data having distance information substantially matching the in-focus distance information acquired by the in-focus distance information acquiring unit, or distance information within a predetermined distance range including the in-focus distance information. The image data area is classified as a short-distance subject image data area or an intermediate-distance subject image data area.
It is characterized by
The image capturing device according to claim 7. Distance information generating means for generating distance information corresponding to the focal length of each of the plurality of regions obtained by dividing the one screen;
Indicating means for indicating at least one of the plurality of areas;
In-focus distance information obtaining means for obtaining distance information of the area instructed by the instructing means as in-focus distance information from the distance information generating means, and
The distance classification means includes
An area having distance information substantially coincident with the in-focus distance information acquired by the in-focus distance information acquisition means or an area having distance information within a predetermined distance range including the in-focus distance information is classified into a first group. ,
An area having distance information within a predetermined distance range from the focusing distance information acquired by the focusing distance information acquisition means, or distance information within a predetermined distance range from a predetermined distance range including the focusing distance information. Divide the area as a second group,
An area having distance information outside the predetermined distance range from the focusing distance information acquired by the focusing distance information acquisition means, or distance information outside the predetermined distance range from a predetermined distance range including the focusing distance information Divide the region having
It is characterized by
The image capturing device according to claim 4. Distance information generating means for generating distance information corresponding to the focal length of each of the plurality of regions obtained by dividing the one screen;
Indicating means for indicating at least one of the plurality of areas;
In-focus distance information obtaining means for obtaining distance information of the area instructed by the instructing means as in-focus distance information from the distance information generating means, and
The distance classification means includes
An area having distance information substantially coincident with the in-focus distance information acquired by the in-focus distance information acquisition means or an area having distance information within a predetermined distance range including the in-focus distance information is classified into a second group. ,
A region having distance information closer to the focus distance information acquired by the focus distance information acquisition means or a region having distance information closer to a predetermined distance range including the focus distance information is classified as a first group. ,
A region having distance information farther than the focus distance information acquired by the focus distance information acquisition means or a region having distance information farther than a predetermined distance range including the focus distance information is defined as a third group. Divide,
It is characterized by
The image capturing device according to claim 4. Distance information generating means for generating distance information corresponding to the focal length of each of the plurality of areas obtained by dividing the one screen;
Indicating means for indicating at least one of the plurality of areas;
In-focus distance information acquisition means for obtaining distance information of the area instructed by the instruction means from the distance information generation means as in-focus distance information;
Mode setting means for setting the first shooting mode and the second shooting mode;
When the first mode is set by the mode setting means, the distance classification means is
An area having distance information substantially coincident with the in-focus distance information acquired by the in-focus distance information acquisition means or an area having distance information within a predetermined distance range including the in-focus distance information is classified into a first group. ,
An area having distance information within a predetermined distance range from the focusing distance information acquired by the focusing distance information acquisition means, or distance information within a predetermined distance range from a predetermined distance range including the focusing distance information. Divide the area as a second group,
An area having distance information outside the predetermined distance range from the focusing distance information acquired by the focusing distance information acquisition means, or distance information outside the predetermined distance range from a predetermined distance range including the focusing distance information When the second mode is set by the mode setting unit, the allocating unit includes:
An area having distance information substantially coincident with the in-focus distance information acquired by the in-focus distance information acquisition means or an area having distance information within a predetermined distance range including the in-focus distance information is classified into a second group. ,
A region having distance information closer to the focus distance information acquired by the focus distance information acquisition means or a region having distance information closer to a predetermined distance range including the focus distance information is classified as a first group. ,
A region having distance information farther than the focus distance information acquired by the focus distance information acquisition means or a region having distance information farther than a predetermined distance range including the focus distance information is defined as a third group. Divide,
It is characterized by
The image capturing device according to claim 4. Pixel extraction means for extracting a pixel having the same color component as the main color component of the one area instructed by the instruction means from an adjacent area of the one area;
The distance classifying unit classifies the pixels extracted by the pixel extracting unit into the same filter as the first region.
It is characterized by
The image imaging device according to claim 9. The distance information generation means generates distance information corresponding to the focal length of each of the plurality of regions by a distance measurement sensor, or changes the focal point using the imaging means to shoot a plurality of subject images. The image imaging device according to any one of claims 8 to 13, wherein distance information corresponding to a focal length of each of the plurality of regions is generated from image data of the subject image. An image dividing step for dividing one screen of image data into a plurality of areas;
A distance dividing step of dividing the distance to the subject in the plurality of regions into at least three distance groups;
Applying a first spatial frequency filter for edge enhancement to image data of each region of the first distance group, applying a second spatial frequency filter for lowering high frequency to image data of each region of the second distance group; A filter processing step of applying a third spatial frequency filter having a higher frequency drop than the characteristics of the second spatial frequency filter to the image data of each region of the three-distance group;
An image processing method comprising: An imaging step of capturing an image of a subject that has passed through the optical system;
An image dividing step of dividing the captured image data one screen into a plurality of regions;
A distance dividing step of dividing the distance to the subject in the plurality of regions into at least three distance groups;
The image data of each region of the second distance group is applied to the image data of each region of the first distance group by applying the first two-dimensional spatial frequency filter for edge enhancement or the second two-dimensional spatial frequency filter for lowering the high frequency. 2nd spatial frequency filter of 2nd spatial frequency reduction or the 1st spatial frequency filter of edge emphasis is applied to each area image data of the 3rd distance group, The filtering step to apply,
An image capturing method comprising:

Images