JP2012060540A - Image processing apparatus, image processing method, and program for image processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of generating a blurred image while suppressing memory consumption in a short period.SOLUTION: The image processing apparatus is provided with image generation means and composition means. The image generation means generates a plurality of blurred source images to which blurring processing is applied using different blurring intensities, from a first reduced image obtained by reducing a first image of a subject taken by focusing at a first focal position. When composing a blurred image obtained by applying blurring processing to a first pixel being a pixel included in the first image, the composite means determines a pixel value after applying the blurring processing to the first pixel by use of a second pixel being a pixel of the blurring source image corresponding to the position of the first pixel.

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program that generate an image obtained by performing blurring processing on part or all of an image.

2. Description of the Related Art Conventionally, a technique for generating an image obtained by blurring a partial area or the entire image captured by a camera or the like is known.
For example, in connection with the above technique, there is known a soft focus method that can obtain a soft focus image in a pseudo manner by combining an in-focus image and an out-of-focus image with one touch without using an adapter lens. ing.

  Also, a partial image cut out from an in-focus image and a partial image cut out from the image captured at each focus position are combined to synthesize the image and restricted by the distance relationship between the subject and the background. A camera device that obtains an image in which only the background is blurred is known.

  Also known is a digital camera that generates a subject-enhanced image that includes a defocused image equivalent to an image that is captured when the background region is out of focus by performing smoothing processing such as filtering processing on the background region. It has been.

  In addition, an image of a subject area photographed at a lens position is focused on the subject area, and an image of a background area photographed at a macro position is fitted on the background area. A digital camera for synthesizing an image is known.

Japanese Patent Laid-Open No. 06-350889 Japanese Patent Laid-Open No. 2005-039680 JP 2008-079193 A JP 2009-284404 A

1 and 2 show an example of processing in which the image processing apparatus generates a blurred image.
In FIG. 1, the image processing apparatus generates a blurred image 130 by performing a blurring process on the input image 110 while changing the degree of blurring according to the blur information 120 that includes the degree of local blurring as information. .

  The blurring process can be performed by, for example, averaging pixel values of pixels included in the filter 111. In this case, the image processing apparatus can change the degree of blur by changing the size of the filter 111 according to the blur information 120.

  However, when the blurring process shown in FIG. 1 is performed, a large load is applied to the image processing apparatus. Further, since the blurring process is performed for each pixel included in the input image 110, the image processing apparatus is further burdened. As a result, the processing executed by the image processing apparatus becomes heavy, and there is a disadvantage that it takes time for blurring processing.

  In FIG. 2, the image processing apparatus generates the blurred image 220 by combining the input image 110 and the original blur image 211, 212, 213, or 214 according to the blur information 120. The blurred original images 211, 212, 213, and 214 are images that are blurred to a different extent with respect to the input image 110.

  In this case, the blurred original images 211, 212, 213, and 214 must always be held in the memory. Therefore, the blurring process shown in FIG. 2 has a drawback that the amount of memory consumption increases.

  The present image processing apparatus has been made in view of the above-described problems, and the problem to be solved is an image processing apparatus and an image processing method capable of generating a blurred image while suppressing memory consumption in a short time. And an image processing program.

According to one aspect of the image processing apparatus, the image processing apparatus includes the following components.
The image generation means includes a plurality of original blur images to which blur processing is applied using different blur intensities from a first reduced image obtained by reducing the first image obtained by focusing on the first in-focus position and capturing an image of the subject. Is generated.

  The synthesizing unit may synthesize the blurred original image corresponding to the position of the first pixel when synthesizing the blurred image obtained by applying the blurring process to the first pixel that is a pixel included in the first image. The pixel value after applying the blurring process to the first pixel is determined using the second pixel which is a pixel.

  According to the present image processing apparatus, it is possible to provide an image processing apparatus, an image processing method, and an image processing program capable of generating a blurred image while suppressing memory consumption in a short time.

It is a figure which shows the example of the process which an image processing apparatus produces | generates a blurred image. It is a figure which shows the example of the process which an image processing apparatus produces | generates a blurred image. It is a figure explaining the image processing apparatus which concerns on a 1st Example. It is a figure explaining the outline | summary of the blurring process by the image processing apparatus which concerns on a 2nd Example. It is a figure explaining the flow of the blurring process by the image processing apparatus which concerns on a 2nd Example. It is a flowchart which shows the outline | summary of the blurring process which the image processing apparatus which concerns on a 2nd Example performs. It is a figure explaining the blur map which concerns on a 2nd Example. It is a flowchart which shows the production | generation process of the blurring map which concerns on a 2nd Example. It is a flowchart which shows the production | generation process of the blurred image by the image processing apparatus which concerns on a 2nd Example. It is a figure explaining the flow of the image processing by the image processing apparatus which concerns on a 3rd Example. It is the figure which expanded the screen size of the blurring original image shown in FIG. It is a figure explaining the outline | summary of the blurring process which the image processing apparatus which concerns on a 3rd Example performs. It is a figure explaining selection of the pixel concerning the 3rd example. It is a figure explaining the 6-point interpolation process which concerns on a 3rd Example. It is a flowchart which shows the production | generation process of the blurring image by the image processing apparatus which concerns on a 3rd Example. It is a figure which shows the specific structural example of the image processing apparatus which concerns on the 1st-3rd Example. It is a figure which shows the structural example of an apparatus provided with the image processing apparatus which concerns on the 1st-3rd Example.

  Hereinafter, an example of this embodiment will be described with reference to FIGS. Note that the embodiments described below are merely examples, and are not intended to exclude various modifications and technical applications that are not explicitly described below. In other words, the present embodiment can be implemented with various modifications such as combining the embodiments without departing from the spirit of the present embodiment.

FIG. 3 is a diagram illustrating the image processing apparatus 300 according to the present embodiment.
The image processing apparatus 300 includes an image generation unit 310 and a synthesis unit 320.
The image generation unit 310 applies a plurality of blur processings using different blur intensities from the first reduced image 340 obtained by reducing the first image 330 obtained by focusing on the first in-focus position and capturing the subject. A blur original image 350 is generated.

  The first image 330 and the first reduced image 340 may be acquired from, for example, an imaging device (not shown) that generates the first image 330 and the first reduced image 340, or may be an imaging device (not shown). You may read from the memory | storage device memorize | stored.

  For example, the image generation unit 310 calculates an average value of pixel values within a certain range centered on the pixel to be blurred among the pixels included in the first reduced image 340, and uses the calculated average value to calculate the blur target. By replacing the pixel value of the pixel, the blur original image 350 can be generated. In this case, the image generation unit 310 can adjust the blur intensity by the size of a certain range around the pixel to be blurred.

  When the synthesizing unit 320 synthesizes the blurred image 360 obtained by applying the blurring process to the first pixel that is a pixel included in the first image 330, the pixel of the blurred original image 350 corresponding to the position of the first pixel. The pixel value after applying the blurring process to the first pixel is determined using the second pixel. When combining, the selected original blur image 350 may be enlarged to the size of the first image 330 and then combined with the first image 330.

  The synthesizing unit 320 can select a blur original image from the plurality of blur original images 350 according to various criteria as necessary. For example, the original image to be selected may be determined according to the blur intensity, and the original blur image may be selected according to the blur information in which the blur intensity is designated for each pixel corresponding to the first image 330. The synthesizing unit 320 may select a plurality of, for example, two blurring original images from the plurality of blurring original images 350. In this case, the image processing apparatus 100 can generate an image to be used for composition from the two selected blur original images.

  FIG. 4 is a diagram illustrating an outline of the blurring process performed by the image processing apparatus 400 according to the present embodiment. The following (1) to (5) correspond to (1) to (5) shown in FIG. However, the following (1) to (5) and (1) to (5) described in FIG. 4 are described for ease of explanation, and are not intended to limit the order of processing.

  (1) The image processing apparatus 400 acquires an image 410 focused on a desired subject from, for example, a memory 1602 or a DSP (Digital Signal Processor) 1609 described later. The acquired image 410 is referred to as “subject focused image 410”. The subject focused image 410 according to the present embodiment is YUV format data, but various types of data such as RGB format data can be used.

  Further, the image processing apparatus 400 acquires an image 411 obtained by reducing the subject focused image 410 at a certain rate from, for example, a memory 1602 or a DSP 1609 described later. The acquired image 411 is referred to as “subject focused reduced image 411”. The subject focused reduced image 411 according to the present embodiment is YUV format data, but data in various formats such as RGB format data can be used.

  (2) The image processing apparatus 400 generates a plurality of reduced images 1, 2, 3, and 4 in which the subject focused image 410 is reduced. The image processing apparatus 400 may generate the reduced images 1, 2, 3, and 4 by reducing the subject in-focus image 410 at different reduction ratios. It is assumed that reduced images 1, 2, 3, and 4 are generated by reducing the focal image 410 at the same reduction rate.

  When the reduced images 1, 2, 3, and 4 are generated, the image processing apparatus 400 performs a blurring process on the reduced images 1, 2, 3, and 4. When performing the blurring process, the image processing apparatus 400 performs the blurring process by increasing the blurring intensity in the order of the reduced images 1, 2, 3, and 4. The reduced images 1, 2, 3, and 4 that have been subjected to the blurring process are referred to as “bluring original images”. Also, the blur intensity refers to the degree of blur. FIG. 4 shows an example in which four blurred original images 421, 422, 423, and 424 having different blur strengths are generated from the subject focused reduced image 411.

  For the blurring process, a general method such as averaging of pixel values can be used. For example, in the averaging method, an average value of pixel values within a certain range centered on a pixel to be blurred is obtained, and the pixel value of the pixel to be blurred is replaced with the obtained average value. The blur intensity can be adjusted according to the size of a certain range around the pixel to be blurred. Hereinafter, a certain range centered on the pixel to be blurred is referred to as “filter size”. In the case of YUV format data, the pixel value refers to each of the Y value, U value, and V value in a certain pixel.

  The blur strength can be adjusted by the size of the filter and the number of times the blur process is repeated on the reduced image. For example, the blurring intensity increases as the filter size used for averaging pixel values increases, and the blurring intensity decreases as the filter size decreases. Also, the greater the number of blurring processes performed on the same image, the greater the blurring intensity, and the smaller the number of blurring processes, the smaller the blurring intensity.

  In this embodiment, a value from 0 to 255 is used as a blur intensity value to be described later. Larger values indicate greater blur intensity. 0 indicates a state in which the blurring process is not performed. The original blur images 421, 422, 423, and 424 are associated with the blur intensity values 64, 128, 192, and 255, respectively. The subject focused image 410 is associated with a blur intensity value of 0.

  The image processing apparatus 400 uses a first blurred original image and a second blurred image from the subject focused image 410 and the blurred original images 421, 422, 423, and 424 in accordance with a blurred intensity value included in a blurred map 440 described later. Select the original blur image.

  For example, when the blur intensity value is 0, the image processing apparatus 400 selects the subject focused image 410 as the first blur original image, and when the blur intensity value is 64, the blur original image 421 is the first blur source. Select as an image. When the blur intensity value is a value between 0 and 64, the image processing apparatus 400 selects the subject focused image 410 as the first blur original image and the blur original image 421 as the second blur original image. Select.

  (3) The image processing apparatus 400 acquires an image 431 obtained by reducing an image 430 focused on something other than a desired subject, for example, a landscape or the like at a certain rate. An object other than the desired subject is called a “background”. The image 430 is referred to as “background focused image 430”, and the image 431 is referred to as “background focused reduced image 431”. The background focused reduced image 431 according to the present embodiment is YUV format data, but data in various formats such as RGB format data can be used.

The subject focused image 410 is an image obtained by capturing the same subject and the same background with the same composition as the background focused image 430 and focusing on the background. The background focused image 430 is an image obtained by capturing the same subject and the same background with the same composition as the subject focused image 410 and focusing on the background.
In FIG. 4, the background focused image 430 is indicated by a dotted line because the background focused image 430 is described only for explaining the background focused reduced image 431.

  (4) The image processing apparatus 400 generates a blur map 440 from the subject focused reduced image 411 and the background focused reduced image 431. The blur map 440 is information including a blur position and a value representing the blur strength. A value representing the blur intensity is referred to as a “blur intensity value”. The blur map 440 according to the present embodiment includes the blur position and blur intensity value of the pixel in the background area of the subject focused image 410.

  The blur position is information indicating the position of a pixel to be subjected to blur processing among the pixels included in the subject focused image 410. In this embodiment, since the background is subject to blur processing, the blur position includes information indicating the position of the pixel in the background area, for example, XY coordinates.

  The image processing apparatus 400 performs a smoothing process on the blur map 440. This smoothing process can be realized, for example, by averaging pixel values included in the blur map 440 with a filter size of 32 × 32 pixels.

  (5) The image processing apparatus 400 combines the subject focused image 410 and the blurred original image 420 according to the blurred map 440 to generate a blurred image 450. For example, the image processing apparatus 400 uses the first blur original image and the second blur original image used for the synthesis from the subject focused image 410 and the blur original image 420 according to the blur intensity value included in the blur map 440. Select. Then, the image processing apparatus 400 calculates the pixel value of the pixel to be blurred from the selected first and second blurred original images, and sets the calculated pixel value in the blurred image 450.

FIG. 5 is a diagram for explaining the flow of blurring processing by the image processing apparatus 400 according to the present embodiment.
The image processing apparatus 400 acquires the subject focused image 410 from a memory 1602 or a DSP 1609 described later. Further, the image processing apparatus 400 acquires the subject focused reduced image 411 from a memory 1602 or a DSP 1609 described later.

  The subject focused reduced image 411 can be obtained by reducing an image captured by the camera 1610, which will be described later, at a constant reduction rate by the DSP 1609. The reduction rate is appropriate depending on the size of the subject focused image 410. It is desirable to set to. For example, when the size of the subject focused image 410 is small, the reduction rate is reduced, and when the size of the subject focused image 410 is large, the reduction rate is increased, and the reduction rate is appropriately set. It is desirable.

  When the subject focused reduced image 411 is acquired, the image processing apparatus 400 generates blur original images 421, 422, 423, and 424 having different blur strengths for the subject focused reduced image 411, respectively.

  The blur intensity of the blur original images 421, 422, 423, and 424 can be changed by changing the filter size and performing the blur process. Further, the blur intensity of each blur original image can be changed by changing the number of blur processes performed on each of the blur original images 421, 422, 423, and 424. For example, by performing blur processing once for the blur original image 421, twice for the blur original image 422, three times for the blur original image 423, and four times for the blur original image 424, The blur intensity can be changed. Also, the blur intensity of each blur original image can be changed by changing the combination of the filter size and the number of times of blur processing. In this embodiment, it is assumed that the blurring intensity increases in the order of the blurring original images 421, 422, 423, and 424.

  The image processing apparatus 400 generates a blurred image 450 by combining the subject focused image 410 and the blurred original images 421, 422, 423, and 424 in accordance with the blurred map 440.

  For example, when the blur intensity value included in the blur map 440 is “0”, the image processing apparatus 400 sets the pixel value of the blur target pixel among the pixels included in the subject focused image 410 to the blur image 450.

  When the blur intensity value included in the blur map 440 is greater than “0” and equal to or less than “64”, the image processing apparatus 400 selects the subject focused image 410 and the blur source image 421. Then, in accordance with the blur intensity value, interpolation between the pixel value of the pixel to be blurred among the pixels included in the subject focused image 410 and the pixel value of the pixel to be blurred among the pixels included in the blur original image 421 is performed. Set the value to the blurred image 450.

  When the blur intensity value included in the blur map 440 is “64”, the image processing apparatus 400 sets the pixel value of the pixel to be blurred among the pixels included in the blur source image 421 in the blur image 450.

  When the blur intensity value included in the blur map 440 is larger than “64” and equal to or smaller than “128”, the image processing apparatus 400 selects the blur original image 421 and the blur original image 422. Then, an interpolation value between the pixel value of the pixel to be blurred among the pixels included in the blur original image 421 and the pixel value of the pixel to be blurred among the pixels included in the blur original image 422 according to the blur intensity value. Is set to the blurred image 450.

  When the blur intensity value included in the blur map 440 is “128”, the image processing apparatus 400 sets the pixel value of the blur target pixel among the pixels included in the blur source image 422 in the blur image 450.

  When the blur intensity value included in the blur map 440 is larger than “128” and equal to or smaller than “192”, the image processing apparatus 400 selects the blur original image 422 and the blur original image 423. Then, an interpolation value between the pixel value of the pixel to be blurred among the pixels included in the blur original image 422 and the pixel value of the pixel to be blurred among the pixels included in the blur original image 423 according to the blur intensity value. Is set to the blurred image 450.

  When the blur intensity value included in the blur map 440 is “192”, the image processing apparatus 400 sets the pixel value of the blur target pixel among the pixels included in the blur source image 423 in the blur image 450.

  When the blur intensity value included in the blur map 440 is larger than “192” and equal to or smaller than “255”, the image processing apparatus 400 selects the blur original image 423 and the blur original image 424. Then, an interpolation value between the pixel value of the pixel to be blurred among the pixels included in the blur original image 423 and the pixel value of the pixel to be blurred among the pixels included in the blur original image 424 according to the blur intensity value. Is set to the blurred image 450.

  When the blur intensity value included in the blur map 440 is “255”, the image processing apparatus 400 sets the pixel value of the blur target pixel among the pixels included in the blur source image 424 in the blur image 450.

  When the above processing is completed, for example, a processing result 520 shown in FIG. 5 can be obtained. When compared with the background portion 511 of the subject focused image 410, the background portion 521 of the processing result 520 is blurred.

FIG. 6 is a flowchart illustrating an outline of the blurring process executed by the image processing apparatus 400 according to the present embodiment.
When the start of the blurring process is instructed from the input unit 1603 or the like described later, the image processing apparatus 400 starts the blurring process (step S600). Then, the image processing apparatus 400 moves the process to step S601.

  In steps S601, S602, and S603, the image processing apparatus 400 acquires a subject focused image 410, a subject focused reduced image 411, and a background focused reduced image 431 from a memory 1602 and a DSP 1609 described later.

  In step S <b> 604, the image processing apparatus 400 generates a plurality of original blur images 420 from the subject focused reduced image 411. In the present embodiment, four blur original images 421, 422, 423, and 424 having different blur strengths are generated.

  In step S605, the image processing apparatus 400 generates a blur map 440 from the subject focused reduced image 411 and the background focused reduced image 431. The blur map 440 will be specifically described with reference to FIGS.

  In step S606, the image processing apparatus 400 generates a blurred image 450 by combining the subject focused image 410 and the blurred original images 421, 422, 423, and 424 in accordance with the blur map 440. The process of generating the blurred image 450 will be specifically described with reference to FIG.

  FIG. 7 is a diagram illustrating the blur map 440 according to the present embodiment. The following (1) and (2) correspond to (1) and (2) shown in FIG. However, the following (1) and (2) and (1) and (2) described in FIG. 7 are described for ease of explanation, and are not intended to limit the order of processing.

  (1) When the image processing apparatus 400 acquires the subject focused reduced image 411 from the memory 1602 or the DSP 1609 described later, the image processing apparatus 400 converts the subject focused reduced image 411 into a gray scale image, and applies the converted image to the converted image. Edge detection processing is performed. For the edge detection process, a known technique such as a filter process using a Laplacian filter can be used. However, it is natural that edge detection may be performed using other known techniques.

  Through the above processing, the image processing apparatus 400 can generate an edge image 710 in which an edge is extracted from the subject focused reduced image 411. The edge image 710 is an image including pixel values of −128 to +127, for example. In the edge image 710 shown in FIG. 7, the edge portion of the subject is displayed with a solid line.

  (2) Similarly, when the image processing apparatus 400 acquires the background focused reduced image 431 from the memory 1602 or the DSP 1609 described later, the image processing apparatus 400 converts the background focused reduced image 431 into a grayscale image, and the converted image Edge detection processing is performed on the. This edge detection process can also use a known technique, for example, a filter process using a Laplacian filter.

  With the above processing, the image processing apparatus 400 can generate an edge image 720 obtained by extracting an edge from the background focused reduced image 431. The edge image 720 is also an image including pixel values of −128 to +127, for example. In the edge image 720 shown in FIG. 7, the edge portion of the background excluding the subject is displayed with a solid line.

  (3) When the edge images 710 and 720 are generated, the image processing device 400 calculates, for each pixel, the difference between the absolute value of the pixel value of the edge image 710 and the absolute value of the pixel value of the edge image 720, An absolute value difference image 730 is generated. Since the pixel value of the pixel included in the absolute value difference image 730 is obtained from the difference between the absolute values of the pixel values of the edge images 710 and 720, the pixel value is −128 to +128. For example, when the absolute value difference is obtained by subtracting the pixel value of the pixel included in the edge image 710 from the pixel value of the pixel included in the edge image 720, the pixel value of the pixel included in the absolute value difference image 730 is calculated. Of these, an edge including a pixel having a pixel value greater than 0 can be determined as an edge of the background portion. In the absolute value difference image 730 shown in FIG. 7, the edge portion of the subject and the edge portion of the background are displayed by solid lines.

  The image processing apparatus 400 generates an image in which only the edge of the background portion is extracted from the absolute value difference image 730, and adjusts the pixel values of the pixels included in the generated image to be a value of 0-255. The image obtained by the above processing is used as the blur map 440. Note that the image processing apparatus 400 may further perform a smoothing process on the blur map 440. This smoothing process can be realized, for example, by averaging pixel values included in the blur map 440 with a filter size of 32 × 32 pixels.

FIG. 8 is a flowchart showing the generation process of the blur map 440 according to the present embodiment.
In step S <b> 801, the image processing apparatus 400 converts the subject focused reduced image 411 acquired in step S <b> 602 to grayscale, and then performs edge detection processing on the converted image to generate an edge image 710. Similarly, the image processing apparatus 400 converts the background focused reduced image 431 acquired in step S603 to grayscale, and then performs edge detection processing on the converted image to generate an edge image 720.

In step S802, the image processing apparatus 400 acquires absolute values of pixel values of pixels included in the edge images 710 and 720, respectively.
In step S803, the image processing apparatus 400 calculates an absolute value difference image 730 by calculating a difference between absolute values of pixel values of pixels at the same position included in the edge images 710 and 720 for each pixel. In this embodiment, the absolute value of the pixel value of the pixel included in the edge image 710 is subtracted from the absolute value of the pixel value of the pixel included in the edge image 720.

  In step S804, the image processing apparatus 400 generates a blur map 440 by extracting only the background region, in this embodiment, the edge included in the background region, from the absolute value difference image 730. In the present embodiment, among the pixels included in the absolute value difference image 730, a pixel having a pixel value greater than 0 can be determined as a pixel in the background area. In this embodiment, the absolute value difference image 730 is obtained by subtracting the absolute value of the pixel value of the pixel included in the edge image 710 from the absolute value of the pixel value of the pixel included in the edge image 720 including the edge portion of the background. This is because it is generated.

  When the absolute value difference image 730 is generated by subtracting the absolute value of the pixel value of the pixel included in the edge image 720 from the absolute value of the pixel value of the pixel included in the edge image 710, the absolute value difference image 730 Among the included pixels, a pixel having a pixel value smaller than 0 can be determined as a pixel in the background area.

  In step S805, the image processing apparatus 400 performs a smoothing process on the blur map 440 generated in step S804. When the smoothing process ends, the image processing apparatus 400 ends the blur map 440 generation process (step S806).

FIG. 9 is a flowchart illustrating the generation process of the blurred image 450 by the image processing apparatus 400 according to the present embodiment.
In step S <b> 901, the image processing apparatus 400 selects one pixel to be processed from the pixels included in the subject focused image 410. Then, the image processing apparatus 400 refers to the blur map 440 (step S902), and acquires the blur intensity value for the pixel to be processed (step S903).

  Here, the blur intensity value according to the present embodiment is a value from 0 to 255. As shown in FIG. 4, the blur intensity values “0”, “64”, “128”, “192”, and “192” serving as a reference for selecting the subject focused image 410 and the blur original images 421, 422, 423, or 424 are selected. “255” is generally referred to as “boundary value”.

  If the blur intensity value acquired in step S903 does not match the boundary value (YES in step S904), the image processing apparatus 400 moves the process to step S905. In this case, the image processing apparatus 400 selects two images from the subject focused image 410 and the blurred original images 421, 422, 423, or 424 in accordance with the blur intensity value (step S905). A specific example of this image selection processing has been described with reference to FIG.

  In the present embodiment, the two images selected in step S905 are referred to as “first blur original image” and “second blur original image”, respectively. The subject focused image 410 may be the first blurred original image.

  In step S906, the image processing apparatus 400 enlarges the first blur original image selected in step S905 to an image having the same size as the subject focused image 410, for example. If the subject focused image 410 is selected as the first blur original image in step S906, the image processing apparatus 400 does not perform the process in step S906.

  In step S907, the image processing apparatus 400 acquires the pixel value of the pixel at the same position as the pixel to be processed from the enlarged first blurred original image. This pixel value is referred to as a “first pixel value”.

In step S908, the image processing apparatus 400 enlarges the second blur original image selected in step S905 to an image having the same size as the subject focused image 410, for example.
In step S909, the image processing apparatus 400 acquires the pixel value of the pixel at the same position as the pixel to be processed from the enlarged second blurred original image. This pixel value is referred to as a “second pixel value”.

In step S910, the image processing apparatus 400 calculates an interpolated value of the pixel value acquired in steps S907 and S909. Assuming that the first pixel value is R1, the second pixel value is R2, and the boundary values are Th1 and Th2, when the relationship is Th2>S> Th1, the first pixel value is obtained from the following equation (1). And an interpolation value of the second pixel value can be calculated.
R1 * (S-Th1) / (Th2-Th1) +
R2 * (Th2-S) / (Th2-Th1) (1)
After calculating the interpolation value, the image processing apparatus 400 moves the process to step S914.

  On the other hand, if the blur intensity value acquired in step S903 matches the boundary value (NO in step S904), the image processing apparatus 400 proceeds to step S911. In this case, the image processing apparatus 400 selects one image from the subject focused image 410 and the blurred original images 421, 422, 423, or 424 in accordance with the blur intensity value (step S911). A specific example of selecting the image has been described with reference to FIG.

In this embodiment, the image selected in step S911 is referred to as “third blurred original image”. The subject focused image 410 may be the third blurred original image.
In step S912, the image processing apparatus 400 enlarges the third blur original image selected in step S911 to an image having the same size as the subject focused image 410, for example. When the subject focused image 410 is selected as the third blur original image in step S911, the image processing apparatus 400 does not perform the process of step S912.

  In step S913, the image processing apparatus 400 acquires the pixel value of the pixel at the same position as the pixel to be processed from the third blurred original image. Then, the image processing apparatus 400 moves the process to step S914.

  In step S914, the image processing apparatus 400 sets the interpolated value obtained in step S910 or the pixel value obtained in step S913 to the pixel value of the pixel of the blurred image 450 at the same position as the pixel to be processed. .

  When the processes in steps S901 to S914 have been performed on all the pixels included in the subject focused image 410 (YES in step S915), the image processing apparatus 400 ends the generation process of the blurred image 450 (step S916). If there is a pixel that has not been subjected to the processing in steps S901 to S914 among the pixels included in the subject focused image 410 (NO in step S915), the image processing apparatus 400 proceeds to step S901 and proceeds to step S901. The process of ~ S914 is performed.

FIG. 10 is a diagram for explaining the flow of image processing performed by the image processing apparatus 1000 according to the present embodiment.
The image processing apparatus 1000 acquires the subject focused image 410 from a memory 1602 or a DSP 1609 described later. The image processing apparatus 1000 acquires subject focused reduced images 1011, 1012, and 1013 from a memory 1602 and a DSP 1609 described later. FIG. 10 illustrates a case where the image processing apparatus 1000 acquires a subject focused reduced image in which the reduction ratios are increased in the order of the subject focused reduced images 1011, 1012, and 1013.

  The subject focused reduced images 1011, 1012, and 1013 can be obtained by the DSP 1609 reducing images captured by the camera 1610, which will be described later, at different reduction rates. It is desirable to set appropriately according to the size of the subject focused image 410.

  Upon obtaining the subject focused reduced images 1011, 1012, and 1013, the image processing apparatus 1000 performs the original blurred images 1021, 1022, 1023, and 1024 obtained by performing the blurring process on the subject focused reduced images with different blur strengths. Is generated.

  The original blur images 1021 and 1022 shown in FIG. 10 are subject focused reduced images obtained by subjecting the subject focused reduced image 1011 to blur processing with different blur strengths. Also, the blur original image 1023 is a subject focused reduced image obtained by performing a blurring process on the subject focused reduced image 1011. The blur original image 1024 is a subject focused reduced image obtained by performing a blurring process on the subject focused reduced image 1011.

  As in the second embodiment, the blurring original images 1021, 1022, 1023, and 1024 can be changed in blurring intensity by changing the filter size of each blurring original image. Further, the blur intensity of each blur original image can be changed by changing the number of blur processes performed on each of the blur original images 1021, 1022, 1023, and 1024. Also, the blur intensity of each blur original image can be changed by changing the combination of the filter size and the number of times of blur processing. In the present embodiment, the blur intensity increases in the order of the blur original images 1021, 1022, 1023, and 1024.

  The image processing apparatus 1000 generates the blurred image 1040 by combining the subject focused image 410 and the blurred original images 1021, 1022, 1023, and 1024 according to the blur map 440.

  For example, when the blur intensity value included in the blur map 440 is “0”, the image processing apparatus 1000 sets the pixel value of the pixel to be blurred among the pixels included in the subject focused image 410 to the blur image 1030.

  When the blur intensity value included in the blur map 440 is larger than “0” and equal to or smaller than “64”, the image processing apparatus 1000 selects the subject focused image 410 and the blur original image 1021. Then, in accordance with the blur intensity value, interpolation between the pixel value of the blur target pixel among the pixels included in the subject focused image 410 and the pixel value of the blur target pixel among the pixels included in the blur source image 1021 is performed. The value is set to the blurred image 1030.

  When the blur intensity value included in the blur map 440 is “64”, the image processing apparatus 1000 sets the pixel value of the blur target pixel in the blur image 1030 among the pixels included in the subject focused image 1021.

  When the blur intensity value included in the blur map 440 is larger than “64” and equal to or smaller than “128”, the image processing apparatus 1000 selects the blur original image 1021 and the blur original image 1022. Then, an interpolation value between the pixel value of the pixel to be blurred among the pixels included in the original image 1021 and the pixel value of the pixel to be blurred among the pixels included in the original image 1022 according to the blur intensity value. Is set to the blurred image 1030.

  When the blur intensity value included in the blur map 440 is “128”, the image processing apparatus 1000 sets the pixel value of the blur target pixel in the blur image 1030 among the pixels included in the subject focused image 1022.

  When the blur intensity value included in the blur map 440 is larger than “128” and equal to or smaller than “192”, the image processing apparatus 1000 selects the blur original image 1022 and the blur original image 1023. Then, according to the blur intensity value, an interpolation value between the pixel value of the pixel to be blurred among the pixels included in the blur original image 1022 and the pixel value of the pixel to be blurred among the pixels included in the blur original image 1023. Is set to the blurred image 1030.

  When the blur intensity value included in the blur map 440 is “192”, the image processing apparatus 1000 sets the pixel value of the blur target pixel among the pixels included in the subject focused image 1023 in the blur image 1030.

  When the blur intensity value included in the blur map 440 is larger than “192” and equal to or smaller than “255”, the image processing apparatus 1000 selects the blur original image 1023 and the blur original image 1024. Then, an interpolation value between the pixel value of the pixel to be blurred among the pixels included in the original image 1023 and the pixel value of the pixel to be blurred among the pixels included in the original image 1024 according to the blur intensity value. Is set to the blurred image 1030.

  When the blur intensity value included in the blur map 440 is “255”, the image processing apparatus 1000 sets the pixel value of the blur target pixel among the pixels included in the subject focused image 1024 in the blur image 1030.

  When the above processing is completed, for example, a processing result 1040 shown in FIG. 10 can be obtained. When compared with the background portion 1051 of the subject focused image 410, the background portion 1041 of the processing result 1040 is blurred.

  In FIG. 10, the subject focused reduced images in which the reduction ratios are increased in the order of the subject focused reduced images 1011, 1012, and 1013 are used, but the present invention is not limited to this. The subject focused reduced images may have different reduction ratios.

  FIG. 10 shows an example in which three subject focused reduced images 1011, 1012, and 1013 are used, but this is not intended to limit the number of subject focused reduced images. For example, two subject focused reduced images may be used, or five or more subject focused reduced images may be used.

  Further, two or more blurring original images generated from the same subject focused reduced image need to have different blur strengths, but the blurring original images generated from different subject focused reduced images have the same blur strength. May be different or different.

  Here, in order to clarify the correspondence between the positions of the pixels to be blurred in the subject focused image 410 and the blurred original images 1021, 1022, 1023, and 1024, the screens of the blurred original images 1021, 1022, 1023, and 1024 are displayed. FIG. 11 shows the same size. In the following description, this embodiment will be described with reference to FIG. Black dots “·” shown in the subject focused image 410, the subject focused reduced images 1011 to 1013, and the blurred original images 1021 to 1024 indicate pixels. For example, since the blurred original image 1021 is enlarged and displayed, the distance between the pixels is larger than that of the subject focused reduced image 410.

  FIG. 12 is a diagram illustrating an outline of the blurring process performed by the image processing apparatus 1000 according to the present embodiment. Note that (1) to (4) described in FIG. 12 correspond to (1) to (4) described below. However, the following (1) to (4) and (1) to (4) described in FIG. 12 are described for ease of explanation, and are not intended to limit the order of processing.

(1) The image processing apparatus 1000 selects one processing target pixel 1210 from the pixels included in the subject focused image 410 that is the target of the blurring process.
(2) The image processing apparatus 1000 refers to the blur map 440 and acquires the blur intensity value of the pixel 1210 to be processed.

  (3) From the position of the processing target pixel 1210 and the blur intensity value, six pixels used for the six-point interpolation process described later are included in the subject focused image 410 and the blur original images 1021, 1022, 1023, and 1024. Select from the pixels to be selected. For example, the image processing apparatus 1000 can select six pixels to be used for a six-point interpolation process described later as follows.

  First, the image processing apparatus 1000 selects one or two images from the subject focused image 410 and the blurred original images 1021, 1022, 1023, and 1024 according to the blur intensity value. The selection of this image can be performed by, for example, processing similar to the processing in steps S904 to S905 and S911 shown in FIG. FIG. 12 illustrates a case where the original blur images 1023 and 1024 are selected.

  Next, the image processing apparatus 1000 includes three pixels 1221 among the pixels surrounding the position 1220 in the blurred original image 1023 corresponding to the position of the processing target pixel 1210 in the subject focused image 410 that is the target of the blurring process. , 1222 and 1223 are selected. Further, the image processing apparatus 1000 includes three pixels 1231 out of the pixels surrounding the position 1230 in the blurred original image 1024 corresponding to the position of the processing target pixel 1210 in the subject focused image 410 that is the target of the blurring process. Select 1232 and 1233.

FIG. 13 is a diagram illustrating pixel selection according to the present embodiment.
When selecting a pixel, a right triangle having apexes at three pixels included in the original image 1023 is considered. As shown in FIG. 13, the position 1220 in the original blur image 1023 is surrounded by a right triangle whose apexes are three pixels 1221, 1222, and 1223. In this case, the image processing apparatus 1000 selects three pixels 1221, 1222, and 1223.

  Similarly, the position 1230 in the blurred original image 1024 is surrounded by a right triangle whose apexes are three pixels 1231, 1232, and 1233. In this case, the image processing apparatus 1000 selects three pixels 1231, 1232, and 1233.

(4) A six-point interpolation process is performed on the selected six pixels 1221, 1222, 1223, 1231, 1232, and 1233 to calculate one interpolation value.
FIG. 14 is a diagram for explaining the six-point interpolation processing according to the present embodiment. In the following description, in the xyz coordinate system shown in FIG. 14, the x-axis direction is referred to as the horizontal direction, and the y-axis direction is referred to as the vertical direction.

  First, the image processing apparatus 1000 performs an interpolation process on the three pixels 1221, 1222, and 1223 on the blurred original image 1023 among the six selected pixels 1221, 1222, 1223, 1231, 1232, and 1233. . The interpolation value obtained as a result is referred to as “first interpolation value”.

Here, the pixel values of the pixels 1221, 1222 and 1223 are d1, a1 and c1, respectively. Further, the ratio of the horizontal coordinate of the position 1220 in the blurred original image 1023 corresponding to the position of the pixel 1210 to be processed is h11: h12. The ratio of the vertical coordinates of the position 1220 in the blurred original image 1023 corresponding to the position of the pixel 1210 to be processed is v11: v12. In this case, the first interpolation value can be obtained by the following equation (2).
a1 + (d1-a1) * (v11 / (v11 + v12))
-(C1-d1) * (h11 / (h11 + h12)) (2)

  Similarly, the image processing apparatus 1000 performs interpolation processing on three pixels 1231, 1232, and 1233 on the blurred original image 1024 among the six selected pixels 1221, 1222, 1223, 1231, 1232, and 1233. Do. The interpolation value obtained as a result is referred to as a “second interpolation value”.

Here, the pixel values of the pixels 1231, 1232, and 1233 are c2, b2, and a2, respectively. In addition, the ratio of the horizontal coordinates of the position 1230 in the blurred original image 1024 corresponding to the position of the pixel 1210 to be processed is h21: h22. The ratio of the vertical coordinates of the position 1230 in the original blur image 1024 corresponding to the position of the pixel 1210 to be processed is v21: v22. In this case, the second interpolation value can be obtained by the following expression (3).
a2 + (b2-a2) * (h21 / (h21 + h22))
+ (C2-b2) * (v21 / (v21 + v22)) (3)

When the first interpolation value and the second interpolation value are calculated, the image processing apparatus 1000 calculates a six-point interpolation value from the first interpolation value and the second interpolation value. For example, the first interpolation value is R1, the second interpolation value is R2, and the blur intensity value is S. Further, of the boundary values when selecting the original blur images 1023 and 1024, the upper limit value is Th1, and the lower limit value is Th2. In this case, a six-point interpolation value can be calculated from the first interpolation value and the second interpolation value by the following equation (4).
R1 * (S-Th1) / (Th2-Th1)
+ R2 * (Th2-S) / (Th2-Th1) (4)

  Since the outline of the blurring process executed by the image processing apparatus 1000 according to the present embodiment is the same as the process described with reference to FIG. 6, the generation process of the blurred image 1040 by the image processing apparatus 1000 will be described below.

  FIG. 15 is a flowchart illustrating the generation process of the blurred image 1040 by the image processing apparatus 1000 according to the present embodiment. Note that the processing from step S1500 to S1504 is the same as the processing from step S900 to S904 shown in FIG.

  If the blur intensity value acquired in step S1503 does not match the boundary value (YES in step S1504), the image processing apparatus 1000 moves the process to step S1005. In this case, the image processing apparatus 1000 selects two images from the subject focused image 410 and the blurred original images 1021, 1022, 1023, and 1024 according to the blur intensity value (step S1505). A specific example of this image selection process has been described with reference to FIG.

  In the present embodiment, the two images selected in step S1505 are referred to as “first blur original image” and “second blur original image”, respectively. The subject focused image 410 may be the first blurred original image.

  In step S1506, the image processing apparatus 1000 selects three pixels from the first blurred original image selected in step S1505. In step S1507, the image processing apparatus 1000 calculates a first interpolation value for the pixel values of the three pixels selected in step S1506.

  In step S1508, the image processing apparatus 1000 selects three pixels from the second blur original image selected in step S1505. In step S1509, the image processing apparatus 1000 calculates a second interpolation value for the pixel values of the three pixels selected in step S1508.

  In step S1510, the image processing apparatus 1000 calculates a six-point interpolation value for the six pixels selected in steps S1506 and S1508 from the first interpolation value and the second interpolation value.

A specific description of steps S1506 to S1510 has been described with reference to FIGS.
After calculating the 6-point interpolation value, the image processing apparatus 1000 shifts the processing to step S1514.

  On the other hand, if the blur intensity value acquired in step S1503 matches the boundary value (NO in step S1504), the image processing apparatus 1000 shifts the process to step S1011. In this case, the image processing apparatus 1000 selects one image from the subject focused image 410 and the blurred original images 1021, 1022, 1023, and 1024 according to the blur intensity value (step S1511). A specific example of selecting the image has been described with reference to FIG.

In this embodiment, the image selected in step S1511 is referred to as “third blurred original image”. The subject focused image 410 may be the third blurred original image.
In step S1512, the image processing apparatus 1000 selects three pixels from the third blur original image selected in step S1511. In step S1513, the image processing apparatus 1000 calculates a third interpolation value for the pixel values of the three pixels selected in step S1512. In addition, since the process of step S1512 and S1513 is the process similar to step S1506 and S1507, concrete description is abbreviate | omitted.

  In step S1514, the image processing apparatus 1000 sets the six-point interpolation value obtained in step S1510 or the pixel value acquired in step S1513 for the blurred image 1030.

  When the processing of steps S1501 to S1514 has been performed for all the pixels included in the subject focused image 410 (YES in step S1515), the image processing apparatus 1000 ends the generation processing of the blurred image 1040 (step S1516). If there is a pixel that has not been subjected to the processing in steps S1501 to S1514 among the pixels included in the subject focused image 410 (NO in step S1515), the image processing apparatus 1000 shifts the processing to step S1501 and performs step S1501. Processes of S1514 are performed.

  FIG. 16 is a diagram illustrating a specific configuration example of the image processing apparatus according to the first to third embodiments. Note that the image processing apparatus 400 and the image processing apparatus 1000 can be realized by information processing apparatuses having the same configuration. Therefore, only a specific configuration example of the image processing apparatus 400 will be described, and a description of a specific configuration example of the image processing apparatus 1000 will be omitted.

  16 includes a CPU 1601, a memory 1602, an input device 1603, an output device 1604, an external storage device 1605, a medium drive device 1606, a network connection device 1608, a DSP 1609, and a camera 1610. . These devices are connected to a bus so that data can be exchanged between them.

The CPU 1601 is an arithmetic device that executes a program that implements blurring processing according to the present embodiment in addition to executing peripheral devices and various software.
The memory 1602 is a volatile storage device used to execute a program. For example, a random access memory (RAM) can be used as the memory 1602. The memory 1602 can store a subject focused image 410, a subject focused reduced image 411, a background focused image 430, a background focused reduced image 431, a blur original image 420, and the like.

The input device 1603 is data input means from the outside. For example, a keyboard or a mouse can be used as the input device 1603.
The output device 1604 is a device that outputs data or the like to a display device or the like. Note that the output device 1604 can also include a display device.

  The external storage device 1605 is a non-volatile storage device that stores a program for realizing the blurring processing according to the present embodiment, in addition to a program and data necessary for the operation of the image processing apparatus 400. For example, a magnetic disk storage device can be used as the external storage device 1605.

  The medium driving device 1606 outputs the data of the memory 1602 and the external storage device 1605 to a portable storage medium 1607, for example, a floppy disk, an MO disk, a CD-R, a DVD-R, or the like, or from the portable storage medium 1607. It is a device that reads programs and data.

The network connection device 1608 is a device connected to the network 1609.
The DSP 1609 performs a predetermined process on the image captured by the camera 1610, for example, a process of reducing at a constant reduction rate. The DSP 1609 stores the image subjected to the predetermined processing in the memory 1602, the external storage device 1605, the portable storage medium 1607, or the like as necessary.

  The camera 1610 is an imaging device that can generate an image by focusing on a desired object. For example, the camera 1610 generates a subject focused image 410 focused on the subject, a background focused image 430 focused on the background, and the like. The camera 1610 outputs the generated image to the DSP 1609, the memory 1602, the external storage device 1605, the portable storage medium 1607, or the like as necessary.

  Note that a non-transitory medium can be used as a storage medium readable by an information processing device such as the memory 1602, the external storage device 1605, and the portable storage medium 1607.

  The configuration diagram of the image processing apparatus 400 illustrated in FIG. 16 is an example. Therefore, the present invention is not intended to limit the image processing apparatus 400 to the configuration shown in FIG. For example, the external storage device 1605, the medium driving device 1606, the network connection device 1608, the DSP 1609, the camera 1610, and the like are not essential components, and can be incorporated into the configuration of the image processing device 400 as necessary.

  FIG. 17 is a diagram illustrating a configuration example of an apparatus 1700 including the image processing apparatus according to the first to third embodiments. An apparatus 1700 illustrated in FIG. 17 includes an image processing apparatus 1710, a DSP 1720, and a camera 1730.

  The image processing device 1710 is the image processing device according to the first, second, or third embodiment shown in FIG. However, the image processing apparatus 1710 does not include the DSP 1609 and the camera 1610 shown in FIG.

  The DSP 1720 generates a subject focused reduced image 411, a background focused reduced image 431, and the like by performing a predetermined process on the image captured by the camera 1730, for example, a process of reducing at a constant reduction rate.

  The camera 1730 is an imaging device that can generate an image by focusing on a desired object. For example, the camera 1730 generates a subject focused image 410 focused on the subject, a background focused image 430 focused on the background, and the like.

  In the above configuration, when the image processing apparatus 1710 requests the DSP 1720 to take an image, the DSP 1720 instructs the camera 1730 to take an image. Then, the camera 1730 captures an image focused on a predetermined focal position, for example, the subject focused image 410 or the background focused image 430 in accordance with an instruction from the DSP 1720.

  The camera 1730 outputs the captured image to the DSP 1720. The DSP 1720 transmits the image received from the camera 1730 to the image processing apparatus 1710. If the image processing device 1710 has designated a reduction ratio, the DSP 1720 may reduce the image received from the camera 1730 at the reduction rate designated by the image processing device 1710, for example, the subject focused reduced image 411 or The background focused reduced image 431 is transmitted to the image processing apparatus 1710.

The image processing device 1710 stores the image received from the DSP 1720 in the memory 1602, the external storage device 1605, the portable storage medium 1607, or the like as necessary.
The device 1700 can be applied to various forms such as a mobile phone, a mobile terminal, and a camera.

  In the second embodiment, the case where four blurring original images 421, 422, 423, and 424 are used has been described. In the third embodiment, the case where four blurring original images 1021, 1022, 1023, and 1024 are used has been described. However, the second and third embodiments are not intended to limit the number of original images to be used to four. For example, the number of blurring original images to be used may be one or two or more.

  In the above description, the subject focused reduced image 410 can be cited as an example of the first image. The subject focused reduced images 411 and 1011 to 1013 can be cited as examples of the first reduced image. Further, the original blur images 421 to 424 and 1021 to 1024 can be cited as examples of the original blur image. The image generation unit and the synthesis unit can be realized by causing the CPU 1601 to execute a predetermined program.

  Since the image processing apparatus 300 according to the first embodiment generates the blurring original image 350 by performing the blurring process on the first reduced image 340 obtained by reducing the first image 330 at a certain rate, the blurring is performed. The number of pixels to be processed can be reduced. As a result, the load on the image processing apparatus 300 can be reduced, and the blurring processing time can be shortened.

  Further, the image processing apparatus 300 combines the first image 330 and the blurred original image 350 obtained by blurring the first reduced image 340 obtained by reducing the first image 330 at a certain ratio, thereby blurring the image 360. Is generated. Since the blurred original image 350 is a reduced image, the memory consumption can be reduced.

Therefore, the image processing apparatus 300 can generate a blurred image while suppressing memory consumption in a short time.
Since the image processing apparatus 400 according to the second embodiment creates the blur original images 421 to 424 from the subject focused reduced image 411 obtained by reducing the subject focused image 410 at a certain rate, the pixels to be subjected to the blurring process. The number can be reduced. As a result, the burden on image processing can be reduced, and the blurring processing time can be shortened.

  Also, the image processing apparatus 400 blurs the subject focused image 410 and the blur original images 421 to 424 obtained by blurring the subject focused reduced image 411 obtained by reducing the subject focused image 410 at a certain ratio. An image 520 is generated. Since the blurred original images 421 to 424 are reduced images, it is possible to reduce the memory consumption.

Therefore, the image processing apparatus 400 can generate the blurred image 520 while suppressing the memory consumption in a short time.
Since the image processing apparatus 1000 according to the third embodiment generates the blur original images 1021 to 1024 from the subject focused reduced images 1011 to 1013 obtained by reducing the subject focused image 410, the number of pixels to be subjected to the blur processing is determined. Can be reduced. As a result, the image processing apparatus 1000 can reduce the burden on the image processing, so that the blurring processing time can be shortened.

  Further, the image processing apparatus 1000 can further reduce the number of pixels to be subjected to the blurring process by using the subject focused reduced images 1011 to 1013 in which the reduction rate is increased stepwise. As a result, the image processing apparatus 1000 can further reduce the burden on image processing. As a result, the image processing apparatus 1000 can further reduce the blurring processing time.

Further, the image processing apparatus 1000 combines the subject focused image 410 with the blurred original images 1021 to 1024 obtained by blurring the subject focused reduced images 1011 to 1013 obtained by reducing the subject focused image 410, thereby generating the blurred image 1040. Generate. Since the blurred original images 1021 to 1024 are reduced images, the amount of memory consumption can be reduced. Further, the image processing apparatus 1000 can further reduce the memory consumption by using the subject focused reduced images 1011 to 1013 in which the reduction rate is increased stepwise.
Therefore, the image processing apparatus 1000 can generate the blurred image 1040 while suppressing the memory consumption in a short time.

The following supplementary notes are further disclosed with respect to the embodiments including the above examples.
(Appendix 1)
Image generation for generating a plurality of blur original images to which blur processing is applied using different blur intensities, from a first reduced image obtained by reducing a first image obtained by imaging a subject while focusing on a first focus position Means,
A second pixel that is a pixel of the original image corresponding to the position of the first pixel when a blurred image obtained by applying a blurring process to the first pixel that is a pixel included in the first image is synthesized; Combining means for determining a pixel value after applying the blurring process to the first pixel using
An image processing apparatus comprising:
(Appendix 2)
The edge component of the subject included in the first reduced image and the edge component of the subject included in the second reduced image obtained by reducing the second image obtained by capturing the subject while focusing on the second focus position And a blur map generating means for generating a blur map including a blur intensity value indicating a blur intensity for each pixel included in the first image based on the comparison result,
Further comprising
In the synthesis of the blurred image, the synthesizing unit refers to the blur intensity associated with the first pixel from the blur map, and refers to the blur that is referred to from the pixel values of the plurality of blur source pixels and the first pixel. Determine the pixel value corresponding to the intensity,
The image processing apparatus according to appendix 1, wherein:
(Appendix 3)
The synthesizing unit acquires a blur intensity value for the first pixel included in the first image from the blur map, selects the blur original image according to the acquired blur intensity value, and selects the selected A pixel value to be used for the composition is determined from a pixel value of a second pixel at a position corresponding to the position of the first pixel among the pixels included in the blur original image, and the determined pixel value is Set the pixel value of the first pixel;
The image processing apparatus according to Supplementary Note 2, wherein
(Appendix 4)
The synthesizing unit includes: the first blur original image associated with the lower limit value of the range including the acquired blur intensity value; and the second blur original image associated with the upper limit value of the range. A first pixel value that is a pixel value of a pixel at a position corresponding to the position of the first pixel among the pixels included in the first blur original image, and the second blur original image. An interpolation value obtained by complementing a second pixel value that is a pixel value of a pixel at a position corresponding to the position of the first pixel in the included image with a ratio of the blur intensity value in the range; Setting the calculated interpolation value to the pixel value of the first pixel;
The image processing apparatus according to appendix 3, characterized in that:
(Appendix 5)
The first reduced image is a plurality of reduced images obtained by reducing the first image at different ratios.
The image processing apparatus according to Supplementary Note 2, wherein
(Appendix 6)
The synthesizing unit acquires a blur intensity value for the first pixel included in the first image from the blur map, selects the blur original image according to the acquired blur intensity value, and selects the selected A pixel value to be used for the synthesis is calculated from pixel values of a plurality of pixels surrounding a position corresponding to the position of the first pixel among pixels included in the blur original image, and the calculated pixel value is calculated as the first pixel value. Set the pixel value of one pixel,
The image processing apparatus according to appendix 5, characterized in that:
(Appendix 7)
The synthesizing unit includes: the first blur original image associated with the lower limit value of the range including the acquired blur intensity value; and the second blur original image associated with the upper limit value of the range. The ratio of the position of the first pixel between the pixels is selected from pixel values of a plurality of pixels surrounding the position corresponding to the position of the first pixel among the pixels included in the first blur original image. The complemented first pixel value is calculated, and pixel values of a plurality of pixels surrounding a position corresponding to the position of the first pixel among the pixels included in the second blur original image are calculated. A second pixel value complemented by a ratio of the position of one pixel is calculated, and an interpolation value is calculated by complementing the first pixel value and the second pixel value by a ratio of a blur intensity value in the range. The calculated interpolation value is set to the pixel value of the first pixel. ,
The image processing apparatus according to appendix 6, wherein:
(Appendix 8)
The blur map generating means includes
Smoothing means for smoothing a region where pixel values of pixels included in the first reduced image and the second reduced image change discontinuously;
A blur intensity value calculating means for calculating the blur intensity value from a difference between pixel values of the first reduced image and the second reduced image, and generating the blur map including the blur intensity value for each pixel;
When the sum of the difference values in the area of the predetermined size in the blur map exceeds a threshold value, a blur intensity value equal to or less than the threshold value among blur intensity values included in the area of the predetermined size is a predetermined value exceeding the threshold value. Correction means for correcting to,
The image processing apparatus according to appendix 2, further comprising:
(Appendix 9)
A first image obtained by imaging a subject with a focus on the first in-focus position and a first reduced image obtained by reducing the first image are acquired from the storage unit;
Generating a plurality of blur original images obtained by blurring the first reduced image with different blur intensities;
A blur original image selected from the plurality of blur original images and the first image are combined to generate a blur image including a blur region having a blur intensity different in part or all.
Image processing method.
(Appendix 10)
A first image obtained by imaging a subject with a focus on the first in-focus position and a first reduced image obtained by reducing the first image are acquired from the storage unit;
Generating a plurality of blur original images obtained by blurring the first reduced image with different blur intensities;
A blur original image selected from the plurality of blur original images and the first image are combined to generate a blur image including a blur region having a blur intensity different in part or all.
An image processing program for causing an information processing apparatus to execute processing.
(Appendix 11)
Imaging means for focusing on the first in-focus position and imaging a subject to generate a first image;
Reduced image generating means for reducing the first image to generate a first reduced image;
Image generating means for generating a plurality of original images of blurring the first reduced image with different blur strengths;
A synthesizing unit that synthesizes the blur original image selected from the plurality of blur original images and the first image to generate a blur image including a blur region having a different blur intensity in part or all;
An information processing apparatus comprising:

300 Image Processing Device 310 Image Generation Unit 320 Composition Unit 330 First Image 340 First Reduced Image 350 Blur Original Image

Claims (6)

Image generation for generating a plurality of blur original images to which blur processing is applied using different blur intensities, from a first reduced image obtained by reducing a first image obtained by imaging a subject while focusing on a first focus position Means,
A second pixel that is a pixel of the original image corresponding to the position of the first pixel when a blurred image obtained by applying a blurring process to the first pixel that is a pixel included in the first image is synthesized; Combining means for determining a pixel value after applying the blurring process to the first pixel using
An image processing apparatus comprising: The edge component of the subject included in the first reduced image and the edge component of the subject included in the second reduced image obtained by reducing the second image obtained by capturing the subject while focusing on the second focus position And a blur map generating means for generating a blur map including a blur intensity value indicating a blur intensity for each pixel included in the first image based on the comparison result,
Further comprising
In the synthesis of the blurred image, the synthesizing unit refers to the blur intensity associated with the first pixel from the blur map, and refers to the blur that is referred to from the pixel values of the plurality of blur source pixels and the first pixel. Determine the pixel value corresponding to the intensity,
The image processing apparatus according to claim 1. The first reduced image is a plurality of reduced images obtained by reducing the first image at different ratios.
The image processing apparatus according to claim 2. A first image obtained by imaging a subject with a focus on the first in-focus position and a first reduced image obtained by reducing the first image are acquired from the storage unit;
Generating a plurality of blur original images obtained by blurring the first reduced image with different blur intensities;
A blur original image selected from the plurality of blur original images and the first image are combined to generate a blur image including a blur region having a blur intensity different in part or all.
Image processing method. A first image obtained by imaging a subject with a focus on the first in-focus position and a first reduced image obtained by reducing the first image are acquired from the storage unit;
Generating a plurality of blur original images obtained by blurring the first reduced image with different blur intensities;
A blur original image selected from the plurality of blur original images and the first image are combined to generate a blur image including a blur region having a blur intensity different in part or all.
An image processing program for causing an information processing apparatus to execute processing. Imaging means for focusing on the first in-focus position and imaging a subject to generate a first image;
Reduced image generating means for reducing the first image to generate a first reduced image;
Image generating means for generating a plurality of original images of blurring the first reduced image with different blur strengths;
A synthesizing unit that synthesizes the blur original image selected from the plurality of blur original images and the first image to generate a blur image including a blur region having a different blur intensity in part or all;
An information processing apparatus comprising: