JP5203159B2 - Image processing method, image processing system, and image processing program - Google Patents

Description

The present invention relates to an image processing method, an image processing system, and an image processing program, and more particularly, to an image processing method, an image processing system, and an image processing program for improving the image quality of an image according to the image content.

2. Description of the Related Art Conventionally, there has been known a method of analyzing the properties of an image using a digital image processing technique and converting the image into a high quality image for human vision. For example, in Japanese Patent No. 2692531, “Image Sharpening Method and Apparatus”, when sharpening an image, instead of performing uniform sharpening for all images, an edge region is detected for each image. The degree of sharpening is adjusted based on the value obtained by normalizing the integrated value of the intensity of the high-frequency component or high-frequency band component in the region with the area of the edge region. Similarly, a method of adjusting the contrast and saturation of an image to an optimum value by adapting to the image is also known.

Japanese Patent No. 2692531 (pages 9-16)

However, in the conventional technique, the above-described property analysis is performed on the entire image, and in order to determine the processing according to the result, there is a case where there are objects with different properties in one image. Therefore, it was not possible to execute high image quality according to the target. For example, when adjusting the sharpness for an image in which a blue sky spreads over a building street, subjectively, an image with a high sharpness for an image portion of an artifact such as a building street has a high image quality. As a result, for the image part of the blue sky, it is determined that an image with a more gentle change in luminance value has a high image quality. Since the sharpening process has an effect of amplifying image noise, it may be determined that the image quality has deteriorated due to the influence in the blue sky region. As such, the optimum degree of sharpening is different. However, when the same sharpening is uniformly applied to the entire image, the average value is used, so that the effect may not be improved.

An object of the present invention is to provide a method, system, and program for solving the above-described problems, classifying partial areas of an image, and performing optimum image processing for each area, thereby improving the image quality of the entire image. is there.

The image processing method of the present invention identifies that an original image includes one or a plurality of targets, sets a target region, applies image processing according to the target to the target region, It is characterized by obtaining results.

The image processing method of the present invention obtains the likelihood of a target area including one or a plurality of targets for each position of the original image, and applies image processing according to the target according to the target area likelihood. And processing results are obtained.

Furthermore, the image processing method of the present invention creates one or a plurality of target processing result images obtained by applying image processing suitable for one or a plurality of target images to the original image, and each of the original images For the position, the target image including the target is obtained, and a final image is obtained by integrating the processing result images for the target according to the target region.

Furthermore, the image processing method of the present invention is characterized in that the object includes at least one of a person or a face, sky / clouds, an artifact, a plant, a collective person, and a natural landscape.

Furthermore, the image processing method of the present invention is characterized in that the image processing includes at least one of sharpening processing, contrast enhancement processing, saturation enhancement processing, and white balance processing.

Furthermore, the image processing system of the present invention includes a region index calculating unit that calculates the likelihood of one or a plurality of target regions in each part of the original image, and the original image according to the region index obtained by the region index calculating unit. And an image conversion means for applying an image conversion process.

Furthermore, the image processing system of the present invention includes an image conversion unit that creates one or a plurality of target processing result images obtained by applying image processing suitable for one or a plurality of target images to the original image, A region index calculation unit that calculates a target region probability that the target is included for each position of the original image as a region index, and an image integration unit that integrates the processing result images for the target according to the region index. Features.

Furthermore, the image processing system of the present invention is characterized in that the object includes at least one of a person or a face, sky / clouds, an artifact, a plant, a collective person, and a natural landscape.

Furthermore, the image processing system of the present invention is characterized in that the image processing includes at least one of sharpening processing, contrast enhancement processing, saturation enhancement processing, and white balance processing.

By adopting the configuration of the present invention, it is possible to perform high image quality processing according to the image area, compared with the conventional method in which high image quality processing has been performed based on the properties of the entire image. On the other hand, it is possible to convert to a higher quality image.

  Next, embodiments of the present invention will be described in detail with reference to the drawings.

The operation of the image processing system of this embodiment will be described with reference to FIG. 1 which shows the first embodiment of the present invention in block form.

An example of the original image 1 is assumed to be as shown in the schematic diagram of FIG. In the original image 1, a person in the foreground and a landscape such as a house and a tree and the sky are photographed in the background. In the present embodiment, the original image 1
about,
(1) For human skin, apply sharpening weakly.
(2) Do not sharpen the sky or clouds. The blue sky increases the saturation.
(3) Sharpen the person's clothes and background strongly. Emphasize the contrast and saturation so that they are sharp.
To improve the image quality for human vision. That is, the area identifying unit 11 identifies the skin area of the person with respect to the original image 1. Then, a region image 13 which is a binary image in which the identified region is “1” and the others are “0” is created. For the example of FIG.
The portion indicated by “white” is the portion “1” that is the identified skin region. The image conversion means 15 applies a high image quality process suitable for the skin area to the portion where the area image 13 of the original image 1 is “1”. Similarly, the area identifying means 12 identifies the sky / cloud area, and the area image 14 is created. The image conversion means 16 applies a high image quality process suitable for the sky / cloud region to the portion where the region image 14 of the original image 1 is “1”. Further, a general image quality enhancement process is applied to the entire original image 1 by the image conversion means 17. In contrast to the output of the image conversion means 17, the image composition means 18 outputs the output of the image conversion means 15 for the part where the “skin” area image 13 is “1”, and the area image 14 is “sky / cloud”. For the portion of “1”, the output of the image conversion means 16 is synthesized. When an image is synthesized for each pixel, the image quality may change abruptly at the joint between the regions, and an unnatural image may be generated. Therefore, it is appropriate that the image synthesis unit 18 includes a blurring process between the regions. is there.

In this specification, the target to be identified is described as a person, sky, or cloud. However, an embodiment in which separate processing is performed for each target with different target image quality is desirable. It is possible to identify situations in which images are often taken, such as buildings, natural scenery such as mountains, gathering people, etc., and perform image quality enhancement processing in each situation.

In this specification, the applied image processing is “sharpening”, “contrast emphasis”, and “saturation emphasis”, but includes “white balance processing”, “color change processing to memory color”, etc. But you can.

The first embodiment is ideal, but usually the identification of the object is not perfect, so there may be areas where incorrect image transformation is applied. Based on this, the second embodiment is shown in FIG.
Based on First, the original image 1 is analyzed by the first and second region index calculation means 21 and 22 to determine whether or not it has a property defined for each pixel. As the region properties, for example, the first and second region index calculating means 21 and 22 evaluate the properties of two regions of human skin and sky / cloud, respectively.

Each area index calculating means 21 and 22 determines the likelihood of the area from the image, and outputs the area likelihood as an index having a value of 0 to 1, for example. With reference to FIG. 5, the function of the area | region index calculation means 21 which calculates human skin area-likeness from the color of each pixel is shown. Here, the color of the pixel (
R, G, B) values. If black is (0, 0, 0) and white is (1, 1, 1), all colors are included in a cube as shown in FIG. 5 in the RGB color space. Among these, the region 211 that seems to be the most skin color is “1”, the surrounding region 212 is “0.7”, the surrounding region 213 is “0.3”, and the others are “0”. The index of is preset. Each pixel value is examined to see if it is included in these areas, and the corresponding index F1 is output. Of course, instead of setting discrete values as shown in FIG. 5, the index may be set so as to change continuously like a normal distribution function.

Next, referring to FIG. 6, region index calculating means 21 for determining the likelihood of a human skin, particularly a face region.
Examples will be described. Image data is first input to the image scaling means 214. This is to cope with a change in the size of the face captured in the image. The image data is first reduced to a size of, for example, 0.8 times by the first scaling unit 2151. The reduced image data is output to the first similarity determination unit 2161 and at the same time, the second scaling unit 2
Also output to 152. Similarly, the second scaling unit 2152 reduces the size of the image by 0.8 times (
The original image is reduced to 0.64 times the size). The result is the second similarity determination means 2162.
Are simultaneously output to the third scaling unit 2153. This is repeated as necessary to cope with variations in face size. The similarity determination means 2161 to 2163
The brightness of the image is normalized according to the situation, and the similarity with the face template 217 is calculated as a value of 0-1. When a face is present, a high output is obtained in the similarity determination unit whose size matches the template, so that the maximum value selection unit 218 is configured to use the similarity determination unit 2161-21.
The maximum value of output 63 is output as the face area index F2.

When calculating the face area index, the human skin area index F1 is also calculated at the same time, and the determination accuracy can be increased. In that case, a new face area index F can be obtained by a combination of the human skin area index F1 and the face area index F2. For example, the formula (1)
An implementation method such as (2) or (2) is possible, but is not limited thereto.

F = F1 ・ F2 (1)
F = (F1 + F2) / 2 (2)

Similarly, an embodiment of the area index calculating means 22 for calculating the area index C indicating the sky-cloud area likelihood will be described with reference to FIG. Hue saturation calculation means 221 calculates the hue H, saturation S, and brightness V of the pixel from the value of each pixel. For example, conversion to a known HSV space is performed according to Equation 3.

  here,

Then, in a two-dimensional expression excluding luminance as shown in FIG.
However, the saturation S can be defined as the distance from the center. Here, the region 227 is defined as the color region likelihood C1 = 0.3 of the sky or cloud, the region 228 is defined as C1 = 0.6, and the region 229 is defined as C1 = 0.9. The chromaticity evaluation means 222 performs this calculation and outputs C1.

In addition, the uniformity calculation means 223 calculates the uniformity C of the color change from the periphery of each pixel.
2 is evaluated. For example, although it can evaluate by Formula (4), it is not restrict | limited to this.

  Here, V is R, G, B.

The index composition means 224 calculates the sky / cloud area index C from C1 and C2 obtained above.
The calculation method can be realized by Equation (5) or Equation (6), but is not limited to this.

C = C1 ・ C2 (5)
C = (C1 + C2) / 2 (6)

The area index calculation means 21 outputs the face area index F and the area index calculation means 22 outputs the sky / cloud area index C for each pixel, which becomes the area index image 31 and the area index image 32, respectively.

The image conversion means 4 receives the original image 1 and the area index images 31 and 32, applies image quality enhancement processing according to the area, and outputs the resulting image. With reference to FIG. 9, an embodiment of the image conversion means 4 will be described. The sharpening means 41, the contrast enhancement means 42, and the saturation enhancement means 43 perform image processing on the original image 1, and the degree of each processing is the sharpening amount calculation means 44, the contrast enhancement amount calculation means 45, and the saturation. Calculation is performed by the enhancement amount calculation means 46.

For example, when the sharpening means 41 performs sharpening as shown in Equation (7) and the sharpening amount is adjusted by α,

  here,

In the general region, α = 1, but when the amount of sharpening is small in the face region and the image is not applied at all in the sky / cloud region, the sharpening amount calculating means 44, for example, calculates α Calculate as in (8).

  α = 1-0.5F-C (8)

  Here, it is assumed that F and C have no value at the same time.

Further, it is assumed that the contrast emphasizing means 42 measures the contrast of the original image 1 as shown in Expression (9), obtains the original contrast value K, and converts it into the target contrast value K ′ by Expression (10).

  here,

In the sky / cloud region, contrast is enhanced in the same manner as in general, but in the face region, when the face color is not changed, the contrast enhancement amount calculation unit 45 calculates the enhancement amount β as shown in Expression (11). To do.

  β = 1−F (11)

Similarly, the saturation enhancement amount calculation unit 46 is configured to calculate the saturation enhancement amount for the saturation enhancement unit 43 that performs saturation enhancement on a known image in accordance with the value of the region index. .

Subsequently, a third embodiment of the present invention will be described with reference to FIG. If it is considered that the result image to be obtained is equivalent to that of the second embodiment, a face area index and a sky / cloud area index are stored in each pixel of the area index image 31 and the area index image 32, respectively. Correspondingly, the image conversion means 410 applies image conversion processing suitable for the face area to the entire original image 1. Similarly, the image conversion means 420 applies image conversion processing suitable for the sky / cloud region to the entire original image 1. The image conversion unit 430 applies general image conversion processing to the entire original image. Image conversion means 41
A configuration example of 0 is shown in FIG. The image conversion unit 410 includes a sharpening unit 411, a contrast enhancement unit 412, and a saturation enhancement unit 413, which are processing units similar to the image conversion unit 4. The image conversion means 4 calculates and obtains this enhancement amount. Since the image conversion means 410 is an image conversion means for the face area, it performs image conversion with the optimum enhancement amount and outputs the converted image. To do. The image conversion means 420 performs image conversion suitable for the sky / cloud region with the same configuration, and outputs the processing result. The image conversion means 430 performs image conversion suitable for other general areas and outputs a processing result.

The image integration unit 5 integrates the images to which the respective image conversions are applied. For example, the output images of the image conversion means 410, 420, and 430 are respectively g1 (x, y), g2 (x, y),
Let g3 (x, y) and the region index images 31 and 32 be F (x, y) and C (x, y), respectively. The image integration means 5 calculates Expression (12) for each pixel, and results image result (x,
y) is output.

result (x, y) = F (x, y) g1 (x, y) + C (x, y) g2 (x, y)
+ (1-FC) g3 (x, y) (12)

  It is assumed here that F (x, y) and C (x, y) do not have values at the same time. However, in implementation, F (x, y) and C (x, y) can be configured to have values simultaneously. In that case, it is necessary to devise such a restriction that α in Expression (8) or (1-FC) in the third term on the right side of Expression (12) does not take a negative value.

By this image integration processing, a result image is generated by performing image conversion corresponding to each region.

In the above embodiment, each processing means is a component of the present invention, but each processing can also be realized by a computer program. A fourth embodiment of the present invention in that case will be described with reference to FIG. When the process Ru is started, first region index calculation step 23 is executed. In the region index calculation step 23, the original image 1 is input, and for example, the facial region likelihood calculation is executed by a program in each pixel of the original image 1. As a result, the area index image 31 having the face area index indicated by the expression (1) or (2) is output to each pixel. Next, a region index calculation step 24 is executed. In the region index calculation step 24, the original image 1 is input, and, for example, calculation of the sky / cloud region likelihood in each pixel of the original image 1 is executed by a program. As a result, the region index image 32 having the sky / cloud region index represented by the equation (5) or the equation (6) is output to each pixel. Subsequently, an image conversion step 6 is executed. The image conversion step 6 further includes a sharpening step 61, a contrast enhancement step 62, and a saturation enhancement step 63. The respective image processes are stored in the region index image 31 and the region index image 32 for the original image 1. In accordance with the region index, the degree of processing is adjusted and executed as in equations (7) and (10), and the result is output as a result image 7.

As another embodiment of the present invention for realizing each processing as a computer program, a fifth embodiment will be described with reference to FIG. First, for the original image 1,
For example, a region index calculation step 23 for calculating the likelihood of a face region is executed, and the region index image 3
1 is generated. Next, an image conversion step 83 that is an image conversion for the face area is performed on the original image 1 to generate a processing result image 33. Similarly, for example, the region index calculation step 24 for calculating the likelihood of the sky / cloud region is executed, the region index image 32 is generated, and the image conversion step 84 for performing image conversion for the sky / cloud region is executed, and the processing result An image 34 is generated. Subsequently, an image conversion step 85 for performing an image conversion process on a general region is executed on the original image 1 to generate a processing result image 35. Finally, an image integration step 9 for combining the respective processing result images 33, 34, and 35 in accordance with the region index of the region index images 31 and 32 in each pixel is executed, and a final result image 7 is output. In this case, since the processing contents of the two region index calculation steps 23 and 24 and the three image conversion steps 83, 84, and 85 are completely independent, the execution order is arbitrary, and depending on the computer, high-speed execution is performed by parallel processing. It is also possible to do.

It is a block diagram which shows the structure of the 1st Embodiment of this invention. It is a figure which shows the example of an image for demonstrating the function of the 1st Embodiment of this invention. It is a figure which shows that the part shown with white of the 1st Embodiment of this invention is the identified skin area | region. It is a block diagram which shows the structure of the 2nd Embodiment of this invention. It is a figure explaining the function of the area | region index calculation means which calculates the human skin area | region likeness from the color of each pixel of the 2nd Embodiment of this invention. It is a figure explaining the Example of the area | region index | exponent calculation means which determines especially the face area | region likeness in the human skin in the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the area | region index calculation means which calculates the area | region index which shows the area | region likeness of the sky and the cloud of the 2nd Embodiment of this invention. It is a figure explaining the Example of the area | region index calculation means which calculates the area | region index which shows the area | region likeness of the sky and the cloud in the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the Example of the image conversion means in the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the 3rd Embodiment of this invention. It is a block diagram which shows the structure of the image conversion means in the 3rd Embodiment of this invention. It is a flowchart explaining the 4th Embodiment of this invention. It is a flowchart explaining the 5th Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Original image 11, 12 Area identification means 13, 14 Area image 15, 16, 17 Image conversion means 18 Image composition means 21, 22 Area index calculation means 23, 24 Area index calculation steps 31, 32 Area index images 33, 34, 35 processing result image 4,410,420,430 image conversion means 41 sharpening means 42 contrast enhancement means 43 saturation enhancement means 44 sharpening amount calculation means 45 contrast enhancement amount calculation means 46 saturation enhancement amount calculation means 5 image integration means 6, 83, 84, 85 Image conversion step 7 Result image 9 Image integration step

Claims (6)

For each part of the original image, the color of the pixel and the uniformity of the color change from the surroundings are used to obtain a plurality of types of target areas, and each part of the original image is determined according to the types of target areas. An image processing method characterized by applying image processing and obtaining a processing result. The image processing method according to claim 1, wherein the original image includes sky and clouds as targets . A region index calculating means for calculating a region index indicating a plurality of types of target regions by using the pixel color and the uniformity of color change from the surroundings for each part of the original image, and each part of the original image An image processing system comprising: image conversion means for applying image conversion processing in accordance with a plurality of area indices obtained by the area index calculation means. The image processing system according to claim 3, wherein the original image includes sky and clouds as targets . For each part of the original image, a region index calculating step for calculating a region index indicating the likelihood of a plurality of types of target areas using the pixel color and the uniformity of color change from the surroundings , and each part of the original image An image processing program for causing a computer to execute an image conversion step of applying an image conversion process according to a plurality of region indexes obtained by the region index calculation means. The image processing program according to claim 5, wherein the original image includes sky and clouds as targets .

Images