KR102342283B1 - Image processing apparatus, image processing method, and computer readable medium thereof - Google Patents

Abstract

An image processing apparatus and the like for calculating a diffuse reflection component or a specular reflection component from a single input image with high precision are provided.
The image processing apparatus 1 includes: component calculating units 20 and 30 for calculating a diffuse reflection component and a specular reflection component for each pixel of an input image; a filter processing unit 40 for performing filter processing on the diffuse reflection component image, the specular reflection component image, and the input image, respectively; a component combining unit 50 for generating a combined image by combining the filtered diffuse reflection component image and the filtered specular reflection component image; and evaluation units (60, 70) for evaluating the separation accuracy of the diffuse reflection component or the specular reflection component based on the combined image and the filter-processed input image.

Description

An image processing apparatus, an image processing method, and a computer readable recording medium on which the image processing method is recorded

Various embodiments relate to an image processing apparatus, an image processing method, and a computer-readable recording medium on which the image processing method is recorded.

In image measurement using shading or color for three-dimensional shape restoration or the like, it is sometimes a premise that the image does not have a specular reflection component. In this case, it is necessary for high-precision measurement to separate the diffuse reflection component and the specular reflection component with high precision to remove the specular reflection component.

Moreover, in image recognition, since the specular reflection component becomes a characteristic of a material, a specular reflection component can be detected and used for recognition.

In addition, in image processing, when improving glossiness, it is necessary to extract the range of the specular reflection component to be controlled from the input image.

In order to separate the diffuse reflection component and the specular reflection component, there are mainly two methods.

One is a method of separating from a plurality of images, for example, a method of using photometric stereo or a polarization filter. However, in this method, it is necessary to capture and use a plurality of images, and it cannot be applied to image quality improvement of devices such as displays.

Another method of separating from a single image is a method using a model of color fluctuation due to noise and an assumption that object colors change smoothly on the surface. For example, Japanese Patent Application Laid-Open No. 2013-065215 discloses a method for calculating the diffuse reflection component of a target pixel without being affected by image noise, based on the assumption that the colors of the reflected light are the same if they are the same surface. Color variations due to noise are modeled, and the accuracy of separation of diffuse reflection components is improved by using the model.

However, the method of separating the diffuse reflection component and the specular reflection component from a single image may fail at a portion that becomes a two-color boundary such as an object boundary or a pixel value where saturation becomes 0, and is separated from a plurality of images. Separation precision is lowered compared with the method. For example, with the method described in Japanese Patent Laid-Open No. 2013-065215, the separation accuracy may be lowered for noise that does not fit the noise model.

In addition, when image processing such as filter processing is performed on each of the diffuse reflection component and the specular reflection component with low separation precision for the purpose of obtaining a visual effect, artifacts may occur and image quality may deteriorate.

19 is an image according to the prior art in a case where the separation precision between the diffuse reflection component and the specular reflection component is low. 1910 of FIG. 19 shows an input image, and the shadow of a red tomato and a light purple tomato are reflected on a white background. Also, there is a luster around the green stems of the tomatoes.

1920 in Fig. 19 shows the image of the diffuse reflection component, the tomato is in a red state, but the background is black, and the outline part (boundary surface), the glossy part, and the shadow part of the tomato are pink. And, it can be seen that the separation of the diffuse reflection component has failed in the vicinity of the interface of the tomato.

1930 of FIG. 19 shows an image in which the specular reflection component is obtained based on the diffuse reflection component of 1920, the specular reflection component is emphasized by 1.8 times in the input image, and when compared with the input image, the portion that was the boundary of the tomato, the glossy portion , it can be seen that artifacts have occurred in the shadowed area, etc.

The present invention has been made to solve such a problem, and an object of the present invention is to provide an image processing apparatus, an image processing method, and an image processing program capable of calculating a diffuse reflection component or a specular reflection component from a single input image with high accuracy.

An image processing apparatus according to an exemplary embodiment includes: a component calculator configured to calculate a diffuse reflection component and a specular reflection component for each pixel of an input image; a filter processing unit that performs filter processing on the diffuse reflection component image, the specular reflection component image, and the input image, respectively; a component combining unit for generating a combined image by combining the filtered diffuse reflection component image and the filtered specular reflection component image; and an evaluation unit that evaluates the separation precision of the diffuse reflection component or the specular reflection component based on the combined image and the filter-processed input image.

According to an embodiment, the evaluation unit may calculate a difference value between the combined image and the filtered input image for each pixel, and evaluate the separation precision by comparing the difference value with a predetermined threshold value.

According to an exemplary embodiment, an image processing apparatus includes: a correction value calculator configured to calculate a correction value of a diffuse reflection component or a specular reflection component for each pixel that the evaluation unit determines to correct the diffuse reflection component or the specular reflection component; and a component correction unit that corrects the diffuse reflection component or the specular reflection component by using the correction value, wherein the component calculating unit includes a diffuse reflection component and a specular reflection component based on the input image, the corrected specular reflection component or the corrected diffuse reflection component. The reflection component is calculated, and the evaluation unit may determine that the diffuse reflection component or the specular reflection component of the pixel having a difference value equal to or greater than a predetermined threshold is corrected.

According to an embodiment, the evaluation unit may evaluate the separation precision according to whether each pixel value of the filtered diffuse reflection component image or the filter-processed specular reflection component image is negative.

According to an embodiment, a correction value calculating unit for calculating a correction value of the diffuse reflection component or the specular reflection component for each pixel that the evaluation unit determines to correct the diffuse reflection component or the specular reflection component; and a component correction unit that corrects the diffuse reflection component or the specular reflection component using the correction value, wherein the component calculating unit includes a diffuse reflection component and a modified diffuse reflection component based on the input image, the corrected specular reflection component or the corrected diffuse reflection component The specular reflection component is calculated, and the evaluation unit may determine that the diffuse reflection component or the specular reflection component of the pixel in which the pixel value of the filtered diffuse reflection component image or the filtered specular reflection component image is negative is corrected.

According to an exemplary embodiment, the correction value calculator may calculate a diffuse reflection component or a specular reflection component of a neighboring pixel satisfying a predetermined condition among neighboring pixels of the pixel determined to be corrected as a correction value.

According to an embodiment, the predetermined condition is combined with the difference between the pixel value of the pixel determined to be corrected in the input image and the pixel value of the neighboring pixel, and the pixel value of the filtered input image in the neighboring pixel A sum of values obtained by multiplying a difference with a pixel value of an image by a predetermined coefficient may be the smallest.

According to an embodiment, when the number of pixels before correction is greater than or equal to a predetermined value or the number of pixels after correction is less than or equal to a predetermined value, the repeat determination unit may further include a repeat determination unit configured to repeat the processing by the component calculating unit.

According to an exemplary embodiment, the evaluation unit may calculate a PSNR value between the combined image and the filtered input image, and may evaluate the separation precision by comparing the PSNR value with a predetermined threshold value.

According to an embodiment, the image processing apparatus further includes an iterative determination unit that determines to re-calculate the PSNR value by changing the intensity of the filter used in the filter processing unit when the PSNR value is less than a predetermined threshold, and the evaluation unit includes: Even if the intensity is greater than or equal to a predetermined value, when the PSNR value is less than a predetermined threshold, the separation precision can be evaluated to be low.

According to an embodiment, the filter processing unit may use a high frequency enhancement filter for filter processing.

According to an embodiment, an image processing apparatus includes: an analysis unit configured to perform frequency analysis on an input image, a diffuse reflection component image or a specular reflection component image, and calculate frequency characteristics of each image; and a second evaluation unit that evaluates the separation accuracy based on the frequency characteristics of the input image and the frequency characteristics of the diffuse reflection component image or the specular reflection component image.

According to an embodiment, the filter processing unit stores the frequency characteristics of the diffuse reflection component image or the specular reflection component image and the intensity of the filter used for the filter processing in advance and stores a plurality of them, and the diffuse reflection component image or the specular reflection component calculated by the analysis unit It is possible to determine the strength of the filter used for the filter processing based on the frequency characteristic of the image.

An image processing apparatus according to an embodiment includes: a component calculator configured to calculate a diffuse reflection component or a specular reflection component for each pixel of an input image; a filter processing unit that performs filter processing on the diffuse reflection component image or the specular reflection component image; and an evaluation unit that evaluates the separation accuracy of the diffuse reflection component or the specular reflection component according to whether each pixel value of the filtered diffuse reflection component image or the filtered specular reflection component image is negative.

In addition, an image processing apparatus according to an embodiment includes: a component calculator configured to calculate a diffuse reflection component or a specular reflection component for each pixel of an input image; an analysis unit that performs frequency analysis on the input image and the diffuse reflection component image or the specular reflection component image, and calculates respective frequency characteristics; and an evaluation unit that evaluates the separation accuracy of the diffuse reflection component or the specular reflection component based on the frequency characteristic of the input image and the frequency characteristic of the diffuse reflection component image or the specular reflection component image.

In addition, an image processing method according to an embodiment may include calculating a diffuse reflection component and a specular reflection component for each pixel of an input image; performing filter processing on the diffuse reflection component image, the specular reflection component image, and the input image, respectively; combining the filtered diffuse reflection component image and the filtered specular reflection component image to generate a combined image; and an operation of evaluating the separation precision of the diffuse reflection component or the specular reflection component based on the combined image and the filtered input image.

In addition, according to an embodiment, a computer program product including a recording medium storing a program for performing an image processing method, the image processing method includes: calculating a diffuse reflection component and a specular reflection component for each pixel of an input image; performing filter processing on each of the diffuse reflection component image, the specular reflection component image and the input image; generating a combined image by combining the filtered diffuse reflection component image and the filtered specular reflection component image; and evaluating the separation precision of the diffuse reflection component or the specular reflection component based on the combined image and the filter-processed input image.

According to the present invention, it is possible to provide an image processing apparatus for calculating a diffuse reflection component or a specular reflection component from a single input image with high precision, an image processing method, and a computer-readable recording medium on which the image processing method is recorded.

BRIEF DESCRIPTION OF THE DRAWINGS It is a figure which shows the operation|movement of the image processing method which concerns on Embodiment 1. FIG.
2 is a block diagram showing a schematic configuration of the image processing apparatus 1 according to the first embodiment.
3 is a flowchart showing a processing sequence of the image processing method according to the first embodiment.
FIG. 4 is a diagram for explaining a method for calculating a diffuse reflection component and a specular reflection component and a calculation result according to the first embodiment.
5 is a diagram for explaining a difference processing and a difference image according to the first embodiment.
6 is a diagram showing the effect of improving separation precision according to the first embodiment.
7 is another diagram showing the effect of improving the separation precision according to the first embodiment.
Fig. 8 is another diagram showing the effect of improving the separation precision according to the first embodiment.
9 is a diagram showing the effect of improving image quality and texture according to the first embodiment.
Fig. 10 is a diagram showing the operation of the image processing method according to the second embodiment.
11 is a block diagram showing a schematic configuration of the image evaluation apparatus 2 according to the second embodiment.
12 is a diagram for explaining a filter according to the second embodiment.
13 is a diagram for explaining a relationship between a PSNR value and an artifact.
14 is a diagram showing the operation of the image processing method according to the third embodiment.
15 is a block diagram showing a schematic configuration of the image evaluation apparatus 3 according to the third embodiment.
16 is a diagram illustrating a power spectrum parameter-filter coefficient pair stored in a coefficient storage unit according to an exemplary embodiment.
17 is a diagram for explaining a power spectrum according to the third embodiment.
18 is a separate diagram for explaining a power spectrum according to the third embodiment.
Fig. 19 is an image in the case where the separation precision of the diffuse reflection component and the specular reflection component according to the prior art is low.

(Embodiment 1)

The image processing apparatus and image processing method according to the first embodiment calculates a diffuse reflection component and a specular reflection component from a single image, respectively, and performs filter processing etc. on the diffuse reflection component and the specular reflection component to obtain a diffuse reflection component and a specular reflection component The separation precision of the reflection component is evaluated for each pixel, and the diffuse reflection component with low separation accuracy is corrected using the correction value calculated by the independently defined energy function, etc., thereby improving the separation accuracy of the diffuse reflection component and the specular reflection component for each pixel will make it

Hereinafter, an image processing apparatus and an image processing method according to the first embodiment will be described with reference to the drawings.

First, FIG. 1 shows an example of the operation of the image processing method according to the first embodiment.

Referring to FIG. 1 , in step S110, the image processing apparatus may calculate a diffuse reflection component and a specular reflection component for each pixel of an input image.

In step S120, the image processing apparatus may perform filter processing on the diffuse reflection component image composed of the calculated diffuse reflection component, the specular reflection component image composed of the calculated specular reflection component, and the input image, respectively.

In step S130, the image processing apparatus may generate a combined image by combining the filtered diffuse reflection component image and the filtered specular reflection component image.

In step S140, the image processing apparatus can evaluate the separation precision of the diffuse reflection component or the specular reflection component based on the combined image and the filtered input image.

The configuration of the image processing apparatus according to the first embodiment will be described.

2 is a block diagram showing a schematic configuration of the image processing apparatus 1 according to the first embodiment.

The image processing apparatus 1 includes an image input unit 10 , a diffuse reflection component calculation unit 20 , a specular reflection component calculation unit 30 , a filter processing unit 40 , a component combining unit 50 , and a difference calculation unit 60 . , a correction determination unit 70 , a correction value calculation unit 80 , a component correction unit 90 , a repetition determination unit 100 , and the like. In addition, the diffuse reflection component calculation unit 20 and the specular reflection component calculation unit 30 may be combined to form a component calculation unit, or the difference calculation unit 60 and correction determination unit 70 may be combined to form an evaluation unit. You may.

The image input unit 10 receives, for example, an image from the outside of the image processing apparatus 1 , and converts the input image to a diffuse reflection component calculation unit 20 , a specular reflection component calculation unit 30 , and a filter processing unit 40 . , and output to the correction value calculating unit 80 or the like.

The diffuse reflection component calculation unit 20 calculates a diffuse reflection component for each pixel from a single input image, and the specular reflection component calculation unit 30 , the filter processing unit 40 , the correction value calculation unit 80 , and the component correction unit 90 . ), etc.

The specular reflection component calculating unit 30 calculates a specular reflection component for each pixel from the input image by using the dichroic reflection model of Shaper, and outputs it to the filter processing unit 40 or the like. A method of calculating the specular reflection component using the dichroic reflection model will be described later.

The filter processing unit 40 filters the input image, the diffuse reflection component image composed of the diffuse reflection component, and the specular reflection component image composed of the specular reflection component using a high-frequency enhancement filter such as an unsharp mask or a high-frequency filter. Each is performed, and the result is output to the component combining unit 50 , the difference calculating unit 60 , the correction value calculating unit 80 , and the like.

The component combining unit 50 generates a combined image by combining the filter-processed diffuse reflection component image and the filtered specular reflection component image using the above-described dichroic reflection model, the difference calculating unit 60 and the correction value output to the calculation unit 80 or the like.

The difference calculating unit 60 calculates the difference between the pixel values of the filter-processed input image and the generated combined image for each pixel, and outputs it to the correction determination unit 70 .

The correction determination unit 70 compares the pixel value difference with a predetermined threshold value for each pixel, determines that the diffuse reflection component of the pixel whose difference is greater than the threshold value is corrected, and the diffuse reflection component of the pixel whose difference is equal to or less than the threshold value is determined not to be corrected, and the determination result is output to the correction value calculating unit 80 . Here, the pixel having a difference greater than the threshold is a pixel having low separation precision between the diffuse reflection component and the specular reflection component.

The correction value calculating unit 80 calculates a correction value of the diffuse reflection component for the pixel determined to be corrected for the diffuse reflection component, and outputs it to the component correction unit 90 . A method of calculating the correction value of the diffuse reflection component will also be described later.

The component correction unit 90 replaces the diffuse reflection component with the correction value for the pixel determined to be corrected for the diffuse reflection component, and outputs it to the repetition determination unit 100 .

The repetition determination unit 100 determines whether the correction level of the diffuse reflection component of the image satisfies a predetermined condition, and if the correction level of the diffuse reflection component does not satisfy the predetermined condition, the component correction unit 90 The specular reflection component is calculated again by using the result of correction by , as an input to the specular reflection component calculating unit 30 . In addition, when the correction level of the diffuse reflection component of the image satisfies a predetermined condition, the diffuse reflection component and the specular reflection component calculated for each pixel, or the corrected diffuse reflection component and the specular reflection component are applied to the image processing apparatus It is output to the outside of (1).

Further, in the image processing apparatus 1 according to the first embodiment, each component such as the image input unit 10 and the diffuse reflection component calculation unit 20 is provided in the image processing apparatus 1 which is, for example, a computer. It can be realized by executing a program under the control of an arithmetic device (not shown) such as a processor.

More specifically, the image processing apparatus 1 is realized by loading a program stored in a storage unit (not shown) into a main storage device (not shown) and executing the program under the control of an arithmetic device such as a processor. In addition, each component is not limited to realization by software by a program, and can also be implemented by some combination of hardware, firmware, and software.

In addition, the image processing apparatus 1 may be used in an image display apparatus such as a TV, a smart phone, a tablet PC, a wearable camera, an HMD, and an automobile display.

The above-described program can be stored using various types of recording media, that is, a non-transitory computer readable medium, and supplied to the image processing apparatus 1 . Non-transitory computer-readable media includes various types of tangible storage media. Here, 'non-transitory' means that the storage medium does not include a signal and is tangible, and does not distinguish that data is semi-permanently or temporarily stored in the storage medium.

Examples of non-transitory computer-readable media include magnetic recording media (eg, flexible disk, magnetic tape, hard disk drive), magneto-optical recording media (eg, optical magnetic disk), CD-ROM (Read Only Memory), CD-R, CD-R/W, semiconductor memory (eg, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), Flash ROM, RAM (random access memory)).

Further, the program may be supplied to the image processing apparatus 1 by various types of transitory computer readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer readable medium may supply the program to the image processing apparatus 1 through a wired communication path such as an electric wire and optical fiber, or a wireless communication path.

In addition, the image processing apparatus or method according to the disclosed embodiments may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product may include a S/W program and a computer-readable storage medium in which the S/W program is stored.

Next, the operation of the image processing apparatus 1 according to the first embodiment, ie, an image processing method, will be described in more detail.

3 is a flowchart showing a processing procedure of the image processing method according to the first embodiment.

When the image processing apparatus 1 starts operation, the image input unit 10 receives an image (step S10), and the diffuse reflection component calculation unit 20 calculates a diffuse reflection component of the input image for each pixel (step S20). ). The diffuse reflection component calculating unit 20 calculates each diffuse reflection component using a conventional method.

Next, the specular reflection component calculating unit 30 calculates a specular reflection component for each pixel from the input image using the dichroic reflection model of Shaper (step S30).

Specifically, the specular reflection component calculating unit 30 inputs the pixel value P and the diffuse reflection component α*D of the input image in the dichroic reflection model as shown in Equation (1), and the specular reflection component β*S to calculate

Here, α is a diffuse reflection intensity coefficient, β is a specular reflection intensity coefficient, D is a diffuse reflection chromaticity, S is a specular reflection chromaticity indicates.

FIG. 4 is a diagram for explaining a method of calculating a diffuse reflection component and a specular reflection component and a calculation result according to the first embodiment. A diffuse reflection component and a specular reflection component are calculated from the input image 410 of the tomato.

In the diffuse reflection component image 420 shown in Fig. 4, the tomato is in a red state, but the background is black, and the outline part (boundary surface), the glossy part, and the shadow of the tomato are peach. In addition, in the specular reflection component image 430, the background and the glossy part are gray, and the tomato, the stem, and the tomato's shadow are black.

Next, the filter processing unit 40 performs filter processing on the input image, the diffuse reflection component image, and the specular reflection component image, respectively, using a high-frequency enhancement filter, a high-frequency filter, or the like (step S40). The diffuse reflection component or specular reflection component of a pixel with low separation precision is emphasized by filter processing, such as a high frequency enhancement filter.

Next, the component combining unit 50 linearly combines the filtered diffuse reflection component image and the filtered specular reflection component image using the dichroic reflection model to generate a combined image (step S50).

Next, the difference calculation unit 60 calculates the difference between the pixel values of the filter-processed input image and the generated combined image for each pixel (step S60).

5 is a diagram for explaining difference processing and a difference image according to the first embodiment. Here, unsharp masking or high-frequency filtering is used for the filter processing, and the pixel value difference between the filtered input image 510 and the combined image 520 is calculated as the difference image 530 . In the difference image shown in Fig. 5, only pixels having a difference value greater than a predetermined threshold are shown.

In the combined image 520 , a part of the interface, apex, and shadow of the red tomato is colored white, and in the difference image 530 , the white part and the like are shown as pixels with a large difference. As described above, a pixel having a difference greater than the threshold is a pixel having a low separation precision between the diffuse reflection component and the specular reflection component.

Next, the correction determination unit 70 compares the pixel value difference with a predetermined threshold value for each pixel, and determines the pixel having a difference greater than the threshold value as a pixel with low separation accuracy of the diffuse reflection component or the specular reflection component, and the It is determined that the diffuse reflection component of the pixel is corrected. Further, the correction determination unit 70 determines that the pixel whose difference is equal to or less than the threshold is a pixel with high separation precision between the diffuse reflection component and the specular reflection component, and determines that the diffuse reflection component of the pixel is not corrected (step S70).

Such judgment and judgment are based on the assumption that a combined image combined by applying a filter process to each reflection component image and an image obtained by subjecting an input image to the same filter process will become the same image.

As a result of comparing the pixel value difference with a predetermined threshold, the correction determination unit 70 determines that the diffuse reflection component of the pixel having a large difference value in the difference image shown in FIG. 5 is corrected, for example.

Then, with respect to the pixel (hereinafter, referred to as a “pixel of interest”) that the correction determination unit 70 has determined to correct the diffuse reflection component (YES in step S70), the correction value calculating unit 80 selects one of the pixels surrounding the pixel of interest. The diffuse reflection component of the peripheral pixel satisfying the predetermined condition is calculated as a correction value of the diffuse reflection component of the pixel of interest (step S80).

Specifically, the correction value calculator 80 defines an energy function (or similarity function, also called a cost function) as an objective function as in Equation (2), and the neighboring pixel having the smallest value of the energy function A diffuse reflection component of is taken as the correction value of the pixel of interest (A and B are weighting coefficients).

That is, the correction value calculating unit 80 selects, in an input image, e.g., an RGB image, a pixel having the same color or a color similar to the color of the pixel of interest, and having a high separation precision between the diffuse reflection component and the specular reflection component, as a peripheral pixel. search is performed, and the diffuse reflection component of the neighboring pixel is determined as the correction value of the pixel of interest.

Also, the correction value calculator 80 may set one of the weighting coefficients A and B to 0 to search for neighboring pixels.

In addition, the correction value calculating unit 80 may limit the range of searching for surrounding pixels to a range calculated by the number of pixels having a specular reflection component greater than a predetermined threshold and can be regarded as having a specular reflection component. may be As the threshold value at this time, for example, the average value of the entire image for the specular reflection component, the value that maximizes the Fisher ratio (discrimination ratio) in Otsu's threshold value determination method, Kittler's threshold value determination method A value that minimizes the standard for the average misidentification rate can be used.

Next, the component correction unit 90 replaces the diffuse reflection component of the pixel of interest with the correction value (step S90).

Next, the repetition determination unit 100 determines that the correction level of the diffuse reflection component of the image is a predetermined condition, for example, the number of pixels with low separation precision of the diffuse reflection component is less than or equal to a predetermined value, or the number of corrections of the diffuse reflection component is greater than or equal to a predetermined value, etc. It is determined whether or not to repeat the correction process of the diffuse reflection component according to whether the condition of is satisfied (step S100).

Then, if the repetition determination unit 100 determines that the correction process is repeated because the correction level of the diffuse reflection component of the image does not satisfy the predetermined condition (YES in step S100), for example, it returns to step S30 and returns to the diffuse reflection The specular reflection component is calculated again by using the output of the component correction processing (step S90) as an input of the specular reflection component calculation processing (step S30).

In addition, when it is determined that the correction level of the diffuse reflection component of the image satisfies the predetermined condition and the correction process is not repeated (NO in step S100), the repetition determination unit 100 performs steps S20, The diffuse reflection component calculated in step 30, the specular reflection component, or the diffuse reflection component corrected in step S90 are output to the outside of the image processing apparatus 1, and the image processing is terminated.

In addition, when the correction determining unit 70 determines that the diffuse reflection component of a certain pixel is not to be corrected (NO in step S70), the process proceeds to step S100, and the repetition determination unit 100 repeats the diffuse reflection component correction process. determine whether

Further, in the image processing method according to the first embodiment, as the number of repetitions of the correction processing for the diffuse reflection component increases, the separation accuracy between the diffuse reflection component and the specular reflection component can be expected to be improved.

6 is a diagram showing the effect of improving the separation precision according to the first embodiment. A computer graphic (CG) image in which a sphere having red and green checkered patterns is drawn on a black background is used as the input image. 610 and 620 of FIG. 6 are images each representing a diffuse reflection component, and 630 and 640 of FIG. 6 are images representing a specular reflection component, respectively. In addition, 610 and 630 in FIG. 6 each indicate an image before separation precision improvement, and 620 and 640 in FIG. 6 each indicate an image after separation precision improvement.

In these images, with respect to the peak signal-to-noise ratio (PSNR) to ground truth or true value, 4.5 dB in the diffuse reflection component and 9.51 dB in the specular reflection component were confirmed, respectively.

In addition, if image processing such as filter processing is performed independently on the diffuse reflection component and the specular reflection component calculated or corrected as described above, the image quality, particularly the texture, glossiness, etc. of the image displayed on a display or the like can be improved. .

7 is another diagram showing the effect of improving the separation precision according to the first embodiment. 710, 720, and 730 in Fig. 7 denote an input image, a specular reflection component enhancement image before enhancement, and a specular reflection component enhancement image after enhancement, respectively.

It can be seen that the specular reflection component-emphasized image after the separation precision improvement has improved gloss compared to the input image, and the artifacts near the tomato boundary are reduced compared with the specular reflection component-emphasized image before the improvement.

Fig. 8 is another view showing the effect of improving the separation precision according to the first embodiment. Reference numerals 810 and 820 in Fig. 8 denote specular reflection component enhanced images before improvement, and 830 and 840 in Fig. 8 denote specular reflection component enhanced images after improvement. In addition, 820 and 840 of FIG. 8 are partially enlarged images of 810 and 830 of FIG. 8 , respectively. A CG image as shown in Fig. 6 is used for the input image.

In the specular reflection component-emphasized image before separation precision improvement, artifacts, such as black lines, appear at the boundary between red and green in a checkered pattern. can

Fig. 9 is a view showing the image quality and texture improvement effect of the image according to the first embodiment. Reference numeral 910 in Fig. 9 denotes an input image, and reference numeral 920 in Fig. 9 denotes an image in which image processing is separately performed on the diffuse reflection component and the specular reflection component. In any image, red-purple petals and a yellow or yellow-green calyx in the center are stamped, but in the image after image processing in 920 of FIG.

In addition, various modifications are possible in the image processing apparatus 1 or the image processing method according to the first embodiment.

For example, in the image processing apparatus 1 according to the first embodiment, the correction value calculating unit 80 calculates the correction value of the diffuse reflection component of the pixel of interest based on the energy function, and the component correction unit 90 Although the diffuse reflection component of the pixel of interest is corrected, the correction value calculating unit 80 and the component correction unit 90 perform a filter process such as a median filter on the area centered on the pixel of interest to obtain the diffuse reflection component of the pixel of interest. You can also make it editable. In this case, for example, the median value calculated by the median filter becomes the correction value of the diffuse reflection component.

Further, in the image processing apparatus 1 according to the first embodiment, the correction determining unit 70 determines that the diffuse reflection component is to be corrected for each pixel, and corrects the diffuse reflection component in the subsequent processing. However, the correction determination unit 70 A pixel that 70 determines that the specular reflection component is corrected for each pixel, the correction value calculating unit 80 calculates a correction value of the specular reflection component, and the component correction unit 90 determines that the specular reflection component is corrected , the specular reflection component may be substituted with its correction value. In this case, the diffuse reflection component can be calculated again using the corrected specular reflection component.

Further, in the image processing apparatus 1 according to the first embodiment, the separation precision of the diffuse reflection component or the specular reflection component for each pixel based on the difference between the input image and the combined image filtered by the correction determination unit 70 was evaluated, but the correction determination unit 70 determines whether the pixel value is negative for each pixel of the filtered diffuse reflection component image or the filter-processed specular reflection component image, It is also possible to evaluate the separation precision of the reflection component.

Specifically, the correction determination unit 70 determines that the pixel value of the filtered diffuse reflection component image or the filter-processed specular reflection component image, that is, a pixel in which any one of six values of R, G, and B is negative, is It is determined that the calculation or separation of the diffuse reflection component or the specular reflection component has failed because it has a pixel value that contradicts the dichroic reflection model, and the diffuse reflection component or the specular reflection component of the pixel is corrected.

Also in this case, the correction value calculation unit 80 calculates the correction value of the pixel by using the energy function shown in Equation (2) from the difference between the pixel value of the input image and the pixel value of the combined image after filter processing, which is calculated separately. can do.

In addition, evaluation of the separation precision based on the difference between the input image and the combined image filtered by the correction determination unit 70 and evaluation of the separation precision based on the pixel values of each reflective component image subjected to the filter process are used together, Based on one evaluation, it is determined that the diffuse reflection component or the specular reflection component of the pixel is to be corrected, and the correction value calculating unit 80 may calculate a correction value using the energy function shown in Equation (2).

In addition, the image processing apparatus 1 or the image processing method according to the first embodiment may be configured as an image evaluation apparatus, an image evaluation method, or the like for evaluating the separation accuracy of a diffuse reflection component or a specular reflection component. For example, in the case of configuring as an image evaluation apparatus, the correction determination unit 70 , the correction value calculation unit 80 , the component correction unit 90 , and the repetition determination unit 100 according to the first embodiment will be described later instead of the As in the second embodiment to be described, the same thing as an evaluation unit for separation accuracy can be provided.

As described above, the image processing apparatus 1 according to the first embodiment includes the component calculating units 20 and 30 for calculating the diffuse reflection component and the specular reflection component for each pixel of the input image, the diffuse reflection component image, A filter processing unit 40 that performs filter processing on the specular reflection component image and the input image, respectively, and a component combining unit 50 that combines the filtered diffuse reflection component image and the filtered specular reflection component image to generate a combined image ) and evaluation units (60, 70) for evaluating the separation accuracy of the diffuse reflection component or the specular reflection component based on the combined image and the filtered input image.

With this configuration, it is possible to appropriately evaluate the separation precision of the diffuse reflection component or the specular reflection component calculated from a single input image, and calculate the diffuse reflection component or the specular reflection component with high accuracy.

Further, in the image processing apparatus 1 according to the first embodiment, the evaluation units 60 and 70 calculate a difference value between the combined image and the filtered input image for each pixel, and calculate the difference value and the predetermined threshold. It is preferable to compare the values to evaluate the separation precision of the diffuse reflection component or the specular reflection component.

With this configuration, it is possible to evaluate the separation accuracy of the diffuse reflection component or the specular reflection component for each pixel.

Further, the image processing apparatus 1 according to the first embodiment calculates the correction value of the diffuse reflection component or the specular reflection component for each pixel determined by the evaluation units 60 and 70 to correct the diffuse reflection component or the specular reflection component. and a component correction unit 90 for correcting a diffuse reflection component or a specular reflection component using the correction value, wherein the component calculation units 20 and 30 are The diffuse reflection component and the specular reflection component are calculated based on the specular reflection component or the corrected diffuse reflection component, and the evaluation units 60 and 70 correct the diffuse reflection component or the specular reflection component of the pixel whose differential value is equal to or greater than a predetermined threshold. It is preferable to judge that

With this configuration, it is possible to correct the diffuse reflection component or the specular reflection component for each pixel.

In addition, the image processing apparatus 1 according to the first embodiment is configured according to whether the value of the diffuse reflection component image filtered by the evaluation unit 70 or the specular reflection component image subjected to the filter processing is negative depending on whether the diffuse reflection component or the specular reflection component image is negative. It is desirable to evaluate the separation precision of the reflection component.

With this configuration, it is possible to more simply evaluate the separation accuracy of the diffuse reflection component or the specular reflection component for each pixel.

In addition, the image processing apparatus 1 according to the first embodiment is a correction that calculates a correction value of the diffuse reflection component or the specular reflection component for each pixel determined by the evaluation unit 70 to correct the diffuse reflection component or the specular reflection component It further includes a value calculating unit 80 and a component correcting unit 90 for correcting a diffuse reflection component or a specular reflection component using the correction value, and the component calculation units 20 and 30 include the input image and the corrected specular reflection. The diffuse reflection component and the specular reflection component are calculated based on the component or the corrected diffuse reflection component, and the evaluation unit 70 diffuses the pixels whose pixel values of the filtered diffuse reflection component image or the filtered specular reflection component image are negative. It is preferable to judge that the reflection component or the specular reflection component is corrected.

With this configuration, it is possible to easily correct the diffuse reflection component or the specular reflection component for each pixel.

Further, in the image processing apparatus 1 according to the first embodiment, the correction value calculating unit 80 is a diffuse reflection component or specular reflection of a neighboring pixel satisfying a predetermined condition among neighboring pixels of the pixel determined to be corrected. It is preferable to calculate the component as a correction value.

With this configuration, it is possible to easily calculate the correction value of the diffuse reflection component or the specular reflection component.

Further, in the image processing apparatus 1 according to the first embodiment, the predetermined condition is the difference between the pixel value of the pixel determined to be corrected in the input image and the pixel value of the neighboring pixel, and the It is preferable that the sum of the values obtained by multiplying the difference between the pixel values of the filtered input image and the pixel values of the combined image by a predetermined coefficient is the smallest.

With such a configuration, the correction value of the diffuse reflection component or the specular reflection component can be more appropriately calculated.

Further, in the image processing apparatus 1 according to the first embodiment, when the number of pixels before correction is equal to or greater than a predetermined value or the number of pixels after correction is less than or equal to a predetermined value, the repeating plate for repeating the processing by the component calculation units 20 and 30 It is preferable to further include the government (100).

With such a configuration, the diffuse reflection component or the specular reflection component can be calculated more accurately.

Further, in the image processing apparatus 1 according to the first embodiment, the filter processing unit 40 preferably uses a high-frequency enhancement filter for the filter processing.

With such a configuration, it is possible to more appropriately evaluate the separation accuracy of the diffuse reflection component or the specular reflection component.

Further, the image processing apparatus 1 according to the first embodiment includes: component calculating units 20 and 30 for calculating a diffuse reflection component or a specular reflection component for each pixel of an input image; a filter processing unit 40 that performs filter processing on the diffuse reflection component image or the specular reflection component image; and an evaluation unit 70 that evaluates the separation accuracy of the diffuse reflection component or the specular reflection component according to whether each pixel value of the filtered diffuse reflection component image or the filtered specular reflection component image is negative.

With such a configuration, it is possible to easily evaluate the separation accuracy of the diffuse reflection component or the specular reflection component.

Further, the image processing method according to the first embodiment includes steps S20 and S30 of calculating a diffuse reflection component and a specular reflection component for each pixel of the input image; step S40 of performing filter processing on the diffuse reflection component image, the specular reflection component image, and the input image, respectively; a step S50 of generating a combined image by combining the filtered diffuse reflection component image and the filtered specular reflection component image; It has steps S60 and S70 of evaluating the separation precision of the diffuse reflection component or the specular reflection component based on the combined image and the filter-processed input image.

With this configuration, it is possible to appropriately evaluate the separation precision of the diffuse reflection component or the specular reflection component calculated from a single input image, and calculate the diffuse reflection component or the specular reflection component with high accuracy.

Further, the image processing program according to the first embodiment includes a procedure for calculating a diffuse reflection component and a specular reflection component for each pixel of an input image to a computer; an order of performing filter processing on the diffuse reflection component image, the specular reflection component image, and the input image, respectively; a procedure of generating a combined image by combining the filtered diffuse reflection component image and the filtered specular reflection component image; This is for executing a procedure of evaluating the separation precision of the diffuse reflection component or the specular reflection component based on the combined image and the input image subjected to the filtering process.

With this configuration, it is possible to appropriately evaluate the separation precision of the diffuse reflection component or the specular reflection component calculated from a single input image, and calculate the diffuse reflection component or the specular reflection component with high accuracy.

(Embodiment 2)

The image processing apparatus 1 or the image processing method according to the first embodiment evaluated the separation precision of the diffuse reflection component or the specular reflection component for each pixel based on the difference between the filtered input image and the combined image. The image evaluation apparatus or image evaluation method according to aspect 2 of is to evaluate the separation precision of the diffuse reflection component or the specular reflection component of the whole image based on the PSNR value between the filter-processed input image and the combined image.

Hereinafter, the image evaluation apparatus and image evaluation method which concern on this Embodiment 2 are demonstrated based on the structure of an image evaluation apparatus.

First, Fig. 10 shows an example of the operation of the image processing method according to the second embodiment.

Referring to FIG. 10 , in step S1010, the image processing apparatus may calculate a diffuse reflection component or a specular reflection component for each pixel of an input image.

In step S1020, the image processing apparatus may perform filter processing on each of the diffuse reflection component image based on the calculated diffuse reflection component, the calculated specular reflection component, and the input image.

In step S1030, the image processing apparatus may generate a combined image by combining the filtered diffuse reflection component image and the filtered specular reflection component image.

In step S1040, the image processing apparatus can evaluate the separation precision of the diffuse reflection component or the specular reflection component based on the PSNR of the combined image and the PSNR of the filtered input image.

In the image evaluation apparatus according to the second embodiment, first, a reflection component is separated into an input image by using an existing method or the like. After that, the coefficient k of unsharp masking is set to a certain value (eg, initial value = 1). Then, using the set coefficient k, unsharp masking filtering is performed with the diffuse reflection component and the specular reflection component, respectively, and the results are linearly combined based on the dichroic reflection model. Using the same coefficient k as the result and the input image, the image subjected to unsharp masking and the peak signal-to-noise ratio (PSNR) are calculated. Assume that the reflection component separation is precisely calculable if its PSNR is above a certain limit. The reason is based on the assumption that an image combined with unsharp masking applied to each reflection component and an image applied with unsharp masking applied to the input image will be the same. If the PSNR is less than or equal to a certain threshold, the coefficient k of unsharp masking is changed to (k=k+1), and the same calculation is repeated. If k does not exceed the limit of PSNR up to a certain value, it is said that the reflection component separation is not performed correctly.

11 is a block diagram showing a schematic configuration of the image evaluation apparatus 2 according to the second embodiment.

The image evaluation apparatus 2 includes an image input unit 110 , a diffuse reflection component calculation unit 120 , a specular reflection component calculation unit 130 , a coefficient determination unit 140 , a filter processing unit 150 , and a component combining unit 160 . , a PSNR calculation unit 170 , a precision evaluation unit 180 , a repetition determination unit 190 , and the like.

Also, in the image evaluation apparatus 2 , the PSNR calculation unit 170 and the precision evaluation unit 180 may be combined to constitute an evaluation unit, and the coefficient determining unit 140 may be included in the filter processing unit 150 .

The image input unit 110 receives, for example, an image from the outside of the image evaluation device 2 , and converts the input image into a diffuse reflection component calculation unit 120 , a specular reflection component calculation unit 130 , and a filter processing unit 150 . output, etc.

The diffuse reflection component calculation unit 120 and the specular reflection component calculation unit 130, similarly to the diffuse reflection component calculation unit 20 and the specular reflection component calculation unit 30 of the first embodiment, extract a diffuse reflection component from a single input image. , specular reflection components are calculated, respectively, and output to the filter processing unit 150 or the like.

The coefficient determining unit 140 determines, for example, the coefficient k of the filter used by the filter processing unit 150 as "1" as its initial value, and outputs it to the filter processing unit 150 , the iteration determination unit 190 , or the like. The coefficient k of the filter will be described later.

The filter processing unit 150 performs sharpening processing using a high-frequency enhancement filter or a high-frequency filter on the input image, the diffuse reflection component image, and the specular reflection component image, respectively, as in the first embodiment, and converts the result to the component image. It is output to the combining unit 160 , the PSNR calculating unit 170 , and the like.

12 is a diagram for explaining a filter according to the second embodiment.

In the second embodiment, in the filter processing, for example, sharpening processing for emphasizing the contrast of each image is performed using unsharp mask processing, in this case, the difference image between the original image 1210 and the smoothed image 1220 . is multiplied by the filter coefficient k to generate an unsharp mask image 1230 .

As in the first embodiment, the component combining unit 160 linearly combines the filtered diffuse reflection component image and the filtered specular reflection component image using the dichroic reflection model to generate a combined image, and a PSNR calculating unit output as (170).

The PSNR calculation unit 170 calculates a PSNR value between the filter-processed input image and the generated combined image, and outputs it to the precision evaluation unit 180 . A conventional technique can be used for calculation of a PSNR value.

The precision evaluation unit 180 compares the PSNR value with a predetermined threshold value (eg, 40 dB), and outputs the result to the repetition determination unit 190 . The precision evaluation unit 180 determines that the diffuse reflection component and the specular reflection component are separated with high precision when the PSNR value is equal to or greater than a predetermined threshold value, and when the PSNR value is less than the predetermined threshold value, the diffuse reflection component and the specular reflection component It is judged that the components are not separated with high precision.

For example, if the PSNR is 40 dB or more, it is assumed that the reflection components are separated with high precision. The reason is that if the PSNR is 40 dB or more, it becomes difficult to visually determine the difference between the images. That is, even if the reflection component separation is performed with high precision and the specular reflection component is emphasized and glossiness is improved in that state, it can be considered that no artifacts are generated in the image with the naked eye.

13 is a diagram for explaining a relationship between a PSNR value and an artifact.

1300A in Fig. 13 shows an example in which separation fails due to low PSNR, 1300A1 is an input image after filter processing, 1300A2 is a combined image, and 1300A3 is an image in which the specular reflection component is emphasized by 1.8 times in the input image.

1300A1 of FIG. 13 is an overall bluish image in which a transparent sphere is reflected on the specular bottom surface, the sphere or background is reflected in the bottom surface, and the background is also reflected in the sphere. The separation precision between the diffuse reflection component and the specular reflection component is low, and the PSNR value is 15.37 dB. The input image 1300A1 after the filter processing and the combined image 1300A2 are significantly different, and many artifacts are generated in the specular reflection component enhanced image 1300A3.

1100B in FIG. 13 shows an example in which separation fails due to high PSNR, 1300B1 is an input image after filter processing, 1300B2 is a combined image, and 1300B3 shows an image in which the specular reflection component is emphasized by 1.8 times in the input image.

1300B in FIG. 13 is an image in which a brown face is drawn on a black background. The separation precision between the diffuse reflection component and the specular reflection component is high, and the PSNR value is 41.08 dB. The difference between the input image 1300B1 after the filter processing and the combined image 1300B2 is small, and no artifacts are generated even in the specular reflection component enhanced image 1300B3, and the gloss around the forehead and cheeks is emphasized.

As shown in Fig. 13, in an image having a PSNR of 40 dB or more, it can be seen that no artifacts are generated even when the specular reflection component is enhanced by 1.8 times.

The repetition determination unit 190 outputs the diffuse reflection component and the specular reflection component to the outside of the image evaluation apparatus 2 when the PSNR value is equal to or greater than a predetermined threshold. In addition, when the PSNR value is less than a predetermined threshold, the iteration determination unit 190 changes the coefficient k of the filter, and once more the processing from the coefficient determination unit 140 to the precision evaluation unit 180, that is, the PSNR value. An instruction to change the filter coefficient k so as to calculate and evaluate the separation precision is output to the coefficient determining unit 140 .

Then, the coefficient determining unit 140 increases the filter coefficient k by 1 to, for example, “2” in order to change the intensity of the filter, in this case, the intensity of the sharpening process from “1”, which is the initial value, to “2” to increase the filter processing unit ( 150) is output. Subsequent processing from the filter processing unit 150 to the repetition determination unit 190 is as described above.

The repetition determination unit 190 repeats the process of calculating the PSNR value and evaluating the separation precision until the filter coefficient k becomes a predetermined value, for example, 20, but when the PSNR value does not exceed the predetermined threshold value, the diffusion It is determined that the reflection component and the specular reflection component are not separated with high precision, and the fact is output to the outside of the image evaluation device 2 .

In addition, the repetition determination unit 190 calculates the diffuse reflection component and the specular reflection component again when the PSNR value does not exceed a predetermined threshold value even when the filter coefficient k becomes a predetermined value, and the diffuse reflection component calculation unit ( 120) and the specular reflection component calculating unit 130 may output such an indication.

Also, in the second embodiment, the coefficient determining unit 140 may determine the initial value of the coefficient k of the filter based on the luminance of the input image. For example, by using the average luminance value of the entire input image, it may be determined that the coefficient k is 0 when the average luminance value is 0, and the coefficient k is 10 when the average luminance value is 255.

Moreover, the image evaluation apparatus 2 and image evaluation method which concern on this Embodiment 2 can also be comprised as an image processing apparatus and an image processing method.

Further, in the image evaluation apparatus 2 according to the second embodiment, as in the first embodiment, the diffuse reflection component or the specular reflection component of the pixel with poor separation accuracy is separated from the pixel with good separation accuracy using an energy function or the like. It may be substituted with a diffuse reflection component or a specular reflection component.

Further, the image evaluation apparatus 2 according to the second embodiment may be configured as an image processing apparatus, and may be used in combination with the image processing apparatus 1 according to the first embodiment. By evaluating the separation precision of the diffuse reflection component or the specular reflection component on a pixel-by-pixel basis by the image processing apparatus 1 according to the first embodiment and evaluating the entire image by the image processing apparatus according to the second embodiment, more Separation precision can be appropriately evaluated.

As described above, in the image processing apparatus 2 according to the second embodiment, in the image processing apparatus 2 according to the second embodiment, the evaluation units 170 and 180 calculate the PSNR value of the combined image and the filter-processed input image, the PSNR value and the predetermined value It is preferable to compare the threshold values to evaluate the separation precision of the diffuse reflection component or the specular reflection component.

With this configuration, it is possible to evaluate the separation accuracy of the diffuse reflection component or the specular reflection component of the entire image.

Further, in the image processing apparatus 2 according to the second embodiment, when the PSNR value is smaller than a predetermined threshold, the repetition determination unit 190 determines to calculate the PSNR value again by changing the intensity of the filter used in the filter processing unit. ), and the evaluation units 170 and 180 evaluate that the separation precision of the diffuse reflection component or the specular reflection component is poor when the PSNR value is less than a predetermined threshold value, even if the filter intensity is greater than or equal to a predetermined value. desirable.

With such a configuration, it is possible to more appropriately evaluate the separation accuracy of the diffuse reflection component or the specular reflection component of the entire image.

(Embodiment 3)

The image processing apparatus, the image evaluation apparatus, etc. according to the first and second embodiments evaluated the separation precision of the diffuse reflection component or the specular reflection component based on the filter-processed input image and the combined image, but the image according to the third embodiment An evaluation apparatus or image evaluation method evaluates the separation precision of a diffuse reflection component or a specular reflection component by frequency-analyzing an input image and a diffuse reflection component image or a specular reflection component image.

Hereinafter, the image evaluation apparatus and image evaluation method which concern on this Embodiment 3 are demonstrated based on the structure of an image evaluation apparatus.

First, Fig. 14 shows an example of the operation of the image processing method according to the third embodiment.

Referring to FIG. 14 , in step S1410, the image processing apparatus may calculate a diffuse reflection component or a specular reflection component for each pixel of an input image.

In step S1420, the image processing apparatus performs frequency analysis on the input image, the diffuse reflection component image based on the calculated diffuse reflection component, and the specular reflection component image based on the calculated specular reflection component, and calculates each frequency characteristic. can

In step S1430, the image processing apparatus can evaluate the separation precision of the diffuse reflection component or the specular reflection component based on the frequency characteristic of the input image, the frequency characteristic of the diffuse reflection component image, and the frequency characteristic of the specular reflection component image.

15 is a block diagram showing a schematic configuration of the image evaluation apparatus 3 according to the third embodiment.

The image evaluation device 3 includes an image input unit 210 , a component calculation unit 220 , an analysis unit 230 , a precision evaluation unit 240 , and the like.

The image input unit 210 receives an image from the outside of the image evaluation device 3 or the like, and outputs the input image to the component calculating unit 220 or the like.

The component calculating unit 220 calculates a diffuse reflection component or a specular reflection component from a single input image, and outputs it to the analysis unit 230 or the like. Calculation of the diffuse reflection component or the specular reflection component may be performed in the same manner as in the first and second embodiments.

The analysis unit 230 performs two-dimensional Fast Fourier Transform (FFT) on the input image and the diffuse reflection component image or the specular reflection component image, respectively, and converts the result into a power spectrum to the precision evaluation unit 240 . print out

17 is a diagram for explaining a power spectrum according to the third embodiment.

In 1700A of Fig. 17, reference numeral 1710 denotes an input image, 1720 denotes an FFT image of the input image, and 1730 denotes a power spectrum, in 1700B in Fig. 17, 1740 denotes a diffuse reflection component image, 1750 denotes an FFT image of the diffuse reflection component image, and 1760 denotes an FFT image and 1760 of the diffuse reflection component image. represents the power spectrum. In addition, the horizontal axis of the power spectrum indicates the frequency, and the vertical axis indicates the power value.

Compared with the power spectrum 1730 of the input image 1710, the power spectrum 1760 of the diffuse reflection component image 1740 has a large number of peaks and a large power value at most frequencies, so that the diffuse reflection component image contains more high-frequency components than the input image, It can be seen that there is This indicates that the separation precision of the diffuse reflection component is not good.

The precision evaluation unit 240 compares the power spectrum based on the input image and the power spectrum based on the diffuse reflection component image or the specular reflection component image to evaluate the separation precision of the diffuse reflection component or the specular reflection component, and the result is used in the image evaluation apparatus (3) is output to the outside. At this time, as described above, the precision evaluation unit 240 evaluates the separation precision based on the number of peaks in the power spectrum (shape of the power spectrum), the power value, and the like.

Moreover, the image evaluation apparatus 3 and the image evaluation method which concern on this Embodiment 3 can also be comprised as an image processing apparatus and an image processing method.

Further, in the image evaluation apparatus 3 according to the third embodiment, similarly to the first embodiment, both of the diffuse reflection component and the specular reflection component can be respectively calculated, and the separation accuracy is poor using an energy function or the like. The diffuse reflection component or the specular reflection component of the pixel may be substituted with the diffuse reflection component or the specular reflection component of a pixel with good separation accuracy.

Further, the image evaluation apparatus 3 according to the third embodiment is configured as an image processing apparatus, and combined with at least one of the image processing apparatus 1 according to the first embodiment or the image processing apparatus according to the second embodiment can also be used. The separation accuracy of the diffuse reflection component or the specular reflection component is evaluated pixel by pixel by the image processing apparatus 1 according to the first embodiment, and the image processing apparatus according to the second and third embodiments evaluates the entire image By doing so, separation precision can be evaluated more appropriately.

At this time, the combined image processing apparatus includes power spectrum parameters such as the number of peaks and power values of the power spectrum based on the diffuse reflection component image or the specular reflection component image, and the coefficient k of a filter suitable for filtering the image (or filter). Intensity), a plurality of power spectrum parameter-filter coefficient pairs 251 may be stored in advance in the coefficient storage unit 250 . The coefficient storage unit 250 may be, for example, a part of the coefficient determination unit 140 . Referring to FIG. 16 , the power spectrum parameter-filter coefficient pair 251 includes the number of peaks of the power spectrum, the power value of the power spectrum, and filter coefficients. This power spectrum parameter-filter coefficient pair 251 can be obtained experimentally, for example.

Then, when a new image is input, the analysis unit calculates a power spectrum based on the diffuse reflection component image or specular reflection component image of the input image, and the number of peaks, power values, etc. of the power spectrum calculated by the coefficient determination unit 140 It is also possible to determine the coefficient k of a filter suitable for the diffuse reflection component image or the specular reflection component image in question by searching among the power spectral parameter-filter coefficient pairs 251 to which a similar number of peaks, power values, and the like are associated.

18 is a separate diagram for explaining a power spectrum according to the third embodiment.

1810 of 1800A of FIG. 18 denotes a first input image, 1820 denotes an FFT image of the first input image, and 1830 denotes a power spectrum. In FIG. 18, reference numeral 1840 of 1800B denotes a second input image, reference numeral 1850 denotes an FFT image of the second input image, and reference numeral 1860 denotes a power spectrum.

The power spectrum 1830 of the first input image of FIG. 18 has a higher number of peaks than the spectrum 1860 of the second input image, and has a higher overall power value and many high-frequency components. Based on the pair 251, for the first input image of Fig. 14, a coefficient k of a filter that does not emphasize high-frequency components too much is determined to be small, and for the second input image of Fig. 14, a coefficient k of a filter that does not emphasize high-frequency components is determined to be small. largely determines

As described above, the image processing apparatus 3 according to the third embodiment performs frequency analysis on an input image and a diffuse reflection component image or a specular reflection component image, and an analysis unit that calculates frequency characteristics of each image (230) and a second evaluation unit 240 for evaluating the separation accuracy of the diffuse reflection component or the specular reflection component based on the frequency characteristic of the input image and the frequency characteristic of the diffuse reflection component image or the specular reflection component image; It is preferable to have

With such a configuration, it is possible to easily evaluate the separation accuracy of the diffuse reflection component or the specular reflection component of the entire image.

In addition, in the image processing apparatus 3 according to the third embodiment, the filter processing units 140 and 150 associate the frequency characteristics of the diffuse reflection component image or the specular reflection component image with the intensity of the filter used for the filter processing in advance, so that a plurality of It is preferable to memorize and determine the intensity of the filter used for the filter processing based on the frequency characteristics of the diffuse reflection component image or the specular reflection component image calculated by the analysis unit 230 .

With such a configuration, it is possible to easily determine the strength of the filter when filtering the diffuse reflection component image, the specular reflection component image, or the like.

In addition, the image processing apparatus 3 according to the third embodiment includes a component calculating unit 220 that calculates a diffuse reflection component or a specular reflection component for each pixel of the input image, and the input image and the diffuse reflection component image or specular reflection component. An analysis unit 230 that performs frequency analysis on an image and calculates respective frequency characteristics, and a diffuse reflection component or specular reflection based on the frequency characteristic of the input image and the frequency characteristic of the diffuse reflection component image or the specular reflection component image An evaluation unit 240 for evaluating the separation accuracy of components is provided.

With such a configuration, it is possible to easily evaluate the separation accuracy of the diffuse reflection component or the specular reflection component.

Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements by those skilled in the art using the basic concept of the present invention as defined in the following claims are also included in the scope of the present invention. belongs to

Claims (23)

An image processing apparatus comprising:
memory for storing one or more instructions, and
at least one processor executing the one or more instructions;
The at least one processor by executing the one or more instructions,
obtaining a diffuse reflection component and a specular reflection component for each pixel of the input image;
Filter processing is performed using a first filter on the diffuse reflection component image based on the obtained diffuse reflection component and the specular reflection component image based on the obtained specular reflection component, and the first filter is applied to the input image. performing filter processing, combining the filtered diffuse reflection component image and the filtered specular reflection component image to generate a combined image;
evaluating the separation precision of the diffuse reflection component or the specular reflection component based on the combined image and the filter-processed input image;
and determining whether to correct at least one of a diffuse reflection component and a specular reflection component of a pixel of the input image based on the separation precision evaluation result.
According to claim 1,
the at least one processor,
calculating a difference value between the combined image and the filter-processed input image for each pixel;
An image processing apparatus for evaluating the separation precision by comparing the difference value with a predetermined threshold value. 3. The method of claim 2,
the at least one processor,
It is determined that the diffuse reflection component or the specular reflection component of the first pixel having a difference value greater than the predetermined threshold is corrected, and a correction value of the diffuse reflection component or the specular reflection component of the first pixel is calculated and correcting the diffuse reflection component or the specular reflection component of the first pixel by using the correction value. According to claim 1,
the at least one processor,
an image processing apparatus for evaluating the separation precision according to whether each pixel value of the filtered diffuse reflection component image or the filtered specular reflection component image is negative. 5. The method of claim 4,
the at least one processor,
It is determined that the diffuse reflection component or the specular reflection component of a second pixel having a negative pixel value of the filtered diffuse reflection component image or the filtered specular reflection component image is corrected;
obtaining a correction value of the diffuse reflection component or the specular reflection component of a second pixel;
An image processing apparatus for correcting the diffuse reflection component or the specular reflection component of the second pixel by using the correction value. 4. The method of claim 3,
the at least one processor,
an image processing apparatus for obtaining, as the correction value, the diffuse reflection component or the specular reflection component of a peripheral pixel of the first pixel that satisfies a predetermined condition among the peripheral pixels. 7. The method of claim 6,
The predetermined condition is
a difference between the pixel value of the first pixel and the pixel value of the neighboring pixels in the input image, and
and a sum of values obtained by multiplying a difference between a pixel value of the filtered input image and a pixel value of the combined image in the surrounding pixels by a predetermined coefficient is the smallest. 4. The method of claim 3,
the at least one processor,
A new diffuse reflection component and a new specular reflection component are calculated based on the input image, the corrected specular reflection component, or the modified diffuse reflection component, and the diffuse reflection component or the specular reflection component is corrected using the correction value. When the number of pixels before being changed is equal to or greater than a predetermined first value, or when the number of pixels for which the diffuse reflection component or the specular reflection component is corrected using the correction value is less than or equal to a predetermined second value, the diffuse reflection component and the specular surface An image processing apparatus that repeats a reflection component calculation process. According to claim 1,
the at least one processor,
calculating a PSNR value between the combined image and the filter-processed input image;
and comparing the PSNR value with a predetermined threshold value to evaluate the separation precision. 10. The method of claim 9,
the at least one processor,
when the PSNR value is smaller than the predetermined threshold, determining to recalculate the PSNR value by changing the strength of the first filter used in the filter processing;
and evaluating the separation precision based on a new PSNR value obtained based on the changed filter strength. According to claim 1,
the at least one processor,
and the first filter used for the filter processing includes a high-frequency enhancement filter. According to claim 1,
the at least one processor,
frequency analysis is performed on the input image and the diffuse reflection component image or the specular reflection component image, and frequency characteristics of each image are calculated;
an image processing apparatus for evaluating the separation precision based on a frequency characteristic of the input image and a frequency characteristic of the diffuse reflection component image or the specular reflection component image. 13. The method of claim 12,
the at least one processor stores a plurality of pre-associated frequency characteristics of the diffuse reflection component image or the specular reflection component image with the intensity of the first filter used for the filter processing;
An image processing apparatus for determining an intensity of the first filter used for the filter processing based on a frequency characteristic of the diffuse reflection component image or the specular reflection component image. An image processing method comprising:
obtaining a diffuse reflection component and a specular reflection component for each pixel of the input image;
Filter processing is performed using a first filter on the diffuse reflection component image based on the obtained diffuse reflection component and the specular reflection component image based on the obtained specular reflection component, and the first filter is applied to the input image. performing filter processing and combining the filtered diffuse reflection component image and the filtered specular reflection component image to generate a combined image;
evaluating the separation precision of the diffuse reflection component or the specular reflection component based on the combined image and the filter-processed input image; and
and determining whether to correct at least one of a diffuse reflection component and a specular reflection component of a pixel of the input image based on the separation precision evaluation result. 15. The method of claim 14,
The evaluation operation is
calculating a difference value between the combined image and the filter-processed input image for each pixel; and
and evaluating the separation precision by comparing the difference value with a predetermined threshold value. 16. The method of claim 15,
determining whether to correct the diffuse reflection component or the specular reflection component of a first pixel in which the difference value is greater than the predetermined threshold value;
calculating a correction value of the diffuse reflection component or the specular reflection component of the first pixel; and
and correcting the diffuse reflection component or the specular reflection component of the first pixel by using the correction value. delete delete delete 15. The method of claim 14,
and evaluating the separation precision according to whether each pixel value of the filtered diffuse reflection component image or the filtered specular reflection component image is negative. 21. The method of claim 20,
determining that the diffuse reflection component or the specular reflection component of a second pixel having a negative pixel value of the filtered diffuse reflection component image or the filtered specular reflection component image is corrected;
obtaining a correction value of the diffuse reflection component or the specular reflection component of a second pixel; and
and correcting the diffuse reflection component or the specular reflection component of the second pixel by using the correction value. 17. The method of claim 16,
and obtaining, as the correction value, the diffuse reflection component or the specular reflection component of a peripheral pixel that satisfies a predetermined condition among the peripheral pixels of the first pixel. 23. The method of claim 22,
The predetermined condition is
a difference between the pixel value of the first pixel and the pixel value of the neighboring pixels in the input image, and
and a sum of values obtained by multiplying a predetermined coefficient by a difference between a pixel value of the filtered input image and a pixel value of the combined image in the neighboring pixels is the smallest.

Images