CN111563859B - Highlight removing method of double-edge-preserving filter - Google Patents
Highlight removing method of double-edge-preserving filter Download PDFInfo
- Publication number
- CN111563859B CN111563859B CN202010426992.XA CN202010426992A CN111563859B CN 111563859 B CN111563859 B CN 111563859B CN 202010426992 A CN202010426992 A CN 202010426992A CN 111563859 B CN111563859 B CN 111563859B
- Authority
- CN
- China
- Prior art keywords
- maximum
- highlight
- chromaticity
- diffuse reflection
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 20
- 238000001514 detection method Methods 0.000 claims abstract description 24
- 238000010586 diagram Methods 0.000 claims description 12
- 238000001914 filtration Methods 0.000 claims description 11
- 230000000694 effects Effects 0.000 abstract description 12
- 230000002146 bilateral effect Effects 0.000 abstract description 8
- 230000008569 process Effects 0.000 abstract description 2
- 238000009499 grossing Methods 0.000 description 4
- 239000000463 material Substances 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000000926 separation method Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 239000000919 ceramic Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000000615 nonconductor Substances 0.000 description 1
- 230000019612 pigmentation Effects 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 229920003023 plastic Polymers 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000008929 regeneration Effects 0.000 description 1
- 238000011069 regeneration method Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
- 239000002023 wood Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20024—Filtering details
- G06T2207/20028—Bilateral filtering
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides a highlight removing method of a double-edge-preserving filter, which uses double thresholds to replace the traditional single threshold in highlight detection, so that the detection effect is more stable. Then, a new estimated maximum diffuse reflection chrominance estimated value is provided by combining the MSF image principle on the basis of a highlight detection algorithm, the estimated value of the method enables the result to be closer to the real maximum diffuse reflection chrominance value, reliable data are provided for effectively removing highlight in the process, finally, according to the characteristic that the maximum diffuse reflection chrominance has local smoothness, the maximum chrominance is smoothed by using a quick combined bilateral filter and a guide filter, the maximum diffuse reflection estimated value is used as a guide value, the maximum diffuse reflection value is quickly diffused and spread in a neighborhood range, and finally, in order to reduce the influence of specular reflection on diffuse reflection, the maximum diffuse reflection chrominance is determined by selecting the maximum value of the three maximum chrominance values, so that the aim of removing the highlight is fulfilled.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a method capable of effectively removing highlight of an image and simultaneously keeping texture details of an original image.
Background
The two-color reflectance model divides the image reflectance component into a diffuse reflectance component and a specular reflectance component. For a wide variety of non-uniform materials, including plastics, wood, ceramics and other opaque non-conductors with uniform pigmentation, where the reflection is a combination of diffuse and specular reflection, a two-color reflection model can be used to describe well the effects of specular reflection is very important in fields including computer vision, object recognition and editing of image content. Since many algorithms in computer vision and object recognition assume that a scene contains only diffuse reflections, their algorithms will go wrong in the presence of specular reflections. Since specular reflection is related to the roughness of the object surface, the effective separation of specular and diffuse components in an image is of considerable importance for the study of the above-mentioned methods.
Disclosure of Invention
The invention provides a reliable highlight detection and effective highlight removal method, provides a new maximum diffuse reflection maximum chromaticity estimation and uses a fast combined bilateral filter and a guide filter as a double edge-preserving filter to smooth a maximum chromaticity diagram, thereby achieving the purposes of fast and efficiently removing highlight and effectively preserving the details of the image.
The technical scheme of the invention is realized as follows:
(1) The double thresholds are used for replacing the traditional single threshold to detect highlight, so that the possible re-detection and missing detection are effectively avoided, and the detection effect is more stable and reliable.
(2) Based on the MSF image principle provided by the prior art and combined with the highlight detection algorithm of the patent, the maximum diffuse reflection estimation value is calculated, and the experimental result shows that the maximum diffuse reflection estimation value of the method is closer to the real maximum diffuse reflection estimation value, so that reliable data are provided for subsequent highlight removal.
(3) And smoothing the maximum chroma by using a quick joint bilateral filter, guiding the smoothing process by using a maximum diffuse reflection chroma estimated value at the same time to ensure that the maximum diffuse reflection chroma is quickly spread in the field, processing a maximum chroma graph by using a guiding filter, taking the maximum diffuse reflection chroma estimated graph as an index graph, and finally taking the maximum chroma and the maximum chroma after filtering. This allows the final result to be more robust and to better fit the true maximum diffuse reflection chromaticity value after adding the maximum chromaticity value of the guiding filter.
A large number of experiments prove that the algorithm has better robustness, and the texture and color information of the image can be effectively restored while highlight is removed.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of highlight detection algorithm of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
According to the prior art, the reflection component of any light can be divided into two components, diffuse reflection and specular reflection, and the color of the image observed at pixel x can be expressed as follows
I(x)＝wd(x)P d (x)+w s (x)F s (x) (1)
Wherein, I = [ I = [ ] r ,I g ,I b ],P d (x) And F s (x) Respectively a diffuse reflection component and a specular reflection component at pixel x, w d And w s Respectively, representing the diffuse and specular reflection coefficients of pixel x, which depend on the scene geometry at the image pixel.
First we define the chromaticity as:
the diffuse reflectance chromaticity and specular reflectance chromaticity are defined next:
in the formula (2), α represents chromaticity related to a pixel color, β in the formula (3) is diffuse reflection chromaticity, and Φ in the formula (4) is specular reflection chromaticity.
Therefore, according to the formulas (2), (3) and (4), the reversed color light can be expressed as shown in the formula (5)
Assuming that the image is generated under white light, phi r ＝φ g ＝φ b =1/3, and F r ＝F g ＝F b = F, the diffuse reflection can be represented by formula (6):
P d ＝I-F s (6)
meanwhile, we define the maximum chromaticity and the maximum diffuse reflection chromaticity as follows:
according to the conclusions made by the prior art, the diffuse reflection component can be expressed by the maximum diffuse reflection chromaticity, as shown in equation (9):
since the surface material may vary from point to point, β max In an actual image, the number of pixels varies, but is limited to 1/3 to 1.
As can be seen from equation (9), the maximum diffuse reflectance chromaticity is required to obtain a diffuse reflectance image, i.e., a specular-reflection-free image. However, estimating the maximum diffuse chromaticity of each pixel from a single image is a difficult problem. However, if it is set to a constant, a "pseudo-encoded" diffuse image can be obtained that has exactly the same geometric profile as the diffuse component of the input image. However, such an image has a large color difference from the original image, and thus a good effect cannot be obtained. We can first obtain an estimate to further remove the highlights.
The highlight detection of the image is quite important for removing the highlight of the image, and most of the current algorithms for removing the highlight of the image are also carried out on the premise of accurately detecting the highlight. Currently, there are two main methods for detecting high light. Firstly, the highlight is judged by making a difference between an original image and an MSF image, and the formula is as follows:
wherein diff means only diffuse reflection component, spec means specular reflection component and diffuse reflection component, V i (p) is the pixel value of the p point in the original image, V msf，i And (p) is the pixel value of a p point in the MSF image, and the threshold value T is the average value of all minimum channels in the original image.
The other is similar to the former, and separation is mainly achieved by comparing the smallest channels, and the formula is as follows:
wherein the threshold T = u +0.5 ∈, u is the mean of the smallest channel of the original image, and ∈ is the standard deviation of u.
The two highlight detection algorithms have similar results, but have the defect of single threshold, so the effect is poor in some image detection. Therefore, aiming at the defect, the invention provides a more efficient detection algorithm which can detect highlight more comprehensively.
When the minimum channel V of the original image min，i When the mean value u of (a) is smaller than the standard deviation epsilon, it indicates that V min,i The values in between are too scattered and the algorithm described above is prone to detect non-highlight regions simultaneously (i.e., re-detection in this context). When u is larger than epsilon, missing detection and re-inspection may occur for some high spots with lower pixel values, therefore, the present invention proposes a more reliable detection algorithm based on this feature, and the algorithm flow is described in fig. 1:
wherein the smallest channel V min ＝min{I r ,I g ,I b U is V min Has a mean value of epsilon of V min And T = u + k epsilon, k =1.35 as chosen herein, may be applied to most images. The algorithm has the greatest advantage that the high-light area is positioned by using double thresholds, so that the problems of re-detection and missing detection, which can occur in the prior art, are greatly reduced.
In the prior art, beta is max The pseudo diffuse reflection image is obtained by setting the value to be 0.5, and the maximum diffuse reflection chromaticity diagram of the image has the special characteristic of local smoothness, so the invention provides a method for estimating the maximum diffuse reflection chromaticity estimation value more accurately.
The maximum diffuse reflectance chromaticity in the prior art is estimated according to the maximum diffuse reflectance chromaticity diagram of the proposed diffuse reflectance image in a pseudo-coding form, but the diffuse reflectance chromaticity is not influenced by the geometric properties, while the color is influenced by the geometric properties and materials, so the method in the prior art has poor robustness. Therefore, a new method is provided for solving the maximum diffuse reflection chromaticity estimation value on the basis of the highlight detection algorithm, and the obtained maximum diffuse reflection chromaticity estimation value is closer to the maximum diffuse reflection chromaticity, as shown in formula (12):
wherein th is determined according to the highlight detection algorithm, and the formula is shown as formula (13):
in formula (12) I max ＝max{I r ,I g ,I b }，I min ＝min{I r ,I g ,I b U is the mean of the smallest channel, epsilon is the standard deviation of the smallest channel, T = u + k epsilon, where k =1.35, in equation (13).
As mentioned above, the non-highlight region is the diffuse component, while the highlight region is a combination of diffuse and specular reflection. Therefore, according to the MSF image principle in the prior art and the highlight detection algorithm, the compensation function th for solving the maximum diffuse reflection estimation value is set as a highlight detection threshold point, so that the robustness is good, the trouble caused by partial strong light can be removed more reliably, and the reliable support is provided for the subsequent highlight removal algorithm.
After obtaining the maximum diffuse reflection chromaticity estimated value, the invention guides the regeneration of the maximum chromaticity diagram based on the rapid joint bilateral filter and the maximum diffuse reflection chromaticity estimated value, thereby obtaining a new maximum chromaticity diagram, the result shows that the maximum chromaticity diagram is very close to the maximum diffuse reflection chromaticity diagram, and the finally obtained new maximum chromaticity diagram has the formula:
wherein F and G in equation (14) are the spatial and value domain weighting functions of the bilateral filter, respectively, and after filtering, the maximum chromaticity α due to specular highlight max Will be reduced and for specular pixels, the maximum chroma after filtering will be reduced
Than α max Closer to the maximum diffuse reflectance chromaticity P max 。
The invention uses the guide filter to filter the maximum chromaticity diagram again, and finally obtains the regenerated maximum chromaticity. The guide filter and the bilateral filter belong to the edge-preserving filter, but the guide filter has better effect on processing the image edge compared with the bilateral filter, and has no gradient deformation near the edge, so the guide filter can well remove highlight and can also save image details. Therefore, the maximum diffuse reflection chroma estimated value provided by the invention is used as the guide map
To guide the maximum chromaticity diagram alpha max The maximum chromaticity value after guidance is shown in formula (15)
Wherein beta is o max Is a graph of the output after filtering,
is a guide drawing, α max Is a graph to be filtered, W pq Are weight values determined from the guidance map. Weight value W pq Can be represented by the following formula
μ k Is the average value, I, of all pixels within the filtering window p And I q Values, σ, referring to two adjacent pixels k And representing the variance of pixel points in the filtering window, wherein epsilon is a punishment coefficient with a small value. The adaptive weight can be obtained according to the analysis of the formula: i is p And I q On both sides of the boundary, I p -μ k And I q -μ k And if not, the same sign is obtained. The weight value of the different sign is far smaller than that of the same sign, so that the pixels in the flat area are weighted with larger weight, the smoothing effect is more obvious, the pixels on two sides of the boundary are weighted with smaller weight, the smoothing effect is weaker, and the effect of keeping the boundary can be achieved.
For maximum chroma alpha affected by specular reflection after filtering with a double edge-preserving filter max Will be reduced while the filtered maximum chromaticity alpha is reduced max Closer to the true value. However, after filtering by the double edge-preserving filter, specular pixels also affect the diffuse reflection pixels. Therefore, in order to reduce the influence of the mirror surface pixels on the diffuse reflection pixels, the new maximum chroma value and the original maximum chroma value which are subjected to the double edge-preserving filter are compared, the maximum chroma value of the new maximum chroma value and the original maximum chroma value is taken as the formula (17), so that compared with the comparison result of a single double edge filter, the influence of the mirror surface pixels can be greatly reduced after the new maximum chroma guided by the guide filter is added, the obtained maximum diffuse reflection chroma is more accurate and has better robustness,
after experiments, the picture generated after the guiding filter is added is found to have better highlight removing effect than that of the picture generated by using a single bilateral filter, the color information of the original picture is better restored due to the effect of the guiding filter, the texture characteristics of the picture are improved, and meanwhile, the obtained highlight-free picture is more natural.
In order to verify the effectiveness of the invention more accurately, peak signal-to-noise ratio (PSNR) and Structural Similarity (SSIM) of an image quality detection standard are used for quantization comparison, and the higher the Peak signal-to-noise ratio and the structural similarity, the closer the Peak signal-to-noise ratio and the structural similarity are to a real original image. Wherein the peak signal-to-noise ratio above 30dB shows no big difference from the original. The peak signal-to-noise ratio and structural similarity of the three algorithms are compared with the results given the real results, as shown in table 1. It can be seen from the table that the present invention is superior to the other two algorithms in both peak signal-to-noise ratio and structural similarity.
TABLE 1 quantitative comparison of experimental data
Table 1 Quantitative Comparison Of Experimental Data
Therefore, through experimental comparison and quantitative comparison of visual perception data, the effect graph obtained by the method provided by the invention is more consistent with an actual result graph, is clearer and more natural, and better retains the detail part of the image, so that the method has more advantages compared with the prior art.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (4)
1. A highlight removing method of a double-edge-preserving filter is characterized by comprising the following steps: comprises the following steps
S1, establishing a two-color reflection model;
the color of an image observed at a pixel x is expressed by the following equation, in which the reflection component of an arbitrary light ray is divided into two components of diffuse reflection and specular reflection
I(x)＝w d (x)P d (x)+w s (x)F s (x)
Wherein, I = [ I = [ ] r ,I g ,I b ],P d (x) And F s (x) Respectively a diffuse reflection component and a specular reflection component at pixel x, w d And w s Diffuse and specular reflectance, respectively, of pixel x, which depends on the scene geometry at the image pixel;
definition of the chromaticity alpha as
The diffuse reflectance chroma beta is defined as
Defining the specular reflection chromaticity phi as
S2, highlight detection;
s3, highlight removal algorithm;
the maximum diffuse reflectance chromaticity was calculated using the following formula
Where th is determined in step S2, I max ＝max{I r ,I g ,I b }，I min ＝min{I r ,I g ,I b }
The maximum chromaticity diagram is as follows
2. The highlight removal method of the double edge-preserving filter according to claim 1, wherein: the light with reverse color obtained according to the formula step S1 is as follows
If the image is generated under white light, then phi r ＝φ g ＝φ b ＝1/3，F r ＝F g ＝F b = F, diffuse reflection of the formula
P d ＝I-F s
Defining the maximum chromaticity and the maximum diffuse reflection chromaticity as
3. The highlight removal method of the double edge-preserving filter of claim 2, characterized in that:
where u is the mean of the smallest channel, epsilon is the standard deviation of the smallest channel, T = u + k epsilon, where k =1.35.
4. The highlight removal method of the double edge-preserving filter according to claim 2, wherein:
using maximum diffuse reflectance chromaticity estimate as a guide map
To guide the maximum chromaticity diagram alpha max The maximum chromaticity value after guidance is as follows
Wherein beta is O max Is a graph of the output after filtering,
is a guide drawing, α max Is the graph to be filtered, W pq Is a weight value determined according to the guide map, the weight value W pq Can be represented by the following formula
μ k Is the average value, I, of all pixels within the filtering window p And I q Values, σ, referring to two adjacent pixels k And representing the variance of the pixel points in the filtering window, wherein epsilon is a punishment coefficient with a small value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010426992.XA CN111563859B (en) | 2020-05-19 | 2020-05-19 | Highlight removing method of double-edge-preserving filter |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010426992.XA CN111563859B (en) | 2020-05-19 | 2020-05-19 | Highlight removing method of double-edge-preserving filter |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111563859A CN111563859A (en) | 2020-08-21 |
| CN111563859B true CN111563859B (en) | 2023-03-31 |
Family
ID=72071039
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010426992.XA Active CN111563859B (en) | 2020-05-19 | 2020-05-19 | Highlight removing method of double-edge-preserving filter |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111563859B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113096057B (en) * | 2021-04-13 | 2022-08-19 | 南京大学 | Highlight elimination method based on four-light-source photometric stereo method |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2013065215A (en) * | 2011-09-16 | 2013-04-11 | Toshiba Corp | Image processing device and image processing method |
| CN109410228A (en) * | 2018-08-22 | 2019-03-01 | 南京理工大学 | Internal wave of ocean detection algorithm based on Method Based on Multi-Scale Mathematical Morphology Fusion Features |
| CN111080686B (en) * | 2019-12-16 | 2022-09-02 | 中国科学技术大学 | Method for highlight removal of image in natural scene |
-
2020
- 2020-05-19 CN CN202010426992.XA patent/CN111563859B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111563859A (en) | 2020-08-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP4112362B2 (en) | System and method for enhancing scanned document images for color printing | |
| US7894685B2 (en) | Method and apparatus for reducing ringing artifacts | |
| JP5270573B2 (en) | Method and apparatus for detecting block artifacts | |
| IES20060564A2 (en) | Improved foreground / background separation | |
| US20040057623A1 (en) | Method for automated processing of digital image data | |
| CN111080686A (en) | Method for highlight removal of image in natural scene | |
| CN115511907B (en) | Scratch detection method for LED screen | |
| CN111563859B (en) | Highlight removing method of double-edge-preserving filter | |
| Xin et al. | Specular reflection image enhancement based on a dark channel prior | |
| EP2070048B1 (en) | Spatial masking using a spatial activity metric | |
| CN115660990A (en) | Endoscope image mirror reflection detection and restoration method based on brightness classification | |
| CN115829967A (en) | Industrial metal surface defect image denoising and enhancing method | |
| Shan et al. | Image highlight removal based on double edge-preserving filter | |
| RU2338252C1 (en) | Method that prevents printing of out of focus pictures | |
| JP4100860B2 (en) | Image processing apparatus, image processing method, and recording medium | |
| Deperlioglu et al. | Practical method for the underwater image enhancement with adjusted CLAHE | |
| Thayammal et al. | Performance analysis of image denoising using deep convolutional neural network | |
| Sun et al. | Adaptive bilateral filter considering local characteristics | |
| Sari et al. | Development of denoising method for digital image in low-light condition | |
| Bruni et al. | Visibility based detection and removal of semi-transparent blotches on archived documents | |
| CN111833271A (en) | Rapid and effective mirror highlight image enhancement method and device and storage medium | |
| US6983073B1 (en) | Enhancement of compressed images | |
| Kisilev et al. | Noise and signal activity maps for better imaging algorithms | |
| Kulkarni et al. | Comparative study of major image enhancement algorithms | |
| Bressan et al. | Local contrast enhancement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |