CN112379529B - Transparent object surface reflected light separation method based on polarization characteristics - Google Patents

Transparent object surface reflected light separation method based on polarization characteristics Download PDF

Info

Publication number
CN112379529B
CN112379529B CN202011303630.8A CN202011303630A CN112379529B CN 112379529 B CN112379529 B CN 112379529B CN 202011303630 A CN202011303630 A CN 202011303630A CN 112379529 B CN112379529 B CN 112379529B
Authority
CN
China
Prior art keywords
image
light
reflected light
polarization
correlation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202011303630.8A
Other languages
Chinese (zh)
Other versions
CN112379529A (en
Inventor
张焱
张景华
石志广
李飚
杨卫平
张宇
索玉昌
刘荻
凌峰
师晓冉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202011303630.8A priority Critical patent/CN112379529B/en
Publication of CN112379529A publication Critical patent/CN112379529A/en
Application granted granted Critical
Publication of CN112379529B publication Critical patent/CN112379529B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/28—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for polarising
    • G02B27/283—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 for polarising used for beam splitting or combining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Algebra (AREA)
  • Pure & Applied Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Optics & Photonics (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

The invention belongs to the field of image processing, and relates to a method for separating reflected light on the surface of a transparent object based on polarization characteristics. The method comprises the following steps: (S1) collecting a polarization image and performing a preprocessing; (S2) carrying out polarization calculation on the preprocessed image to obtain light intensity images in the parallel direction and the vertical direction; (S3) separating the reflected images using the parallel-direction and vertical-direction light intensity images; (S4) obtaining a normalized cross-correlation of the separated reflected light image and the transmitted light image; (S5) extracting and summing the reflection dominant pixel and the transmission dominant pixel with the cross-correlation; (S6) solving the minimum value of the normalized cross correlation by using a gradient descent method to obtain the corresponding polarization degree of the transmitted light and the polarization degree of the reflected light, thereby realizing the separation of the reflected light. The method can realize the separation of the reflected light and the transmitted light on the surface of the transparent object, and has good separation effect. By separating the reflected light, the reflection information can be effectively extracted, and clutter suppression and enhancement are performed on the transmission image.

Description

Transparent object surface reflected light separation method based on polarization characteristics
Technical Field
The invention belongs to the field of optical image processing, relates to processing and analysis of visible light polarization image information, and realizes a separation method of surface reflected light of a transparent object.
Background
When the detector is used for imaging the scenery behind transparent materials such as glass, the obtained image consists of two parts: a transmitted light portion and a reflected light portion. Since the reflection and transmission processes of the surface of the transparent object are simultaneous, the reflected light and the transmitted light are often mixed together and affect each other. With the adoption of a large amount of materials with light transmitting and reflecting effects, such as glass and plastic, in the process of modern city construction and home interior decoration, the problems of separation of transmitted light and reflected light on the surface of a transparent object and inhibition of reflected light cause extensive attention and key research of domestic and foreign scholars. On the one hand, the reflected light can present the virtual image on the object surface, covers attributes such as color detail of scenery behind the glass, and when the incident light source was stronger, still can form the highlight region on the object surface, influences the quality of image, brings great problem for computer processing and human eye identification. On the other hand, the information such as the intensity, the position, the surrounding environment and the like of the reflecting light source can be obtained by extracting and analyzing the reflecting light, and the method has certain application value. Therefore, the separation of the reflected light and the transmitted light on the surface of the transparent object is a very challenging task in the field of computer vision, and has important significance for the application of image segmentation, target recognition, stereo matching and the like.
The existing reflected light separation methods can be roughly classified into two types: the method is based on image characteristics, such as a reflected scene edge blurring characteristic caused by a camera virtual focus, a ghost characteristic generated by simultaneous reflection of the front surface and the rear surface of the glass, and the like. Since the reflected light image and the transmitted light image interfere with each other and overlap each other, the effect of the reflected light separation based on the image characteristics alone is not ideal; the second is based on physical properties, such as polarization characteristics generated in the light wave reflection process. Because the reflected light and the transmitted light on the glass surface have obvious polarization effects and the polarization characteristics of the reflected light and the transmitted light have obvious differences, the separation of the reflected light can be realized by utilizing the polarization characteristics of the reflected light, and the separation method has better separation effect and stability compared with a simple reflected light separation method based on image characteristics.
Disclosure of Invention
Aiming at the problem of separation of reflected light on the surface of a transparent object, the polarization characteristics of a reflection area are analyzed, the correlation between a transmitted light image and a reflected light image is combined, the minimum value of normalized cross correlation between the transmitted light image and the reflected light image is solved by a gradient descent method, and the corresponding reflection polarization degree and transmission polarization degree when the correlation is minimum are obtained. And then according to the distribution relation of the transmitted light and the reflected light in the vertical direction and the parallel direction of the surface of the transparent object, the polarization orthogonal difference algorithm is used for realizing the separation of the reflected light by utilizing the polarization degree of the reflected light and the polarization degree of the transmitted light. The specific technical scheme is as follows:
a method for separating reflected light on the surface of a transparent object based on polarization characteristics comprises the following steps:
(S1) acquiring a visible light polarization image and performing preprocessing;
(S2) carrying out polarization state calculation on the image preprocessed in the step (S1) to obtain a light intensity image in the parallel direction and a light intensity image in the vertical direction;
(S3) selecting the initial polarization degree of the reflected light and the initial polarization degree of the transmitted light, and combining the light intensity image in the parallel direction and the light intensity image in the vertical direction to separate the mixed image of the reflected light and the transmitted light;
(S4) obtaining a normalized cross-correlation of the initial separated reflected light image and the transmitted light image;
(S5) extracting and summing the reflection dominant pixel and the transmission dominant pixel with the cross-correlation;
(S6) solving the minimum value of the normalized cross correlation by using a gradient descent method to obtain the polarization degree of the transmitted light and the polarization degree of the reflected light corresponding to the minimum value, thereby realizing the separation of the reflected light.
Preferably, the preprocessing in the step (S1) includes: including image correction, filtering, registration, and cropping.
Preferably, the parallel direction light intensity image and the vertical direction light intensity image in the step (S2) are calculated by:
according to the representation form of the polarized light, the light intensity calculation formula of the polarized light under different polarizing angles is determined as follows:
Figure GDA0003531052670000031
wherein i and j represent the coordinate position of a pixel point in the image, phimIs the angle of incidence, phi⊥Is the corresponding deflection angle of the vertical direction of the reflecting surface.
Let phi0Angle of departure phi is 0 DEGmAre respectively phi0＝φ0，φ45＝φ0+45°，φ90＝φ0Three-channel polarization degree image I acquired at +90 DEG0，I45，I90Substituting the following formulas into the formulas respectively to obtain:
Figure GDA0003531052670000032
respectively determining the light intensity I in the vertical direction⊥And light intensity in parallel direction I||The following were used:
Figure GDA0003531052670000033
Figure GDA0003531052670000034
preferably, the specific process of the step (S3) is as follows:
according to the polarization orthogonal decomposition principle, the components of the light intensity received by the detector in the vertical direction and the parallel direction are determined as follows:
Figure GDA0003531052670000035
wherein R is⊥And R||Respectively vertical and parallel reflectivity, PRIs the intensity of the light source of the reflected light,
Figure GDA0003531052670000036
and
Figure GDA0003531052670000037
light intensity components, epsilon, in the vertical and parallel directions of the reflected light, respectively⊥And ε||Vertical and parallel direction emissivity respectively,
Figure GDA0003531052670000038
a light intensity component in a vertical direction for the transmitted light,
Figure GDA0003531052670000039
for light intensity components parallel to the direction of transmitted light, PTIs the transmitted light source intensity.
The reflected light and the transmitted light on the surface of the glass belong to polarized light, the polarization degree of the reflected light is gamma, and the polarization degree of the transmitted light is chi:
Figure GDA0003531052670000041
Figure GDA0003531052670000042
then:
Figure GDA0003531052670000043
Figure GDA0003531052670000044
substituting the formulas (8) and (9) into the formula (5) to solve the problem that the light intensity components of the transmitted light in the vertical direction and the parallel direction are:
Figure GDA0003531052670000045
and simultaneously determining the light intensity components of the reflected light in the vertical direction and the parallel direction as follows:
Figure GDA0003531052670000046
the total light intensity is equal to the sum of the light intensities in the vertical direction and the parallel direction, and therefore the light components reflected from the glass surface and the light components transmitted therefrom are determined as follows:
Figure GDA0003531052670000047
preferably, the specific process of the step (S5) is as follows:
normalizing autocorrelation at any pixel point r (i, j) in the image
Figure GDA0003531052670000051
Expressed as:
Figure GDA0003531052670000052
wherein U, V represents the window size, u and v represent the position coordinates of the pixel points in the window,
Figure GDA0003531052670000053
and
Figure GDA0003531052670000054
representing the average pixel gray scale values within the window of the reflected light image and the transmitted light image, respectively.
(S51) reflection dominant pixel point extraction
The specific steps of extracting the reflected light as the dominant pixel point are as follows: first, the mixed image is set to be 0.01 and 0.2, and the mixed image is subjected to separation to obtain a transmitted light image I after separationover-t(ii) a Then selecting gamma 0.99 and chi 0.2 to under-separate the mixed image and obtain the under-separated transmitted light image Iunder-t(ii) a Then find Iover-tAnd Iunder-tThe correlation between the two is obtained to obtain a correlation image Rt(ii) a And finally, setting the pixel point with negative correlation as 1 and the pixel point with positive correlation as 0, thereby realizing reflection leading pixel pointExtracting;
Figure GDA0003531052670000055
wherein R istRepresenting the transmitted light image I after separationover-tAnd an under-separated transmitted light image Iunder-tNormalized cross-correlation image between, MtRepresents a pair of RtThe result after binarization is performed.
(S52) transmission dominant pixel point extraction:
the actual transmission polarization degree value is obtained by selecting pixel points with transmitted light as the main factor, firstly, gamma is set to be 0.5, and chi is set to be 0.01, the mixed image is subjected to separation, and a reflected light image I after separation is obtainedover-rThen, the mixed image is under-separated by selecting gamma 0.99 and chi 0.5, and the under-separated transmitted light image I is obtainedunder-rObtaining R by cross-correlation between the tworAnd binarizing the image:
Figure GDA0003531052670000056
wherein R isrRepresenting the separated reflected light image Iover-rAnd an under-separated reflected light image Iunder-rNormalized cross-correlation image between, MrRepresents a pair of RrThe result after binarization is performed.
(S53) cross-correlation summing of reflection-dominated pixels and transmission-dominated pixels:
reflection dominated normalized cross correlation sum f of pixelsR(γ, χ) is:
Figure GDA0003531052670000061
where row, col represent the number of rows and columns of the image, respectively.
Transmission dominated normalized cross correlation sum f of pixelsT(γ, χ) is:
Figure GDA0003531052670000062
preferably, the specific process of the step (S6) is as follows:
normalized cross-correlation f of reflected and transmitted light imagesR(gamma, chi) and fT(gamma, chi) as a function of the degree of reflection polarization gamma and of the degree of transmission polarization chi, and then respectively for fR(gamma, chi) and fT(gamma, chi) calculating partial derivative, making it fall along gradient direction, obtaining f after several iterationsR(gamma, chi) and fT(γ, χ);
Figure GDA0003531052670000063
where η is the learning rate, n represents the current iteration number, and n +1 represents the next iteration number. When the convergence condition is reached, the iteration is ended, and the normalized cross-correlation minimum value f is obtainedR(γm,χm) And fT(γm,χm) And their corresponding transmission polarization degree gammamAnd degree of reflection polarization χmThereby achieving an optimal separation of reflected and transmitted light.
Preferably, in the process of extracting the reflection leading pixel point, the reflection polarization degree γ is set to 0.01 and 0.99 in sequence.
Preferably, in the process of extracting the transmission leading pixel point, the transmission polarization degree χ is set to be 0.01 and 0.99 in sequence.
In order to fully understand the present invention, the following description is made of the related principles involved in the technical solutions.
When the detector is used for imaging the scenery behind transparent materials such as glass, the obtained image consists of two parts: a transmitted light portion and a reflected light portion. Since the reflection and transmission processes of the surface of the transparent object are simultaneous, the reflected light and the transmitted light are often mixed together and affect each other. Because the reflected light and the transmitted light on the surface of the transparent object are both polarized light and have obvious difference, wherein the polarization direction of the reflected light is mainly perpendicular to the reflecting surface, and the polarization direction of the transmitted light is mainly parallel to the reflecting surface, the separation of the reflected light and the transmitted light can be realized by utilizing the polarization characteristics of the reflected light and the transmitted light.
In order to realize the separation of the reflected light, the invention establishes an equation relation between the reflection light intensity and the transmission light intensity and the reflection polarization degree and the transmission polarization degree by utilizing the distribution relation of the reflection light and the transmission light on the surface of the transparent object in the vertical direction and the parallel direction, thereby realizing the separation of the reflected light directly through the polarization degree. Because the reflection polarization degree and the transmission polarization degree can not be directly measured by a detector, the method solves the normalized cross-correlation minimum value of the reflected light image and the transmitted light image by a gradient descent method, thereby obtaining the corresponding reflection light polarization degree and the transmission light polarization degree, and finally realizing the optimal separation of the transmitted light and the reflected light.
The extraction method of the reflection dominant point and the transmission dominant point in the mixed image comprises the following steps: in the actual decomposition process, the sum of the cross-correlations of all the pixel points in the image is directly solved, and two problems can exist: firstly, the calculation amount is large, and the calculation speed is slow; and secondly, the device is easily interfered by noise. In order to suppress noise interference and improve the calculation speed, the invention provides an algorithm for extracting a dominant pixel point. The dominant pixel is a pixel in which transmitted light information or reflected light information is dominant in the mixed image. Because the polarization degree is related to the ratio of the transmitted light to the reflected light, the larger the difference between the two is, the larger the polarization degree is, and the smaller the difference is, the smaller the polarization degree is. For the reflection leading pixel point and the transmission leading pixel point in the image, the difference between the transmission light intensity and the reflection light intensity is large, so that the pixel points are large in polarization degree and strong in anti-interference performance.
For the pixel point which takes the reflected light as the leading factor, in the process that the reflection polarization degree changes from small to large, the reflected light separation process can be changed from over decomposition to under decomposition, at the moment, the correlation between the transmitted light image and the reflected light image can be changed from negative correlation to positive correlation, namely, the correlation can be changed in a positive and negative way. For the pixel point which takes the transmitted light as the leading factor, when the reflection polarization degree changes from small to large, the relevant change is not obvious, and the phenomenon of positive and negative change does not exist. Therefore, pixel points which take reflected light as a main factor can be extracted through the correlation positive and negative change relation of the transmitted light images in the under-separation and over-separation processes of the reflected light; and then, the actual reflection polarization degree can be accurately and efficiently solved by using the pixel points taking the reflected light as the leading factor.
Similarly, for a pixel point which takes the transmitted light as the leading factor, in the process of changing the transmission polarization degree from small to large, the transmitted light separation process is changed from over decomposition to under decomposition, and at the moment, the correlation between the transmitted light image and the reflected light image is changed from negative correlation to positive correlation, namely, the correlation is changed in a positive and negative way. For the pixel point which takes the reflected light as the leading factor, when the transmission polarization degree changes from small to large, the relevant change is not obvious, and the phenomenon of positive and negative change does not exist. Therefore, pixel points which take the transmitted light as the leading factor can be extracted through the correlation positive and negative change relation of the reflected light images in the under-separation and over-separation processes of the transmitted light images; and then, the actual transmission polarization degree can be accurately and efficiently solved by using the pixel points taking the transmission light as the leading factor.
The beneficial effects obtained by adopting the invention are as follows:
the method can effectively separate the reflected light and the transmitted light on the surface of the transparent object, and based on the polarization characteristics of the reflected light and the transmitted light, the characteristics of the interaction of the reflected light and the transmitted light are utilized, the light intensity distribution relation of the reflected light and the transmitted light in the vertical direction and the parallel direction is combined, the method of polarization orthogonal differential decomposition is adopted, and the reflected light image and the transmitted light image are obtained through the solution of the reflection polarization degree and the transmission polarization degree. And then based on the principle that the correlation between the reflected light image and the transmitted light image is minimum during optimal separation, solving the reflection polarization degree and the transmission polarization degree corresponding to the minimum normalized cross-correlation value of the reflected light image and the transmitted light image by using a gradient descent method, and finally realizing the optimal separation of the reflected light and the transmitted light on the surface of the transparent object. Experimental results show that the method can realize the separation of reflected light under different scenes. Through the separation of the reflected light, on one hand, the information such as the position of the reflected light source on the surface of the object and the intensity of the reflected light source can be obtained from the reflected light, and the information is used as a data base for target reconnaissance, three-dimensional reconstruction and the like, on the other hand, the reflected light on the surface of the transparent object is restrained through the separation of the reflected light, so that the transmission component is enhanced, the imaging quality of scenery behind the glass is improved, and the image processing such as target detection, segmentation and the like under the complex reflected light environment is more favorably realized.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a flow chart of a reflected light separation process;
FIG. 3 is a schematic diagram of the reflection and transmission processes;
FIG. 4 is a process of reflected light over-decomposition and under-decomposition;
FIG. 5 is a process of transmitted light over-decomposition and under-decomposition;
FIG. 6 is a schematic view of a reflection data acquisition;
FIG. 7 is a comparison graph of the separation results of reflected light under different scenes.
Detailed Description
The invention is further illustrated by the following figures and examples.
As shown in fig. 1, which is a general flow chart of the present invention; in order to test the separation effect and reliability of the method, three groups of reflected light images in the real world are obtained by utilizing devices such as a polarization detector and glass. The data acquisition process is shown in fig. 4. First, a mixed image of reflected light and transmitted light in the real world is acquired using a glass and polarization detector (as shown in fig. 7(a1), (b1), (c 1)). The glass is then removed and a background image, i.e. a true transmitted light image, is acquired (as shown in fig. 7(a2), (b2), (c 2)). Fig. 7(a3), (b3), (c3) are transmitted light images separated by the method of the present invention, and fig. 5(a4), (b4), (c4) are reflected light images separated by the method of the present invention. By comparing the transmitted light image separated by the method of the invention (as shown in fig. 7(a3), (b3) and (c 3)) with the real transmitted light background image (as shown in fig. 7(a2), (b2) and (c2)), it can be seen that the transmitted light image separated by the method of the invention is nearly the same as the real background image, and the effectiveness of the method of the invention is verified, and the method of the invention specifically comprises the following steps:
(S1) acquiring a visible light polarization image and performing preprocessing;
acquisition of four channels (different directions of analysis, e.g. I) of a target scene by means of a visible light polarization imaging detection system0、I45、I90、I135) Polarizes the image. Image preprocessing includes image correction, filtering, registration, cropping, etc., the correction serving to offset non-uniformities in the detector response and compensate for non-idealities in the optical components. Filtering is used to reduce image noise and remove dead spots. And the registration is used for eliminating micro-displacement deviation of each polarization image in the same scene, and finally cutting off a frame after the image registration.
(S2) carrying out polarization state calculation on the image preprocessed in the step (S1) to obtain a light intensity image in the parallel direction and a light intensity image in the vertical direction;
according to the representation form of the polarized light, the light intensity calculation formula of the polarized light under different polarizing angles is determined as follows:
Figure GDA0003531052670000101
wherein i and j represent the coordinate position of a pixel point in the image, phimIs the angle of incidence, phi⊥Is the corresponding deflection angle of the vertical direction of the reflecting surface.
Let phi0Angle of departure phi is 0 DEGmAre respectively phi0＝φ0，φ45＝φ0+45°，φ90＝φ0Three-channel polarization degree image I acquired at +90 DEG0，I45，I90Substituting the following formulas into the formulas respectively to obtain:
Figure GDA0003531052670000102
respectively determining the light intensity I in the vertical direction⊥And light intensity in parallel direction I||The following were used:
Figure GDA0003531052670000103
Figure GDA0003531052670000111
(S3) selecting the initial polarization degree of the reflected light and the initial polarization degree of the transmitted light, and combining the light intensity image in the parallel direction and the light intensity image in the vertical direction to separate the mixed image of the reflected light and the transmitted light;
according to the polarization orthogonal decomposition principle, the components of the light intensity received by the detector in the vertical direction and the parallel direction are determined as follows:
Figure GDA0003531052670000112
wherein R is⊥And R||Respectively vertical and parallel reflectivity, PRIs the intensity of the light source of the reflected light,
Figure GDA0003531052670000113
and
Figure GDA0003531052670000114
light intensity components, epsilon, in the vertical and parallel directions of the reflected light, respectively⊥And ε||Vertical and parallel direction emissivity respectively,
Figure GDA0003531052670000115
a light intensity component in a vertical direction for the transmitted light,
Figure GDA0003531052670000116
for light intensity components parallel to the direction of transmitted light, PTIs the transmitted light source intensity.
The reflected light and the transmitted light on the surface of the glass belong to polarized light, the polarization degree of the reflected light is gamma, and the polarization degree of the transmitted light is chi:
Figure GDA0003531052670000117
Figure GDA0003531052670000118
then:
Figure GDA0003531052670000119
Figure GDA00035310526700001110
substituting the formulas (8) and (9) into the formula (5) to solve the problem that the light intensity components of the transmitted light in the vertical direction and the parallel direction are:
Figure GDA0003531052670000121
and simultaneously determining the light intensity components of the reflected light in the vertical direction and the parallel direction as follows:
Figure GDA0003531052670000122
the total light intensity is equal to the sum of the light intensities in the vertical direction and the parallel direction, and therefore the light components reflected from the glass surface and the light components transmitted therefrom are determined as follows:
Figure GDA0003531052670000123
(S4) obtaining a normalized cross-correlation of the initial separated reflected light image and the transmitted light image;
on the premise of knowing the polarization degree of the reflected light, the polarization degree of the transmitted light and the light intensity in the vertical direction and the parallel direction, the reflected light intensity and the transmitted light intensity at each pixel point on the glass surface can be obtained by using the formula (12), so that the reflected light is separated. However, the reflected light and the transmitted light on the glass surface exist simultaneously and are superposed with each other, and the detector cannot be directly used for acquiring the polarization degree of the reflected light and the polarization degree of the transmitted light respectively.
Since the reflected and transmitted light images on the glass surface contain different image content, the resulting reflected and transmitted light images have minimal correlation under ideal separation conditions. Therefore, although the degree of polarization of the reflected light and the degree of polarization of the transmitted light on the glass surface cannot be directly measured, the degree of polarization of the reflected light and the degree of polarization of the transmitted light corresponding to the minimum degree of correlation between the reflected light image and the transmitted light image can be obtained by calculating the correlation between the reflected light image and the transmitted light image, and the reflected light and the transmitted light can be separated. The correlation of two images can be represented by normalized cross-correlation ncc (normalized cross correlation). Normalized cross-correlation at any pixel r (i, j) in image
Figure GDA0003531052670000124
Comprises the following steps:
Figure GDA0003531052670000125
wherein U, V represents the window size, u and v represent the position coordinates of the pixel points in the window,
Figure GDA0003531052670000131
and
Figure GDA0003531052670000132
representing the average pixel gray scale values within the window of the reflected light image and the transmitted light image, respectively.
Defining f (gamma, chi) as the sum of normalized cross correlation of all pixel points in the separated reflected light image and the transmitted light image:
Figure GDA0003531052670000133
(S5) extracting and summing the reflection dominant pixel and the transmission dominant pixel with the cross-correlation;
in the actual decomposition process, the sum of the cross-correlations of all the pixel points in the image is directly solved, and two problems can exist: firstly, the calculation amount is large, and the calculation speed is slow; and secondly, the device is easily interfered by noise. In order to suppress noise interference and improve the calculation speed, the invention provides a method for extracting a leading pixel point. The dominant pixel is a pixel in which transmitted light information or reflected light information is dominant in the mixed image.
(S51) reflection dominant pixel point extraction
The specific steps of extracting the reflected light as the dominant pixel point are as follows: first, the mixed image is set to be 0.01 and 0.2, and the mixed image is subjected to separation to obtain a transmitted light image I after separationover-t(ii) a Then selecting gamma 0.99 and chi 0.2 to under-separate the mixed image and obtain the under-separated transmitted light image Iunder-t(note: when the reflection polarization degree γ is small, the phenomenon of excessive separation of the reflected light occurs in the separation process, as shown in fig. 4(a), so the present invention sets the reflection polarization degree to 0.01 to perform excessive separation of the reflected light, when the reflection polarization degree γ is large, the phenomenon of insufficient separation of the reflected light occurs in the separation process, as shown in fig. 4(c), so the present invention sets the reflection polarization degree to 0.99 to perform insufficient separation of the reflected light, for the transmission polarization degree chi, the variation range of the transmission polarization degree of most transparent objects is 0-0.4, so the median value 0.2 is selected as the value of the transmission polarization degree chi to perform under separation and excessive separation solution, as can be seen from formula 12, the transmission polarization degree chi only affects the intensity of the transmitted light image after separation, and does not change the texture and details of the transmitted light image after separation, no matter whether the transmission polarization degree is 0.1 or 0.4, the correlation between the under separation and the light image obtained through excessive separation is not greatly affected, it is therefore reasonable to set the transmitted light polarization χ to 0.2); then find Iover-tAnd Iunder-tThe correlation between the two is obtained to obtain a correlation image Rt(ii) a Finally, the pixel point with negative correlation is set to be 1, and the pixel point with positive correlation is set to be 0, so that the reflection leading pixel point is extracted;
Figure GDA0003531052670000141
wherein R istRepresenting the transmitted light image I after separationover-tAnd an under-separated transmitted light image Iunder-tNormalized cross-correlation between, MtRepresents a pair of RtThe result after binarization is performed.
(S52) transmission dominant pixel point extraction:
and calculating the actual transmission polarization degree value by selecting the pixel points taking the transmission light as the leading factor. First, the mixed image is set to be 0.5 and 0.01, and the mixed image is separated to obtain a reflected light image I after separationover-rThen, the mixed image is under-separated by selecting gamma 0.99 and chi 0.5, and the under-separated transmitted light image I is obtainedunder-r(Note that when the transmission polarization degree chi is small, the transmitted light over-separation phenomenon appears in the separation process, as shown in figure 5(a), and when the transmission polarization degree chi is large, the transmitted light under-separation phenomenon appears in the separation process, as shown in figure 5(c), therefore, the invention sets the transmission polarization degree to be 0.01 and 0.99 respectively for the transmitted light over-separation and under-separation treatment, and for the reflection polarization degree gamma, the change range of the reflection polarization degree is 0-1, therefore, the median value of 0.5 is selected as the value of the reflection polarization degree gamma), and the cross correlation between the two is obtained to obtain RrAnd binarizing the image:
Figure GDA0003531052670000142
wherein R isrRepresenting the separated reflected light image Iover-rAnd an under-separated reflected light image Iunde-rNormalized cross-correlation image between, MrRepresents a pair of RrThe result after binarization is performed.
(S53) cross-correlation summing of reflection-dominated pixels and transmission-dominated pixels:
Figure GDA0003531052670000151
where row, col represent the number of rows and columns of the image, respectively.
Transmission dominated normalized cross correlation sum f of pixelsT(γ, χ) is:
Figure GDA0003531052670000152
(S6) solving the minimum value of the normalized cross correlation by using a gradient descent method to obtain the polarization degree of the transmitted light and the polarization degree of the reflected light corresponding to the minimum value, thereby realizing the separation of the reflected light.
Normalized cross-correlation f of reflected and transmitted light imagesR(gamma, chi) and fT(γ, χ) as a function of the degree of reflection polarization γ and the degree of transmission polarization χ. Then respectively aim at fR(gamma, chi) and fT(gamma, chi) calculating partial derivative, making it fall along gradient direction, obtaining f after several iterationsR(gamma, chi) and fT(γ, χ);
Figure GDA0003531052670000153
where η is the learning rate, n represents the current iteration number, and n +1 represents the next iteration number. When the convergence condition is reached, the iteration is ended, and the normalized cross-correlation minimum value f is obtainedR(γm,χm) And fT(γm,χm) And their corresponding transmission polarization degree gammamAnd degree of reflection polarization χmThereby achieving an optimal separation of reflected and transmitted light.
As shown in fig. 7(a3), (b3), and (c3) which show the transmitted light image result after separation by the method of the present invention, by comparing with the real background image, it can be seen that there is only a slight difference between the two, i.e. the reflected light is separated and suppressed, and the effectiveness of the method of the present invention is verified.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (9)

1. A method for separating reflected light on the surface of a transparent object based on polarization characteristics is characterized by comprising the following steps:
(S1) acquiring a visible light polarization image and performing preprocessing;
(S2) carrying out polarization state calculation on the image preprocessed in the step (S1) to obtain a light intensity image in the parallel direction and a light intensity image in the vertical direction;
(S3) selecting the initial polarization degree of the reflected light and the initial polarization degree of the transmitted light, and combining the light intensity image in the parallel direction and the light intensity image in the vertical direction to separate the mixed image of the reflected light and the transmitted light;
(S4) obtaining a normalized cross-correlation of the initial separated reflected light image and the transmitted light image;
(S5) extracting and summing the reflection dominant pixel and the transmission dominant pixel with the cross-correlation;
(S6) solving the minimum value of the normalized cross correlation by using a gradient descent method to obtain the polarization degree of the transmitted light and the polarization degree of the reflected light corresponding to the minimum value, thereby realizing the separation of the reflected light.
2. The method for separating the reflected light from the surface of the transparent object based on the polarization characteristics as claimed in claim 1, wherein the preprocessing in the step (S1) comprises: including image correction, filtering, registration, and cropping.
3. The method for separating the reflected light from the surface of the transparent object based on the polarization characteristics as claimed in claim 1, wherein the solving process of the parallel direction light intensity image and the perpendicular direction light intensity image in the step (S2) is as follows:
according to the representation form of the polarized light, the light intensity calculation formula of the polarized light under different polarizing angles is as follows:
Figure FDA0003531052660000011
wherein i and j represent the coordinate position of a pixel point in the image, phimIs the angle of incidence, phi⊥Is the corresponding deflection angle of the vertical direction of the reflecting surface.
Let phi0Angle of departure phi is 0 DEGmAre respectively phi0＝φ0，φ45＝φ0+45°，φ90＝φ0Three-channel polarization degree image I acquired at +90 DEG0，I45，I90Substituting the following formulas into the formulas respectively to obtain:
Figure FDA0003531052660000021
respectively determining the light intensity I in the vertical direction⊥And light intensity in parallel direction I||The following were used:
Figure FDA0003531052660000022
Figure FDA0003531052660000023
4. the method for separating the reflected light from the surface of the transparent object based on the polarization characteristics as claimed in claim 1, wherein the step (S3) of solving the reflected light image and the transmitted light image by combining the polarization degree of the reflected light and the polarization degree of the transmitted light comprises the following steps:
according to the polarization orthogonal decomposition principle, the components of the light intensity received by the detector in the vertical direction and the parallel direction are determined as follows:
Figure FDA0003531052660000024
wherein R is⊥And R||Are respectively in the vertical direction and the horizontal directionLine direction reflectivity, PRIs the intensity of the light source of the reflected light,
Figure FDA0003531052660000025
and
Figure FDA0003531052660000026
light intensity components, epsilon, in the vertical and parallel directions of the reflected light, respectively⊥And ε||Vertical and parallel direction emissivity respectively,
Figure FDA0003531052660000027
a light intensity component in a vertical direction for the transmitted light,
Figure FDA0003531052660000028
for light intensity components parallel to the direction of transmitted light, PTIs the transmitted light source intensity.
The reflected light and the transmitted light on the surface of the glass belong to polarized light, the polarization degree of the reflected light is gamma, and the polarization degree of the transmitted light is chi:
Figure FDA0003531052660000029
Figure FDA0003531052660000031
then:
Figure FDA0003531052660000032
Figure FDA0003531052660000033
substituting the formulas (8) and (9) into the formula (5) to solve the problem that the light intensity components of the transmitted light in the vertical direction and the parallel direction are:
Figure FDA0003531052660000034
and simultaneously determining the light intensity components of the reflected light in the vertical direction and the parallel direction as follows:
Figure FDA0003531052660000035
the total light intensity is equal to the sum of the light intensities in the vertical direction and the parallel direction, and therefore the light components reflected from the glass surface and the light components transmitted therefrom are determined as follows:
Figure FDA0003531052660000036
5. the separation method of the reflected light on the surface of the transparent object based on the polarization characteristics as claimed in claim 1, wherein the specific process of extracting the reflection-dominant pixel points and the transmission-dominant pixel points and the cross correlation summation in the step (S5) is as follows:
(S51) reflection dominant pixel point extraction
The specific steps of extracting the reflected light as the dominant pixel point are as follows: first, the mixed image is set to be 0.01 and 0.2, and the mixed image is subjected to separation to obtain a transmitted light image I after separationover-t(ii) a Then selecting gamma 0.99 and chi 0.2 to under-separate the mixed image and obtain the under-separated transmitted light image Iunder-t(ii) a Then find Iover-tAnd Iunder-tThe correlation between the two is obtained to obtain a correlation image Rt(ii) a Finally, the pixel point with negative correlation is set to be 1, and the pixel point with positive correlation is set to be 0, so that the reflection leading pixel point is extracted;
Figure FDA0003531052660000041
wherein R istRepresenting the transmitted light image I after separationover-tAnd an under-separated transmitted light image Iunder-tNormalized cross-correlation image between, MtRepresents a pair of RtThe result after binarization is performed.
(S52) transmission dominant pixel point extraction:
and calculating the actual transmission polarization degree value by selecting the pixel points taking the transmission light as the leading factor. First, the mixed image is set to be 0.5 and 0.01, and the mixed image is separated to obtain a reflected light image I after separationover-rThen, the mixed image is under-separated by selecting gamma 0.99 and chi 0.5, and the under-separated transmitted light image I is obtainedunder-rObtaining R by cross-correlation between the tworAnd binarizing the image:
Figure FDA0003531052660000042
wherein R isrRepresenting the separated reflected light image Iover-rAnd an under-separated reflected light image Iunder-rNormalized cross-correlation image between, MrRepresents a pair of RrThe result after binarization is performed.
(S53) cross-correlation summing of reflection-dominated pixels and transmission-dominated pixels:
reflection dominated normalized cross correlation sum f of pixelsR(γ, χ) is:
Figure FDA0003531052660000051
where row, col represent the number of rows and columns of the image, respectively.
Transmission dominated normalized cross correlation sum f of pixelsT(γ, χ) is:
Figure FDA0003531052660000052
6. the method for separating the reflected light from the surface of the transparent object based on the polarization characteristics as claimed in claim 5, wherein the step (S6) of solving the minimum value of the normalized cross-correlation by using a gradient descent method to obtain the polarization degree of the transmitted light and the polarization degree of the reflected light corresponding to the minimum value comprises the following specific steps:
normalized cross-correlation f of reflected and transmitted light imagesR(gamma, chi) and fT(gamma, chi) as a function of the degree of reflection polarization gamma and of the degree of transmission polarization chi, and then respectively for fR(gamma, chi) and fT(gamma, chi) calculating partial derivative, making it fall along gradient direction, obtaining f after several iterationsR(gamma, chi) and fT(γ, χ);
Figure FDA0003531052660000053
where η is the learning rate, n represents the current iteration number, and n +1 represents the next iteration number. When the convergence condition is reached, the iteration is ended, and the normalized cross-correlation minimum value f is obtainedR(γm,χm) And fT(γm,χm) And their corresponding transmission polarization degree gammamAnd degree of reflection polarization χm。
7. The method as claimed in claim 6, wherein the separation result is optimized when the cross correlation between the reflected light and the transmitted light is minimized.
8. The method for separating the reflected light on the surface of the transparent object based on the polarization characteristics as claimed in claim 5, wherein the reflection polarization degree γ is set to 0.01 and 0.99 in sequence in the extraction process of the reflection dominant pixel points.
9. The method for separating the reflected light on the surface of the transparent object based on the polarization characteristics as claimed in claim 5, wherein the transmission polarization degree χ is set to 0.01 and 0.99 in sequence in the process of extracting the transmission dominant pixel points.
CN202011303630.8A 2020-11-19 2020-11-19 Transparent object surface reflected light separation method based on polarization characteristics Active CN112379529B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011303630.8A CN112379529B (en) 2020-11-19 2020-11-19 Transparent object surface reflected light separation method based on polarization characteristics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011303630.8A CN112379529B (en) 2020-11-19 2020-11-19 Transparent object surface reflected light separation method based on polarization characteristics

Publications (2)

Publication Number Publication Date
CN112379529A CN112379529A (en) 2021-02-19
CN112379529B true CN112379529B (en) 2022-04-19

Family

ID=74584620

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011303630.8A Active CN112379529B (en) 2020-11-19 2020-11-19 Transparent object surface reflected light separation method based on polarization characteristics

Country Status (1)

Country Link
CN (1) CN112379529B (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101558282A (en) * 2007-02-13 2009-10-14 松下电器产业株式会社 Image processing system, method, device and image format
CN107966412A (en) * 2017-12-05 2018-04-27 天津大学 Based on cross-polarization image stretch and the associated image defogging method of degree of polarization
CN109923855A (en) * 2016-11-15 2019-06-21 索尼公司 Image processing apparatus, image processing method and program
CN111351572A (en) * 2019-11-21 2020-06-30 四川极智朗润科技有限公司 Distributed reflection elimination device and method based on polarization characteristics
CN111539411A (en) * 2020-04-20 2020-08-14 河海大学常州校区 Method for identifying objects of different materials by using polarization information

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101558282A (en) * 2007-02-13 2009-10-14 松下电器产业株式会社 Image processing system, method, device and image format
CN109923855A (en) * 2016-11-15 2019-06-21 索尼公司 Image processing apparatus, image processing method and program
CN107966412A (en) * 2017-12-05 2018-04-27 天津大学 Based on cross-polarization image stretch and the associated image defogging method of degree of polarization
CN111351572A (en) * 2019-11-21 2020-06-30 四川极智朗润科技有限公司 Distributed reflection elimination device and method based on polarization characteristics
CN111539411A (en) * 2020-04-20 2020-08-14 河海大学常州校区 Method for identifying objects of different materials by using polarization information

Also Published As

Publication number Publication date
CN112379529A (en) 2021-02-19

Similar Documents

Publication Publication Date Title
Hou et al. A novel dark channel prior guided variational framework for underwater image restoration
CN108090888B (en) Fusion detection method of infrared image and visible light image based on visual attention model
US20110043603A1 (en) System And Method For Dehazing
Gershikov et al. Horizon line detection in marine images: which method to choose?
Zhang et al. Infrared small target detection based on spatial-temporal enhancement using quaternion discrete cosine transform
WO2018171008A1 (en) Specular highlight area restoration method based on light field image
CN103761731A (en) Small infrared aerial target detection method based on non-downsampling contourlet transformation
Riaz et al. Single image dehazing via reliability guided fusion
CN110189375A (en) A kind of images steganalysis method based on monocular vision measurement
CN109886883A (en) Real-time polarization fog-penetrating imaging image enhancement processing method
CN114612359A (en) Visible light and infrared image fusion method based on feature extraction
CN112561996A (en) Target detection method in autonomous underwater robot recovery docking
Huang et al. Removing reflection from a single image with ghosting effect
CN109886195A (en) Skin identification method based on depth camera near-infrared single color gradation figure
Bi et al. Image dehazing based on polarization information and deep prior learning
CN116823694B (en) Infrared and visible light image fusion method and system based on multi-focus information integration
CN112379529B (en) Transparent object surface reflected light separation method based on polarization characteristics
Gai et al. Blind separation of superimposed images with unknown motions
Shibata et al. Unified image fusion framework with learning-based application-adaptive importance measure
CN115187791A (en) ORB image matching method fusing color and scale features
CN114882231A (en) Infrared small target detection method based on multi-angle prior analysis
Xiao et al. Underwater image classification based on image enhancement and information quality evaluation
Zhou et al. Scale-aware multispectral fusion of RGB and NIR images based on alternating guidance
CN115063524A (en) Reconstruction method and device for 3D face recognition
CN107610170B (en) Multi-view image refocusing depth acquisition method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant