CN112907582A - Image significance extraction defogging method and device for mine and face detection - Google Patents
Image significance extraction defogging method and device for mine and face detection Download PDFInfo
- Publication number
- CN112907582A CN112907582A CN202110330453.0A CN202110330453A CN112907582A CN 112907582 A CN112907582 A CN 112907582A CN 202110330453 A CN202110330453 A CN 202110330453A CN 112907582 A CN112907582 A CN 112907582A
- Authority
- CN
- China
- Prior art keywords
- image
- value
- mine
- saliency
- scale
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 46
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000001514 detection method Methods 0.000 title claims abstract description 19
- 230000005540 biological transmission Effects 0.000 claims abstract description 74
- 238000003860 storage Methods 0.000 claims abstract description 56
- 238000010586 diagram Methods 0.000 claims abstract description 39
- 230000009467 reduction Effects 0.000 claims abstract description 25
- 238000004458 analytical method Methods 0.000 claims abstract description 13
- 238000009499 grossing Methods 0.000 claims description 7
- 230000004927 fusion Effects 0.000 claims description 6
- 238000009826 distribution Methods 0.000 claims description 4
- 125000001475 halogen functional group Chemical group 0.000 abstract description 9
- 239000000428 dust Substances 0.000 abstract description 4
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000007246 mechanism Effects 0.000 description 4
- 238000012544 monitoring process Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 239000003245 coal Substances 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000007500 overflow downdraw method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 239000013598 vector Substances 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/40—Image enhancement or restoration by the use of histogram techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/73—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The invention relates to a method and a device for extracting and defogging image saliency facing a mine and face detection, belongs to the technical field of image processing, and solves the problems of defogged image distortion, supersaturation and edge halo artifacts caused by mine fog dust concentration, uneven light source and different image saliency. The method comprises the following steps: acquiring an original fog storage image under a mine; performing brightness value reduction processing on the original fog storage image based on histogram analysis, and calculating an atmospheric light estimation value according to the fog storage image after brightness value reduction; obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimation value, a preset small-scale numerical value and a preset large-scale numerical value; fusing the large-scale transmission image and the small-scale transmission image based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model; utilizing L2 regularization to constrain the significance extraction defogging model to obtain a fused transmission image; and based on the fused transmission diagram and the atmospheric light estimation value, the atmosphere scattering model is reversely solved to obtain a defogged image.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a mine-oriented image saliency extraction defogging method and device and face detection.
Background
The coal mine underground monitoring system is widely applied at present, and safety monitoring and linkage control under a dangerous area can liberate personnel from a dangerous environment and improve the guarantee capability of coal mine safety production and the monitoring and early warning level of natural disasters of a mine. However, the special environment in a mine well poses a serious challenge to mine image acquisition. In the environment under the ore deposit, a lot of tiny particulate matters have gathered in the air, and these tiny particulate matters can absorb or refract the phenomenon to light production, influence the normal radiation of light to just can produce adverse circumstances phenomenon such as fog, haze when these tiny particulate matters gather too much in a certain region, often can produce very abominable influence to colour, contrast, saturation, detail etc. of the image that monitored control system caught. To address this particular environmental impact under mines, defogging algorithms for a number of different technical lines have been proposed, of which defogging algorithms based on Dark Channel Prior models (DCP) are outstanding. The algorithm finds a priori law that for pixel points of fog-free images without sky, the pixel intensity of at least one color channel is very low. And obtaining global atmospheric light and a transmission map through the prior rule, and finally calculating a defogging map based on the global atmospheric light model.
At present, the global atmospheric light is generally taken as the mean value of the true pixel intensities corresponding to the point 0.1% before the intensity in the dark channel map, and the method reduces the error of the atmospheric light estimation value to some extent, but still has a big problem. The light source under the mine is generally a point light source, and the reflection of the point light source or the mirror surface can cause the collected image to have an excessively bright area, and the pixel intensity in the area is far greater than the intensities of other pixels, so that a serious estimation error is caused when an atmospheric light estimation value is obtained, and serious color distortion of a defogged image is caused. In addition, for the same image, the attention of people to different objects is different, the significance degree of each pixel in the image is different, the defogging supersaturation can be caused by the defogging of the small-scale model by adopting the DCP algorithm, and the vignetting artifact phenomenon can be caused at the edge position of the object by the defogging of the large-scale model by adopting the DCP algorithm.
Therefore, due to the special image acquisition environment in the mine, the problems of defogged image distortion, supersaturation and edge halo artifacts caused by mine fog dust concentration, uneven light source and different image significance degrees are difficult to solve by adopting the existing DCP image defogging scheme.
Disclosure of Invention
In view of the foregoing analysis, embodiments of the present invention provide a method and an apparatus for extracting and defogging image saliency for a mine, and a face detection, so as to solve the problems of image distortion, supersaturation, and edge halo artifacts caused by mine fog-dust concentration, uneven light source, and different image saliency.
In one aspect, an embodiment of the present invention provides a mine-oriented image saliency extraction defogging method, including the following steps:
acquiring an original fog storage image under a mine;
performing brightness value reduction processing on the original fog storage image based on histogram analysis, and calculating an atmospheric light estimation value according to the fog storage image with the brightness value reduced;
obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimation value, a preset small-scale numerical value and a preset large-scale numerical value;
fusing the large-scale transmission diagram and the small-scale transmission diagram based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model;
utilizing L2 regularization to constrain the significance extraction defogging model to obtain a fused transmission image;
and based on the fused transmission diagram and the atmospheric light estimation value, an atmosphere scattering model is reversely solved to obtain a defogged image.
Further, the brightness reduction processing is performed on the original fog storage image based on histogram analysis, and the method comprises the following steps:
dividing the original fog storage image into red, green and blue color channels;
obtaining a histogram according to the image intensity distribution of each color channel, and obtaining the maximum value, the average value and the median of the intensity of each color channel according to the histogram;
calculating the brightness factor lambda of each channel according to the maximum value, the average value and the median of the intensity of each color channelcThe formula is as follows:
in the formula, Maxc、MeacAnd MidcRespectively representing the intensity maximum value, the intensity mean value and the intensity median of each color channel, c represents the color channel, c belongs to { r, g, b }, and r, g and b respectively represent three color channels of red, green and blue in the color channel;
and comparing the brightness factor of each color channel with a brightness threshold set by the image, if the brightness factor is less than or equal to the brightness threshold, setting the intensity of the pixel point with the intensity greater than 0.9 times of the maximum value in the channel as the intensity mean value of the channel, and otherwise, not reducing the value.
Further, the calculating the atmospheric light estimation value according to the fog storage image after the brightness reduction comprises the following steps:
obtaining a dark channel image according to the fog storage image with the brightness reduced;
finding out the pixel point with the maximum intensity of 0.1% in the dark channel map;
finding out pixel points corresponding to the 0.1% pixel points in the fog-stored image after the brightness is reduced;
and taking the intensity average value of the pixel points corresponding to the fog-stored image after the brightness is reduced as an atmospheric light estimation value.
Further, the numerical value of the small scale is set to 1;
the large scale value k, set according to the resolution of the image, is expressed as follows:
in the formula, w and h represent the width and height of the image, respectively.
Further, the large-scale transmission map tpa(x) And a small scale transmission map tpi(x) Is represented as follows:
wherein α represents a long-range fogging parameter, AcRepresenting the color channel value corresponding to the atmospheric light estimation value in the dark channel, omega (x) representing a local area with a pixel point x as the center, IcAnd (y) is a color channel diagram representing the original fog-storing image.
Further, taking the saliency value in the saliency map as a weight, and weighting and fusing the large-scale transmission map and the small-scale transmission map to obtain the saliency extraction defogging model, which is expressed as follows:
in the formula, tsed(x) Significance extraction expressed at pixel point xMist transmission diagram, Wsig(x) And
respectively representing the significance of the pixel points x to extract t in the defogging modelpi(x) And tpa(x) Ratio of (A) to (B), Wsig(x) Is tpi(x) The saliency value of the saliency map corresponding to the point at which is located, and
further, the constraining the saliency extraction defogging model by utilizing the L2 regularization to obtain a fused transmission map includes the following steps:
the significance extraction defogging model was constrained by L2 regularization as follows:
in the formula, λtRepresenting a regularization coefficient, R (t)sed(x) Represents a smoothing term for the transmission map;
r (t)sed(x) Neglect), yields:
smoothing the above formula by using a guide filter to obtain smoothed tsed(x)。
Further, the inverse atmospheric scattering model is solved to obtain a defogged image, which is expressed as:
in the formula, Jsed(x) And (I) represents an original fog storage image under a mine.
In another aspect, an embodiment of the present invention provides a mine-oriented image saliency extraction defogging device, including:
the fog storage image acquisition module is used for acquiring an original fog storage image under a mine;
the atmospheric light estimation module is used for performing brightness reduction processing on the original fog storage image based on histogram analysis and calculating an atmospheric light estimation value according to the fog storage image after the brightness reduction;
the fusion transmission diagram obtaining module is used for obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimation value, a preset small-scale numerical value and a preset large-scale numerical value; fusing the large-scale transmission diagram and the small-scale transmission diagram based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model; utilizing L2 regularization to constrain the significance extraction defogging model to obtain a fused transmission image;
and the defogged image obtaining module is used for reversely solving the atmospheric scattering model based on the fused transmission image and the atmospheric light estimation value to obtain a defogged image.
The embodiment of the invention also provides a face detection method facing a mine, wherein the acquired images in the mine are processed according to the image saliency extraction defogging method of any one of claims 1 to 8 to obtain defogged images; and detecting the face in the defogged image by using a plane rotation face detection method.
Compared with the prior art, the invention can realize at least one of the following beneficial effects:
the invention provides a method and a device for extracting and defogging mine-oriented image saliency and face detection,
1. the method comprises the steps of performing brightness value reduction processing on an original fog storage image collected in a mine through histogram analysis, and calculating an atmospheric light estimation value by using the fog storage image after the brightness value reduction, so that an over-bright area in the image caused by point light source and mirror reflection in the mine is eliminated, the accuracy of the atmospheric light estimation value can be further improved, the quality of a defogged image is further improved, and the color distortion of the defogged image is avoided;
2. the large-scale transmission image and the small-scale transmission image are fused according to the saliency map of the original defogged image, so that the fused transmission image is obtained, the defogged image is finally obtained, the saliency of the image is divided, the saliency map is used as weight fusion, the edge of an object in the image can be better extracted, the small-scale defogging is carried out on the region with high saliency, the large-scale defogging is carried out on the region with low saliency, the quality of the defogged image can be more effectively improved, and the supersaturation and edge halo artifact phenomena of the defogged image are avoided;
3. through the significance extraction defogging model in the fusion process and the L2 regularization constraint, the edge region in the image can be closer to a small-scale transmission image, and the object edge in the defogged image is further prevented from generating halo artifacts.
In the invention, the technical schemes can be combined with each other to realize more preferable combination schemes. Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.
Fig. 1 is a schematic flow chart of a method for extracting and defogging the saliency of an image facing a mine in embodiment 1 of the present invention;
FIG. 2 is a logic block diagram of a method for defogging of mine-facing image saliency extraction in embodiment 1 of the present invention;
fig. 3 is a block diagram of a process of full RIP face detection in embodiment 3 of the present invention.
Detailed Description
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate preferred embodiments of the invention and together with the description, serve to explain the principles of the invention and not to limit the scope of the invention.
Example 1
The specific embodiment 1 of the invention discloses a mine-oriented image saliency extraction defogging method, a flow chart is shown in fig. 1, a logic block diagram is shown in fig. 2, and the method comprises the following steps:
and acquiring an original fog storage image under a mine. Specifically, the original fog storage image can be obtained through a video image shot by a fixed camera or a vehicle-mounted camera under the mine.
And performing brightness value reduction processing on the original fog storage image based on histogram analysis, and calculating an atmospheric light estimation value according to the fog storage image after brightness value reduction.
And obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimation value, the preset small-scale numerical value and the preset large-scale numerical value.
And fusing the large-scale transmission image and the small-scale transmission image based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model.
And (5) constraining the significance extraction defogging model by utilizing L2 regularization to obtain a fused transmission image.
And based on the fused transmission diagram and the atmospheric light estimation value, the atmosphere scattering model is reversely solved to obtain a defogged image.
Compared with the prior art, the image saliency extraction defogging method for the mine performs brightness value reduction processing on the original fog storage image collected in the mine through histogram analysis, fuses the large-scale transmission image and the small-scale transmission image according to the saliency map of the original fog storage image, and utilizes L2 regularization constraint, so that the obtained defogged image avoids the problems of image distortion, supersaturation and edge halo artifacts caused by mine fog dust concentration, light source nonuniformity and different image saliency degrees, and can obtain a high-quality defogged image.
When the method is implemented, the brightness reduction processing is carried out on the original fog storage image based on the histogram analysis, and the method comprises the following steps:
dividing an original fog storage image into three color channels of red, green and blue;
obtaining a histogram according to the image intensity distribution of each color channel, and obtaining the maximum value, the average value and the median of the intensity of each color channel according to the histogram; it should be noted that intensity is also amplitude.
Calculating the brightness factor lambda of each channel according to the maximum value, the average value and the median of the intensity of each color channelcThe formula is as follows:
in the formula, Maxc、MeacAnd MidcRespectively representing the intensity maximum value, the intensity mean value and the intensity median of each color channel, c represents the color channel, and c belongs to { r, g, b }, wherein r, g and b respectively represent three color channels of red, green and blue in the color channel.
And comparing the brightness factor of each color channel with a brightness threshold set by the image, if the brightness factor is less than or equal to the brightness threshold, setting the intensity of the pixel point with the intensity greater than 0.9 times of the maximum value in the channel as the intensity mean value of the channel, and otherwise, not reducing the value. It should be noted that a low luminance factor indicates the presence of an over-bright area in the image; the method has the advantages that the pixel points with the intensity greater than the maximum intensity value of 0.9 times in the image are selected to be subjected to value reduction, the value reduction of the over-bright pixel points in the over-bright area of the image can be better realized, the pixel points with normal brightness cannot be mistakenly subjected to value reduction, the over-bright pixel points cannot be mistaken to be used as the pixel points with normal brightness, the brightness after the value reduction is not too low or the brightness is not sufficiently reduced, and the effect of eliminating the over-bright area of the image caused by point light sources and mirror reflection in a mine can be better achieved by the image after the value reduction.
Preferably, the range of the brightness threshold set for the image is [0,1], and according to the experimental result, the brightness threshold set in this embodiment is 0.7, and set to 0.7 can better and more accurately identify the over-bright area in the image, and by reducing the brightness of the over-bright area, the calculated atmospheric light estimation value is more accurate.
When in implementation, the atmospheric light estimation value is calculated according to the fog storage image after the brightness is reduced, and the method comprises the following steps:
and obtaining a dark channel image according to the fog storage image after the brightness is reduced.
It should be noted that the dark channel map is based on the fact that for a fog-free image without sky, the intensity of the pixels of a portion having at least one color channel is very low, close to 0, and then the minimum value of the intensity of the pixels in each color channel is taken, so that the image is composed of the dark channel map. The dark channel map can be described by:
in the formula (d)Ω(x) Expressing the dark primary color value of a pixel point x in the image, omega (x) expressing a local area with the pixel point x as the center, IcAnd (y) is a diagram representing the corresponding color channels.
The pixel point with the maximum intensity of 0.1% is found in the dark channel map.
And finding out pixel points corresponding to 0.1% of the pixel points in the fog storage image after the brightness is reduced.
And taking the intensity average value of the pixel points corresponding to the fog-stored image after the brightness is reduced as an atmospheric light estimation value.
It should be noted that the fog image after brightness reduction is also divided into three color channels r, g and b, each color channel corresponding to a component of the atmospheric light estimation value, i.e. ar、Ag、AbSpecifically, the intensity average value of corresponding pixel points on each color channel of the fog-stored image with the brightness reduced is used as the atmospheric light estimation value component of the corresponding color channel, that is, the atmospheric light estimation value component is the pixel points corresponding to 0.1% of the pixel points found in the color channel corresponding to the fog-stored image with the brightness reduced, and the intensity average value of the corresponding pixel points of the corresponding color channel is used as the component of the atmospheric light estimation value in the corresponding color channel.
In practice, the value of the small scale is set to 1.
The large scale value k, set according to the resolution of the image, is expressed as follows:
wherein w and h represent the width and height of the image, respectively;
representing a rounded function.
It should be noted that the small-scale value is set to 1, that is, pixel-level defogging is performed, so that the edge region of an object in an image can be defogged more finely, and a halo artifact phenomenon can be better avoided; the large scale is set according to the resolution of the image, the adaptability can be adjusted according to the image, the large scale selection of the images with different sizes is better adapted, and the adaptability is stronger.
In practice, the large scale transmission map tpa(x) And a small scale transmission map tpi(x) Is represented as follows:
wherein, alpha represents a long-range fogging parameter, generally 0.95, AcRepresenting the corresponding color channel value of the atmospheric light estimate in the dark channel, i.e. Ar、Ag、AbΩ (x) denotes a local area centered on the pixel point x, IcAnd (y) is a color channel diagram representing the original fog-storing image.
In implementation, the Significance value in the Significance map is used as a weight, and the large-scale transmission map and the small-scale transmission map are weighted and fused to obtain a Significance extraction defogging model (SED), which is expressed as follows:
in the formula, tsed(x) The significance of the representation at pixel point x is extracted to remove the fog transmission diagram, Wsig(x) And
respectively representing the significance of the pixel points x to extract t in the defogging modelpi(x) And tpa(x) Ratio of (A) to (B), Wsig(x) Is tpi(x) The saliency value of the saliency map corresponding to the point at which is located, and
it should be noted that the original fog storage image is subjected to significance division, and the region with high significance is subjected to small-scale defogging, and the region with high significance contains the edge of an object, so that the phenomenon of halo artifacts generated at the edge of the object can be effectively avoided; the large-scale defogging is carried out on the region with low significance, the supersaturation phenomenon of the defogged image is avoided, and therefore the defogging model is extracted based on the significance of the saliency map fusion, and the quality of the defogged image can be effectively improved.
Specifically, the method for extracting the saliency region of the original fog storage image by adopting an attention-based mechanism to obtain the saliency map of the original fog storage image comprises the following steps:
firstly, establishing an attention mechanism model through a Bayesian rule, wherein the formula is as follows:
in the formula, szRepresenting the significance of the pixel point z; z represents a specific pixel point to be classified, wherein the classification refers to significance and non-significance; c is a binary variable of whether the point is in the target classification or not, 1 represents yes, and 0 represents no; l is a random variable of the position of the point; f is the visual characteristic of the point; f. ofzIs the eigenvalue at z; lzIs the position of the point.
Then, assuming that the eigenvalues are independent of position, and given C ═ 1, we get:
p(F＝fz,L＝lz)＝p(F＝fz)p(L＝Lz)，
p(F＝fz,L＝lz|C＝1)＝p(F＝fz|C＝1)p(L＝lz|C＝1)，
therefore, the probability distribution equation is obtained after the attention mechanism model takes logarithm:
log sz＝-logp(F＝fz)+logp(L＝lz|C＝1)+p(C＝1)p(L＝lz)，
finally, computing the features as linear responses of a difference of gaussians (DoG) filter yields a saliency map.
It should be noted that, by adopting the attention-based mechanism to obtain the saliency map, the saliency map can be obtained according to the attention degree of people on different objects in the image, so that the defogged image obtained by the method is more consistent with the logic based on the attention of the human eyes.
In implementation, the significance extraction defogging model is constrained by utilizing L2 regularization to obtain a transmission graph after fusion, and the method comprises the following steps:
the defogging model constraint on significance extraction by L2 regularization is as follows:
in the formula, λtRepresenting a regularization coefficient, R (t)sed(x) Represents a smoothing term for the transmission map;
it should be noted that the constraint on the weights by using L2 regularization is to find the most suitable value under the weight effect so that the first three terms in the above equation can reach the minimum value, so as to find the optimal tsed(x) So that the edge area and t in the fog-storing imagepi(x) Closest to, and other regions with tpa(x) Most closely, and at the same time, the third term is to avoid invalid output, so as to obtain the optimal tsed(x) And the halo artifact generated at the edge of an object in the image is better inhibited, so that the defogged image has better quality.
R (t)sed(x) Neglect), yields:
smoothing the above formula by using a guide filter to obtain smoothed tsed(x)。
In implementation, the atmosphere scattering model is inversely solved to obtain a defogged image, which is expressed as:
in the formula, Jsed(x) And (I) represents an original fog storage image under a mine.
Specifically, the atmospheric scattering model is expressed as:
I(x)＝Jsed(x)tsed(x)+A(1-tsed(x))，
the atmosphere scattering model is reversely solved to obtain a defogged image Jsed(x)：
Example 2
The specific embodiment 2 of the invention discloses a mine-oriented image saliency extraction defogging device, which comprises:
the fog storage image acquisition module is used for acquiring an original fog storage image under a mine;
the atmospheric light estimation module is used for performing brightness reduction processing on the original fog storage image based on histogram analysis and calculating an atmospheric light estimation value according to the fog storage image after the brightness reduction;
the fusion transmission diagram obtaining module is used for obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimation value, the preset small-scale numerical value and the preset large-scale numerical value; fusing the large-scale transmission image and the small-scale transmission image based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model; utilizing L2 regularization to constrain the significance extraction defogging model to obtain a fused transmission image;
and the defogged image obtaining module is used for reversely solving the atmospheric scattering model based on the fused transmission image and the atmospheric light estimation value to obtain a defogged image.
It should be noted that the present embodiment can be referred to in relation to embodiment 1, and the description is repeated here. As a specific calculation manner of the atmospheric light estimation module in this embodiment 2, a calculation manner of the atmospheric light estimation value in embodiment 1 may be selected; in this embodiment 2, a specific fusion method in embodiment 1 is selected according to a fusion method of a transmission map obtaining module.
Example 3
The specific embodiment 3 of the invention discloses a face detection method facing a mine, wherein the collected images in the mine are processed according to the image saliency extraction defogging method facing the mine in the embodiment 1 to obtain defogged images; and detecting the face in the defogged image by using a plane rotation face detection method.
Specifically, In this embodiment, when the face detection method is used to detect the face In the defogged image, the face detection method is implemented by using a Rotation In Plane (RIP), and is divided into three layers:
detecting a face candidate object facing downwards for the first-layer network and rotating the face candidate object by 180 degrees, namely reducing the RIP range from [ -180 degrees, 180 degrees ] to [ -90 degrees, 90 degrees ]; detecting a face candidate target processed by the first layer of network, rotating the face candidate target by 45 degrees, namely reducing the RIP range from [ -90 degrees and 90 degrees ] to [ -45 degrees and 45 degrees ], and meanwhile, judging whether the face candidate target is a face or not, and discarding the face candidate target if not; and finally, fine adjustment is carried out through a third-layer network, whether the angle is a human face is judged, and an accurate RIP angle is predicted.
For example, in this embodiment, a rough-to-fine mode is adopted in stages, each stage detects a face and aligns the RIP direction to be upright, and only one direction of the face is detected in each stage, so that the network is simple and the time cost is low, as shown in fig. 3, the specific implementation process is as follows:
a first layer network:
firstly, acquiring all face candidate targets through a sliding window and an image pyramid principle; then removing the candidates of the non-face target by utilizing each P layer, regressing the candidates of the face target and adjusting the RIP direction of the candidates to be the vertical direction; and finally merging the candidate targets with high overlapping degree by utilizing non-maximum compression (NMS). That is, the first layer network has three goals: face/non-face classification; frame regression; calibration, expressed as follows:
[f,t,g]＝F1(x)，
wherein x represents each input window; f1Represents a small CNN network detector; f, human face confidence score; t frame regression vectors; g orientation fraction.
The first target face/non-face classification uses the softmax loss function Lcls：
Lcls＝ylogf+(1-y)log(1-f)，
In the formula, y-1 represents a face, and y-0 represents a non-face.
Second target bounding box regression vector loss function Lreg：
Lreg(t,t*)＝S(t-t*)，
In the formula, t and t represent predicted and true values, respectively; s represents a robust smoothing l1 loss.
The bounding box regression target contains 3 aspects:
tw＝w*/w，
ta＝(a*+0.5w*-a-0.5w)/w*，
tb＝(b*+0.5w*-b-0.5w)/w*，
in the formula, a, b and w respectively represent the horizontal and vertical coordinates and the line width of the frame;
thus, the first layer network may be defined as:
in the formula, LcalCoarse directional loss function, λ, representing a predicted faceregAnd λcalRepresenting a balance factor for balancing the parameters of the different loss functions.
Layer two:
further discriminating between faces and non-faces in the second network, regressing bounding boxes, and calibrating candidate faces. Unlike the first layer network, the prediction of the coarse direction phase is the ternary classification angle range of the RIP, i.e., -90, -45 or [ -45,45] or [45,90 ].
In the training stage of the second stage, the initial training image is uniformly rotated within the range of [90,90], and negative samples, namely non-human face samples, are filtered out through the trained first layer network. It will be appreciated that samples that are not within range will not contribute to calibration training.
Layer three:
after the layer two network, all candidate faces are aligned to the vertical quarter RIP range, i.e., -45, 45. At this point, the third tier network can make a final decision to accurately determine whether it is a face and return to the bounding box. And because the RIP is reduced to a small range in the previous stages, the third-layer network can directly regress the accurate RIP angle of the candidate face instead of the rough direction.
In the training stage of the third stage, the initial training image is uniformly rotated within the range of 45,45 degrees, and negative samples are filtered out through the trained second-layer network. The calibration branch is a regression task, trained with smoothed l1 losses.
Compared with the prior art, the method has the advantages that the underground mine image is subjected to defogging treatment firstly to obtain a defogged image, and then face candidate targets in all the defogged images are obtained through a sliding window and an image pyramid principle; then removing the candidates of the non-face target by utilizing each P layer, regressing the candidates of the face target and adjusting the RIP direction of the candidates to be the vertical direction; and finally merging the candidate targets with high overlapping degree by utilizing non-maximum compression (NMS). By defogging the image, a clearer image can be obtained, which is more beneficial to face detection; by flipping the image three times, a coarse RIP prediction based calibration can be effectively achieved with little additional time overhead, i.e., rotating the original image by 90 °, -90 °, and 180 ° to left, right, and down the image. With accurate and fast calibration, the face candidates are gradually calibrated to be upright and are easier to detect. The face detection is carried out in steps, the task of each step is very simple, and the face detection is better carried out while the time cost is reduced. In the detection step, only the change from top to bottom and from left to right is carried out in each step, so that the time cost is very low, the rough calibration mode is more robust, and the RIP prediction is more accurate.
Those skilled in the art will appreciate that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program, which is stored in a computer readable storage medium, to instruct related hardware. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory.
The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.
Claims (10)
1. A mine-oriented image saliency extraction defogging method is characterized by comprising the following steps:
acquiring an original fog storage image under a mine;
performing brightness value reduction processing on the original fog storage image based on histogram analysis, and calculating an atmospheric light estimation value according to the fog storage image with the brightness value reduced;
obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimation value, a preset small-scale numerical value and a preset large-scale numerical value;
fusing the large-scale transmission diagram and the small-scale transmission diagram based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model;
utilizing L2 regularization to constrain the significance extraction defogging model to obtain a fused transmission image;
and based on the fused transmission diagram and the atmospheric light estimation value, an atmosphere scattering model is reversely solved to obtain a defogged image.
2. The mine-oriented image saliency extraction defogging method according to claim 1, wherein said original fog storage image brightness reduction processing based on histogram analysis comprises the following steps:
dividing the original fog storage image into red, green and blue color channels;
obtaining a histogram according to the image intensity distribution of each color channel, and obtaining the maximum value, the average value and the median of the intensity of each color channel according to the histogram;
calculating the brightness factor lambda of each channel according to the maximum value, the average value and the median of the intensity of each color channelcThe formula is as follows:
in the formula, Maxc、MeacAnd MidcRespectively representing the intensity maximum value, the intensity mean value and the intensity median of each color channel, c represents the color channel, c belongs to { r, g, b }, and r, g and b respectively represent three color channels of red, green and blue in the color channel;
and comparing the brightness factor of each color channel with a brightness threshold set by the image, if the brightness factor is less than or equal to the brightness threshold, setting the intensity of the pixel point with the intensity greater than 0.9 times of the maximum value in the channel as the intensity mean value of the channel, and otherwise, not reducing the value.
3. The mine-oriented image saliency extraction defogging method according to claim 1, wherein said calculating an atmospheric light estimate value from a brightness-reduced fog-stored image comprises the steps of:
obtaining a dark channel image according to the fog storage image with the brightness reduced;
finding out the pixel point with the maximum intensity of 0.1% in the dark channel map;
finding out pixel points corresponding to the 0.1% pixel points in the fog-stored image after the brightness is reduced;
and taking the intensity average value of the pixel points corresponding to the fog-stored image after the brightness is reduced as an atmospheric light estimation value.
4. The mine-oriented image saliency extraction defogging method according to claim 1, wherein said small scale value is set to 1;
the large scale value k, set according to the resolution of the image, is expressed as follows:
in the formula, w and h represent the width and height of the image, respectively.
5. The mine-oriented image saliency extraction defogging method according to claim 4, wherein said large-scale transmission map tpa(x) And a small scale transmission map tpi(x) Is represented as follows:
wherein α represents a long-range fogging parameter, AcRepresenting the color channel value corresponding to the atmospheric light estimation value in the dark channel, omega (x) representing a local area with a pixel point x as the center, IcAnd (y) is a color channel diagram representing the original fog-storing image.
6. The mine-oriented image saliency extraction defogging method according to claim 5, wherein the saliency values in the saliency map are taken as weights, and the large-scale transmission map and the small-scale transmission map are weighted and fused to obtain the saliency extraction defogging model, which is represented as follows:
in the formula, tsed(x) The significance of the representation at pixel point x is extracted to remove the fog transmission diagram, Wsig(x) And
respectively representing the significance of the pixel points x to extract t in the defogging modelpi(x) And tpa(x) Ratio of (A) to (B), Wsig(x) Is tpi(x) The saliency value of the saliency map corresponding to the point at which is located, and
7. the mine-oriented image saliency extraction defogging method according to claim 6, wherein said saliency extraction defogging model is constrained using L2 regularization to obtain a fused transmission map, comprising the steps of:
the significance extraction defogging model was constrained by L2 regularization as follows:
in the formula, λtRepresenting a regularization coefficient, R (t)sed(x) Represents a smoothing term for the transmission map;
r (t)sed(x) Neglect), yields:
using guided filters of the above-mentioned typeSmoothing is carried out to obtain t after smoothingsed(x)。
8. The mine-oriented image saliency extraction defogging method according to claim 7, wherein said inverse atmospheric scattering model yields a defogged image represented as:
in the formula, Jsed(x) And (I) represents an original fog storage image under a mine.
9. An image saliency extraction defogging device for a mine is characterized by comprising:
the fog storage image acquisition module is used for acquiring an original fog storage image under a mine;
the atmospheric light estimation module is used for performing brightness reduction processing on the original fog storage image based on histogram analysis and calculating an atmospheric light estimation value according to the fog storage image after the brightness reduction;
the fusion transmission diagram obtaining module is used for obtaining a large-scale transmission diagram and a small-scale transmission diagram according to the atmospheric light estimation value, a preset small-scale numerical value and a preset large-scale numerical value; fusing the large-scale transmission diagram and the small-scale transmission diagram based on the saliency map of the original fog storage image to obtain a saliency extraction defogging model; utilizing L2 regularization to constrain the significance extraction defogging model to obtain a fused transmission image;
and the defogged image obtaining module is used for reversely solving the atmospheric scattering model based on the fused transmission image and the atmospheric light estimation value to obtain a defogged image.
10. A mine-oriented face detection method, which is characterized in that the acquired images in the mine are processed according to the image saliency extraction defogging method of any one of claims 1 to 8 to obtain defogged images; and detecting the face in the defogged image by using a plane rotation face detection method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110330453.0A CN112907582B (en) | 2021-03-24 | 2021-03-24 | Mine-oriented image saliency extraction defogging method and device and face detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110330453.0A CN112907582B (en) | 2021-03-24 | 2021-03-24 | Mine-oriented image saliency extraction defogging method and device and face detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112907582A true CN112907582A (en) | 2021-06-04 |
| CN112907582B CN112907582B (en) | 2023-09-29 |
Family
ID=76109122
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110330453.0A Active CN112907582B (en) | 2021-03-24 | 2021-03-24 | Mine-oriented image saliency extraction defogging method and device and face detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112907582B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060056304A (en) * | 2006-05-04 | 2006-05-24 | 한양대학교 산학협력단 | Apparatus and method for adaptive image enhancement |
| CN105761227A (en) * | 2016-03-04 | 2016-07-13 | 天津大学 | Underwater image enhancement method based on dark channel prior algorithm and white balance |
| US20160267850A1 (en) * | 2015-03-10 | 2016-09-15 | Canon Kabushiki Kaisha | Image display apparatus and control method thereof |
| CN106530246A (en) * | 2016-10-28 | 2017-03-22 | 大连理工大学 | Image dehazing method and system based on dark channel and non-local prior |
| CN107301625A (en) * | 2017-06-05 | 2017-10-27 | 天津大学 | Image defogging algorithm based on brightness UNE |
| CN108596853A (en) * | 2018-04-28 | 2018-09-28 | 上海海洋大学 | Underwater picture Enhancement Method based on bias light statistical model and transmission map optimization |
| CN109961412A (en) * | 2019-03-18 | 2019-07-02 | 浙江大华技术股份有限公司 | A kind of video frame images defogging method and equipment |
| CN110046670A (en) * | 2019-04-24 | 2019-07-23 | 北京京东尚科信息技术有限公司 | Feature vector dimension reduction method and device |
| CN110175972A (en) * | 2019-05-29 | 2019-08-27 | 南京信息职业技术学院 | A kind of infrared image enhancing method based on transmission plot fusion |
| US20200141856A1 (en) * | 2017-07-20 | 2020-05-07 | Shanghai Ruiyu Biotech Co., Ltd. | Method and system for determining fluorescence intensity of fluorescence image |
| CN111785224A (en) * | 2019-04-04 | 2020-10-16 | 海信视像科技股份有限公司 | Brightness driving method |
| CN111932469A (en) * | 2020-07-21 | 2020-11-13 | 泉州职业技术大学 | Significance weight quick exposure image fusion method, device, equipment and medium |
-
2021
- 2021-03-24 CN CN202110330453.0A patent/CN112907582B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20060056304A (en) * | 2006-05-04 | 2006-05-24 | 한양대학교 산학협력단 | Apparatus and method for adaptive image enhancement |
| US20160267850A1 (en) * | 2015-03-10 | 2016-09-15 | Canon Kabushiki Kaisha | Image display apparatus and control method thereof |
| CN105761227A (en) * | 2016-03-04 | 2016-07-13 | 天津大学 | Underwater image enhancement method based on dark channel prior algorithm and white balance |
| CN106530246A (en) * | 2016-10-28 | 2017-03-22 | 大连理工大学 | Image dehazing method and system based on dark channel and non-local prior |
| CN107301625A (en) * | 2017-06-05 | 2017-10-27 | 天津大学 | Image defogging algorithm based on brightness UNE |
| US20200141856A1 (en) * | 2017-07-20 | 2020-05-07 | Shanghai Ruiyu Biotech Co., Ltd. | Method and system for determining fluorescence intensity of fluorescence image |
| CN108596853A (en) * | 2018-04-28 | 2018-09-28 | 上海海洋大学 | Underwater picture Enhancement Method based on bias light statistical model and transmission map optimization |
| CN109961412A (en) * | 2019-03-18 | 2019-07-02 | 浙江大华技术股份有限公司 | A kind of video frame images defogging method and equipment |
| CN111785224A (en) * | 2019-04-04 | 2020-10-16 | 海信视像科技股份有限公司 | Brightness driving method |
| CN110046670A (en) * | 2019-04-24 | 2019-07-23 | 北京京东尚科信息技术有限公司 | Feature vector dimension reduction method and device |
| CN110175972A (en) * | 2019-05-29 | 2019-08-27 | 南京信息职业技术学院 | A kind of infrared image enhancing method based on transmission plot fusion |
| CN111932469A (en) * | 2020-07-21 | 2020-11-13 | 泉州职业技术大学 | Significance weight quick exposure image fusion method, device, equipment and medium |
Non-Patent Citations (8)
| Title |
|---|
| DONG ZHAO等: "Multi-scale Optimal Fusion model for single image dehazing", 《SIGNAL PROCESSING: IMAGE COMMUNICATION》, vol. 74, pages 253 * |
| FAN YAO等: "Jop-alarm: Detecting jump-oriented programming-based anomalies in applications", 《2013 IEEE 31ST INTERNATIONAL CONFERENCE ON COMPUTER DESIGN (ICCD)》, pages 467 - 470 * |
| OMPRAKASH PATEL等: "A comparative study of histogram equalization based image enhancement techniques for brightness preservation and contrast enhancement", 《SIGNAL & IMAGE PROCESSING : AN INTERNATIONAL JOURNAL (SIPIJ)》, vol. 4, no. 5, pages 11 - 25 * |
| SASMITA MAHAKUD等: "Gray Scale Image Restoration Using Advanced Atmospheric Light Estimation", 《2019 FIFTH INTERNATIONAL CONFERENCE ON IMAGE INFORMATION PROCESSING (ICIIP)》, pages 80 - 85 * |
| YINCUI XU等: "Single Image Haze Removal with Improved Atmospheric Light Estimation", 《JOURNAL OF PHYSICS: CONFERENCE SERIES》, vol. 1098, no. 1, pages 1 - 6 * |
| 舒巧玲等: "基于暗通道和多正则化约束的图像去雾方法", 《计算机工程与设计》, vol. 40, no. 7, pages 1950 - 1955 * |
| 郭聪: "图像及视频去雾算法研究及实现", 《中国优秀硕士学位论文全文数据库：信息科技辑》, no. 2018, pages 138 - 610 * |
| 鲍唤唤: "基于天空识别的单幅图像去雾方法", 《中国优秀硕士学位论文全文数据库：信息科技辑》, no. 2020, pages 138 - 1817 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112907582B (en) | 2023-09-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108596849B (en) | Single image defogging method based on sky region segmentation | |
| US8553086B2 (en) | Spatio-activity based mode matching | |
| CN111489330B (en) | Weak and small target detection method based on multi-source information fusion | |
| CN109255326B (en) | Traffic scene smoke intelligent detection method based on multi-dimensional information feature fusion | |
| CN113989613A (en) | Light-weight high-precision ship target detection method coping with complex environment | |
| CN107315990B (en) | Pedestrian detection algorithm based on XCS-LBP characteristics | |
| CN109242032B (en) | Target detection method based on deep learning | |
| CN110060221B (en) | Bridge vehicle detection method based on unmanned aerial vehicle aerial image | |
| CN103186887A (en) | Image demisting device and image demisting method | |
| TW201928788A (en) | Object detecting device, object detecting method and computer-readable medium | |
| CN111161222A (en) | Printing roller defect detection method based on visual saliency | |
| CN112200746A (en) | Defogging method and device for traffic scene image in foggy day | |
| CN112862744A (en) | Intelligent detection method for internal defects of capacitor based on ultrasonic image | |
| Choi et al. | Fog detection for de-fogging of road driving images | |
| CN110188693B (en) | Improved complex environment vehicle feature extraction and parking discrimination method | |
| CN113205494A (en) | Infrared small target detection method and system based on adaptive scale image block weighting difference measurement | |
| CN112907582B (en) | Mine-oriented image saliency extraction defogging method and device and face detection | |
| Marinas et al. | Detection and tracking of traffic signs using a recursive Bayesian decision framework | |
| CN110321828B (en) | Front vehicle detection method based on binocular camera and vehicle bottom shadow | |
| CN116563659A (en) | Optical smoke detection method combining priori knowledge and feature classification | |
| Alluhaidan et al. | Retinex-based framework for visibility enhancement during inclement weather with tracking and estimating distance of vehicles | |
| CN112949389A (en) | Haze image target detection method based on improved target detection network | |
| Sivaanpu et al. | Scene-Specific Dark Channel Prior for Single Image Fog Removal | |
| Long et al. | An Efficient Method For Dark License Plate Detection | |
| Rabha | Background modelling by codebook technique for automated video surveillance with shadow removal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information |
Inventor after: Kou Qiqi Inventor after: Cheng Deqiang Inventor after: You Yangyang Inventor after: Wang Zhenyu Inventor before: Cheng Deqiang Inventor before: You Yangyang Inventor before: Wang Zhenyu Inventor before: Kou Qiqi |
|
| CB03 | Change of inventor or designer information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |