CN108090888B - Fusion detection method of infrared image and visible light image based on visual attention model - Google Patents
Fusion detection method of infrared image and visible light image based on visual attention model Download PDFInfo
- Publication number
- CN108090888B CN108090888B CN201810007446.5A CN201810007446A CN108090888B CN 108090888 B CN108090888 B CN 108090888B CN 201810007446 A CN201810007446 A CN 201810007446A CN 108090888 B CN108090888 B CN 108090888B
- Authority
- CN
- China
- Prior art keywords
- image
- fusion
- visible light
- target
- infrared
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 122
- 230000000007 visual effect Effects 0.000 title claims abstract description 31
- 238000001514 detection method Methods 0.000 title claims abstract description 25
- 238000000034 method Methods 0.000 claims abstract description 33
- 238000013507 mapping Methods 0.000 claims abstract description 21
- 238000002372 labelling Methods 0.000 claims abstract description 17
- 238000007781 pre-processing Methods 0.000 claims abstract description 4
- 238000000354 decomposition reaction Methods 0.000 claims description 9
- 230000002123 temporal effect Effects 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 7
- 230000003068 static effect Effects 0.000 claims description 7
- 230000003044 adaptive effect Effects 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 12
- 238000002474 experimental method Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 5
- 238000007500 overflow downdraw method Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 208000003464 asthenopia Diseases 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000005855 radiation Effects 0.000 description 2
- 239000000779 smoke Substances 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007499 fusion processing Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000011430 maximum method Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/73—Deblurring; Sharpening
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a fusion detection method of an infrared image and a visible light image based on a visual attention model. The method comprises the following steps: respectively preprocessing the collected infrared image and the collected visible light image; extracting an interest target from the preprocessed infrared image and the preprocessed visible light image based on a visual attention model of human eyes; taking the preprocessed visible light image as a background, and carrying out gray level image fusion on the corresponding preprocessed infrared image and the visible light image as the background to obtain a gray level fusion image; and carrying out pseudo-color mapping and labeling on the interest target on the gray level fusion image to obtain and output a target pseudo-color fusion image. The method and the device can solve the problem that the interested target is weakened in the fusion image in the prior art, greatly improve the accuracy and reliability of the detection and identification of the interested target, and reduce the difficulty of target identification.
Description
Technical Field
The application relates to the technical field of image processing, in particular to a fusion detection method of an infrared image and a visible light image based on a visual attention model, which is applicable to detection and identification of weak and small targets or targets with complex backgrounds in a fusion image.
Background
The infrared thermal imaging system mainly depends on the radiation of an object and a background to generate a scene image, but the infrared image has poor contrast and is only sensitive to the radiation of an object scene and is not sensitive to the brightness change of the scene. Compared with an infrared image, the visible light image can provide more target details and is beneficial to human eyes to observe. Therefore, by utilizing the infrared and visible light image fusion technology, the information redundancy and complementarity among multi-source images obtained by a plurality of sensors can be fully embodied in one image.
On the detection of weak and small targets, the infrared and visible light images have the common characteristics of low signal-to-noise ratio, large spatial and temporal correlation of image signals, small spatial and temporal correlation of image noise and the like. The gray level distribution of the infrared target is very concentrated and has a large gray level, while the gray level of the target in the visible light image is almost submerged in the gray level of the background, and is difficult to separate from the background, and the image quality is greatly challenged. In the infrared/visible light images of some scenes, if the gray value of another source image corresponding to a weak and small infrared target is too high or the background is too complex, the recognition of the target in the fusion result is affected.
For example, as shown in fig. 1a to 1d, fig. 1a is an exemplary diagram of an infrared image of a sea surface target, in which a suspected small target (marked with a circle) is located in a distant scene except that one large target and three small targets are obvious in a foreground region. However, the corresponding distant scene of the visible image in fig. 1b is a background with high brightness, so that the small suspected object in fig. 1a is almost completely submerged in the background in fig. 1b and is difficult to separate from the background. This situation therefore poses a challenge to the effectiveness of the image fusion results.
In the prior art, the infrared image in fig. 1a and the visible light image in fig. 1b are fused by using a gray scale fusion method based on wavelet transform or a pseudo color fusion method based on the Waxman algorithm to obtain corresponding fused images, such as the gray scale fused image in fig. 1c and the pseudo color fused image in fig. 1 d.
However, as shown in fig. 1c and 1d, the contrast of the above-mentioned small suspected objects in fig. 1a in the fused image is greatly reduced due to the influence of the background of the medium-wave infrared image, and especially in the Waxman pseudo-color fused image, the above-mentioned small suspected objects are almost completely submerged in the background. If the suspected small target is detected from the gray-scale fusion image or the pseudo-color fusion image, the detection difficulty is much greater than the detection difficulty of the suspected small target in the image of a single source.
In summary, the image fusion method in the prior art may reduce the contrast of the fused target due to the above disadvantages and shortcomings, thereby greatly increasing the difficulty of detecting the suspected target in the image.
Disclosure of Invention
In view of this, the invention provides a fusion detection method of an infrared image and a visible light image based on a visual attention model, so that the problem that an interested target is weakened in a fusion image in the prior art can be solved, the accuracy and reliability of detection and identification of the interested target are greatly improved, and the difficulty of target identification is reduced.
The technical scheme of the invention is realized as follows:
a fusion detection method of an infrared image and a visible light image based on a visual attention model comprises the following steps:
respectively preprocessing the collected infrared image and the collected visible light image;
extracting an interest target from the preprocessed infrared image and the preprocessed visible light image based on a visual attention model of human eyes;
taking the preprocessed visible light image as a background, and carrying out gray level image fusion on the corresponding preprocessed infrared image and the visible light image as the background to obtain a gray level fusion image;
and carrying out pseudo-color mapping and labeling on the interest target on the gray level fusion image to obtain and output a target pseudo-color fusion image.
Preferably, the extracting the interest target from the preprocessed infrared image and visible light image based on the visual attention model of human eyes includes:
generating a brightness saliency map of the static image according to an image sequence consisting of a plurality of preprocessed infrared images;
generating a motion saliency map according to an image sequence consisting of a plurality of preprocessed infrared images;
weighting, adding and fusing the brightness saliency map and the motion saliency map according to a preset proportion to obtain a final feature saliency map of the infrared image;
separating an interest target from a background in a feature saliency map of the infrared image by a local self-adaptive threshold segmentation method according to pixel gray scale similarity and the adjacency of the centroid of the salient region, and extracting the interest target in the infrared image;
generating a brightness saliency map of the static image according to an image sequence consisting of a plurality of preprocessed visible light images;
generating a motion saliency map of the visible light image according to an image sequence consisting of a plurality of preprocessed visible light images;
weighting, adding and fusing the brightness saliency map and the motion saliency map according to a preset proportion to obtain a final feature saliency map of the visible light image;
separating the interest target from the background in the characteristic saliency map of the visible light image by a local adaptive threshold segmentation method according to the pixel gray level similarity and the adjacency of the centroid of the salient region, and extracting the interest target in the visible light image;
and fusing the interest targets in the infrared image and the visible light image by using a preset fusion rule to obtain a final interest target in the target fusion image.
Preferably, the generating a luminance saliency map of the still image from the image sequence consisting of the plurality of preprocessed infrared images comprises:
establishing an image pyramid according to each preprocessed infrared image in the image sequence to obtain a plurality of infrared images with different resolutions;
converting and adjusting the infrared images with different resolutions through difference values to obtain a plurality of characteristic difference graphs;
and merging the obtained feature difference graphs through a normalized operator to obtain a final brightness saliency graph of the static image.
Preferably, the generating a motion saliency map from an image sequence consisting of a plurality of preprocessed infrared images comprises:
obtaining an image motion vector according to an image sequence consisting of a plurality of preprocessed infrared images; the image motion vector includes: intensity differences, spatial consistency differences, and temporal consistency differences;
and generating a motion saliency map according to the intensity difference, the spatial consistency difference and the temporal consistency difference.
Preferably, the predetermined fusion rule is an or rule;
in the or rule, the maximum value of the gray value of the same pixel in the region where the interest target of the corresponding visible light image and the infrared image is located is used as the gray value of the pixel in the target fusion image.
Preferably, the taking the preprocessed visible light image as a background, and performing gray level image fusion on the corresponding preprocessed infrared image and the visible light image as the background to obtain a gray level fusion image includes:
selecting a wavelet basis function and the number of decomposition layers, and respectively performing multi-resolution decomposition on the preprocessed visible light image and the preprocessed infrared image to obtain visible light images and infrared images of different scale spaces;
extracting image edge features from low-frequency components of the visible light image and the infrared image of each different scale space;
and carrying out fusion operation on the visible light images and the infrared images in different scale spaces according to a preset fusion rule to obtain multi-resolution expression of the fusion image, and carrying out wavelet inverse transformation to obtain a gray fusion image.
Preferably, the preset fusion rule is a weighted fusion rule for determining the proportion of the source image according to the gradient and the information entropy.
Preferably, the performing the pseudo-color mapping and labeling of the interest target on the gray-scale fusion image to obtain and output a target pseudo-color fusion image includes:
and inversely mapping the obtained interest target to the gray level fusion image in HSV space, and performing pseudo-color mapping and labeling on the interest target to obtain a target pseudo-color fusion image.
Preferably, the pseudo-color mapping and labeling the target of interest includes:
and marking the interest target extracted according to the source image characteristics in the gray level fusion image by using a preset color.
As can be seen from the above, in the method for detecting fusion of an infrared image and a visible image based on a visual attention model in the present invention, since an interested target can be extracted from a preprocessed infrared image based on the visual attention model of human eyes, then the preprocessed visible image and the infrared image are fused to obtain a gray-scale fusion image, then the gray-scale fusion image is subjected to pseudo-color mapping and labeling of the interested target, the target extracted according to the source image characteristics is labeled with a specific color, and the target pseudo-color fusion image is obtained and output, an effect of highlighting the interested target without affecting the gray-scale background thereof can be achieved, and the problem that the interested target in the prior art is weakened in the fusion image can be solved, and the accuracy and reliability of the detection and the recognition of the interested target can be greatly improved, the difficulty of target identification is reduced. In addition, because the target pseudo-color fusion image is only subjected to pseudo-color mapping and labeling on the local region where the interested target is located, the visual fatigue of human eyes is not easily caused.
Drawings
FIG. 1a is an exemplary diagram of an infrared image of a sea surface target in the prior art.
FIG. 1b is an exemplary diagram of a visible light image of a sea surface target in the prior art.
Fig. 1c is an exemplary diagram of a wavelet transform-based grayscale fusion image of a sea surface object in the prior art.
FIG. 1d is an exemplary diagram of a pseudo-color fusion image based on the Waxman algorithm of a sea surface target in the prior art.
Fig. 2 is an exemplary diagram of a fused image in the embodiment of the present invention.
Fig. 3 is a schematic flow chart of a fusion detection method of an infrared image and a visible light image based on a visual attention model in an embodiment of the present invention.
Fig. 4 is a flowchart illustrating a specific implementation manner of step 32 in the embodiment of the present invention.
Fig. 5 is a flowchart illustrating a specific implementation manner of step 33 in the embodiment of the present invention.
Fig. 6a is an exemplary diagram of a visible light image in a specific experiment in an embodiment of the present invention.
FIG. 6b is an exemplary graph of a long wave infrared image for a particular experiment in an embodiment of the present invention.
Fig. 6c is an exemplary diagram of a fused image obtained by using the WMM method in a specific experiment in the embodiment of the present invention.
Fig. 6d is an exemplary diagram of a fused image obtained by using the WRE method in a specific experiment in the embodiment of the present invention.
Fig. 6e is an exemplary diagram of a target pseudo-color fusion image obtained by using a fusion detection method of an infrared image and a visible light image based on a visual attention model in a specific experiment in an embodiment of the present invention.
Fig. 6f is an exemplary diagram of target-of-interest information contained in a target pseudo-color fused image obtained by using a fusion detection method of an infrared image and a visible light image based on a visual attention model in a specific experiment in an embodiment of the present invention.
Detailed Description
In order to make the technical scheme and advantages of the invention more apparent, the invention is further described in detail with reference to the accompanying drawings and specific embodiments.
In order to overcome the defects and shortcomings of the image fusion method in the prior art (for example, the influence of high saturation colors on a weak and small target as shown in the above fig. 1 d), the invention provides a pseudo-color visible light and infrared image fusion detection method based on a visual attention model.
Fig. 3 is a schematic flow chart of a fusion detection method of an infrared image and a visible light image based on a visual attention model in an embodiment of the present invention. As shown in fig. 3, the method for detecting fusion of an infrared image and a visible light image based on a visual attention model in the embodiment of the present invention includes the following steps:
and step 31, respectively preprocessing the collected infrared image and the collected visible light image.
According to the technical scheme, the infrared image imaging device and the visible light image imaging device which are subjected to optical registration can be used for collecting the required low signal-to-noise ratio infrared image and the required visible light image respectively, and then the collected infrared image and the collected visible light image are preprocessed respectively, so that adverse effects such as noise and distortion are reduced, and the quality of the image to be fused is improved.
In addition, in the technical scheme of the invention, the collected infrared image and the collected visible light image can be respectively preprocessed by using a plurality of processing modes. For example, the processing method may include: gaussian filtering (for removing noise in the image) and geometric registration (for removing distortion in the image).
And step 32, extracting the interest target from the preprocessed infrared image and visible light image based on the visual attention model of human eyes.
For example, in this step, a hot object (i.e., an interest object) that is likely to attract attention may be detected based on a visual attention mechanism (i.e., a visual attention model) of human eyes, so as to extract a desired interest object from the preprocessed infrared image and visible light image, respectively, so that reliable object information may be obtained.
In addition, in the technical solution of the present invention, the step 32 may be implemented by using various implementation methods. The technical solution of the present invention will be described in detail below by taking one implementation manner as an example.
For example, in a preferred embodiment of the present invention, the step 32 may include the following steps:
For example, in an embodiment of the present invention, the step 321 may include:
establishing an image pyramid (e.g., a wavelet pyramid) from each of the preprocessed infrared images in the image sequence to obtain a plurality of infrared images of different resolutions;
converting and adjusting the infrared images with different resolutions through difference values to obtain a plurality of characteristic difference graphs;
and merging the obtained feature difference graphs through a normalized operator to obtain a final brightness saliency graph of the static image.
For example, in an embodiment of the present invention, the step 322 may include:
obtaining an image motion vector according to an image sequence consisting of a plurality of preprocessed infrared images; the image motion vector includes: intensity differences, spatial consistency differences, and temporal consistency differences;
and generating a motion saliency map according to the intensity difference, the spatial consistency difference and the temporal consistency difference.
And 323, weighting, adding and fusing the brightness saliency map and the motion saliency map according to a preset proportion to obtain a final feature saliency map of the infrared image.
And 324, separating the interest target from the background in the feature saliency map of the infrared image by a local adaptive threshold segmentation method according to the pixel gray level similarity and the adjacency of the centroid of the salient region, and extracting the interest target in the infrared image.
Through the steps 321 to 324, the interest target can be extracted from the preprocessed infrared image based on the visual attention model of human eyes.
The specific implementation manner of this step is the same as or similar to that of step 321, and is not described herein again.
At step 326, a motion saliency map of the visible light image is generated from an image sequence consisting of a plurality of preprocessed visible light images.
The specific implementation of this step is the same as or similar to that of step 322, and will not be described herein again.
And 327, weighting, adding and fusing the brightness saliency map and the motion saliency map according to a preset proportion to obtain a final feature saliency map of the visible light image.
The specific implementation manner of this step is the same as or similar to that of step 323, and is not described herein again.
And 328, separating the interest target from the background in the feature saliency map of the visible light image by a local adaptive threshold segmentation method according to the pixel gray scale similarity and the adjacency of the centroid of the salient region, and extracting the interest target in the visible light image.
The specific implementation of this step is the same as or similar to that of step 324, and is not described herein again.
Through the steps 325-328, the interest target can be extracted from the preprocessed visible light image based on the visual attention model of human eyes.
In addition, the steps 325 to 328 and the steps 321 to 324 may be performed synchronously, or may be performed before or after the steps 321 to 324, which is not limited in the present invention.
And 329, fusing the interest targets in the infrared image and the visible light image by using a preset fusion rule to obtain a final interest target in the target fusion image.
For example, in one embodiment of the present invention, the predetermined fusion rule may be an "or" rule. In the above or rule, the maximum value of the gray scale value of the same pixel in the region where the interest target of the corresponding visible light image and infrared image is located is taken as the gray scale value of the pixel in the target fusion image.
Through the steps 321-329, the interest target can be extracted from the preprocessed infrared image and the preprocessed visible light image based on the visual attention model of human eyes.
And step 33, taking the preprocessed visible light image as a background, and performing gray level image fusion on the corresponding preprocessed infrared image and the visible light image as the background to obtain a gray level fusion image.
In this step, the preprocessed visible light image and the preprocessed infrared image are fused in the wavelet domain according to a preset fusion rule, so as to obtain a gray-scale fusion image. In the fusion operation, the preprocessed visible light image may be used as a background, and the corresponding preprocessed infrared image is fused to the preprocessed visible light image to obtain the grayscale fusion image.
In addition, in the technical solution of the present invention, the step 33 can be implemented by using various implementation methods. The technical solution of the present invention will be described in detail below by taking one implementation manner as an example.
For example, in a preferred embodiment of the present invention, the step 33 may include the following steps:
and 331, selecting a wavelet basis function and the number of decomposition layers, and respectively performing multi-resolution decomposition on the preprocessed visible light image and the preprocessed infrared image to obtain the visible light image and the infrared image of different scale spaces.
For example, in a particularly preferred embodiment of the present invention, the pre-processed visible light image and the pre-processed infrared image may be separately subjected to multi-resolution decomposition by a Discrete Wavelet Transform (DWT) method.
And step 332, extracting image edge features from the low-frequency components of the visible light image and the infrared image of each different scale space.
And 333, performing fusion operation on the visible light images and the infrared images in the spaces with different scales according to a preset fusion rule to obtain multi-resolution expression of the fusion image, and performing wavelet inverse transformation to obtain a gray fusion image.
For example, in an embodiment of the present invention, the preset fusion rule may be a weighted fusion rule that determines the specific gravity of the source image according to indexes such as gradient and information entropy.
Through the steps 331-333, the gray level fusion image can be obtained.
And step 34, carrying out pseudo-color mapping and labeling on the interest target on the gray level fusion image to obtain and output a target pseudo-color fusion image.
For example, in an embodiment of the present invention, the obtained interest target may be inversely mapped to the gray-scale fusion image in HSV space, and the target may be subjected to pseudo-color mapping and labeling to obtain a target pseudo-color fusion image.
Among the numerous color spaces, the HSV color space can reduce the complexity of color image processing and is closer to the human eye's perception and interpretation of color. Therefore, in the above preferred embodiment, the HSV color space can be used for pseudo-color mapping labeling.
For another example, in an embodiment of the present invention, preferably, the pseudo-color mapping labeling the target of interest may include:
marking the interest target extracted according to the source image characteristics in the gray-scale fusion image by using a preset color (for example, red, green and the like).
In the technical scheme of the invention, the fused gray level fusion image is expressed by gray level, and only the interested target is highlighted in the gray level fusion image by the pseudo color which accords with the psychological characteristics of human eyes to form the target pseudo color fusion image, so that the target is more prominent when the fusion image is observed by human eyes, thereby achieving the effect of highlighting the interested target by color without influencing the gray level background of the interested target, further greatly improving the accuracy and reliability of the detection and identification of the interested target and reducing the difficulty of target identification. In addition, because the target pseudo-color fusion image is only subjected to pseudo-color mapping and labeling on the local region where the interested target is located, the visual fatigue of human eyes is not easily caused.
The beneficial technical effects of the invention can be introduced through actual experimental data.
For example, the target pseudo-color fusion image obtained by using the method of the present invention can be compared with fusion images obtained by a wavelet decomposition modulo maximum method (referred to as WMM method) and a wavelet decomposition regional energy method (referred to as WRE method) through practical experiments.
As shown in fig. 6a to 6f, the details of the house and the tree are relatively clear in the visible light image shown in fig. 6a, but the target information is completely blocked by two bunches of smoke in the visible light image. In the long-wave infrared image shown in fig. 6b, the outlines of the house and the trees are relatively clear, and simultaneously, target information such as a portrait (which may be called a target one) and a burning body (which may be called a target two) in the foreground is relatively obvious; however, there is also a background portrait target that is less noticeable because the contrast is too low (which may be referred to as target three). Exemplary diagrams of fused images obtained using the WMM method and the WRE method of the related art are shown in fig. 6c and 6d, respectively. As can be seen in fig. 6c and 6d, object two is more evident in the fused image; the visual perception of the object one is weak; under the influence of smoke in the visible light image, the third object is completely submerged. Therefore, if the above interested target is detected in the fused image obtained by using the WMM method and the WRE method, the first and second targets can be detected, but the third target is likely to be lost.
However, if the fusion detection method of the infrared image and the visible light image based on the visual attention model in the invention is used, before image fusion, an interested target can be extracted from the infrared image and the visible light image through the visual attention model based on human eyes, so that reliable target information can be obtained; then, according to a preset fusion rule, carrying out image fusion operation in a wavelet domain; and finally, mapping the interest target to the gray-scale fusion image through pseudo color to obtain a target pseudo-color fusion image, for example, as shown in fig. 6 e. In the target pseudo-color fusion image, three interested targets are highlighted, so that target information becomes more remarkable and reliable, and the target pseudo-color fusion image has stronger expressive force than the gray-scale fusion image in the prior art.
For example, in the fused images of fig. 6c and 6d, a "ghost" appears at the corner of the left house. This is because the edge of an object in an infrared image is blurred and the energy of the edge point is not sufficiently concentrated. In fig. 6e, the obtained pseudo-color fused image of the target effectively improves the phenomenon. In addition, the target pseudo-color fused image contains the located target information, and the target information is more reliable than the result obtained by directly detecting the target information in the fused image, and the target information can be obtained by simple processing such as color segmentation in post-processing (as shown in fig. 6 f); moreover, the target pseudo-color fusion image effectively retains the target information and the detail information of the source image, and simultaneously makes the whole image more clean and finer, which are the advantages embodied by the fusion rule based on the edge characteristics.
The following table 1 shows the objective evaluation results for three different fused images:
| cross entropy | Degree of structural similarity | Entropy of feature fusion | Image signal-to-noise ratio | |
| WMM method | 3.8798 | 0.7983 | 19.4839 | 28.3713 |
| WRE method | 3.8366 | 0.8032 | 20.2699 | 29.8120 |
| The invention | 3.8470 | 0.8251 | 21.0753 | 32.5567 |
As can be seen from table 1, compared with the WMM method and the WRE method in the prior art, the method for detecting fusion of the infrared image and the visible light image based on the visual attention model in the present invention can greatly improve the accuracy and reliability of detection and identification of the target of interest, and reduce the difficulty of target identification.
In summary, in the technical solution of the present invention, because the interested target may be extracted from the preprocessed infrared image and visible light image based on the visual attention model of human eyes, then the preprocessed visible light image and infrared image are fused to obtain the gray-scale fused image, then the gray-scale fused image is subjected to the pseudo-color mapping and labeling of the interested target, the target extracted according to the source image characteristics is marked with a specific color to obtain and output the target pseudo-color fused image, an effect of highlighting the interested target with color without affecting the gray-scale background of the interested target can be achieved, and the problem that the interested target in the prior art is weakened in the fused image can be solved, the accuracy and reliability of the detection and recognition of the interested target can be greatly improved, and the difficulty of the target recognition can be reduced. In addition, because the target pseudo-color fusion image is only subjected to pseudo-color mapping and labeling on the local region where the interested target is located, the visual fatigue of human eyes is not easily caused. In addition, target detection is carried out on the infrared image before image fusion, and the influence of another source image (namely a visible light image) on the low-contrast target in the fusion process can be effectively avoided. Therefore, the fusion detection method of the infrared image and the visible light image based on the visual attention model provided by the invention can be suitable for detecting and identifying weak and small targets or targets with complex backgrounds in the fusion image, and is more effective particularly for targets with complex backgrounds and low contrast.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (4)
1. A fusion detection method of an infrared image and a visible light image based on a visual attention model is characterized by comprising the following steps:
respectively preprocessing the collected infrared image and the collected visible light image;
extracting an interest target from the preprocessed infrared image and the preprocessed visible light image based on a visual attention model of human eyes;
taking the preprocessed visible light image as a background, and carrying out gray level image fusion on the corresponding preprocessed infrared image and the visible light image as the background to obtain a gray level fusion image;
carrying out pseudo-color mapping and labeling on the interest target on the gray level fusion image to obtain and output a target pseudo-color fusion image;
the method for extracting the interest target from the preprocessed infrared image and visible light image based on the human eye visual attention model comprises the following steps:
generating a brightness saliency map of the static image according to an image sequence consisting of a plurality of preprocessed infrared images;
generating a motion saliency map according to an image sequence consisting of a plurality of preprocessed infrared images;
weighting, adding and fusing the brightness saliency map and the motion saliency map according to a preset proportion to obtain a final feature saliency map of the infrared image;
separating an interest target from a background in a feature saliency map of the infrared image by a local self-adaptive threshold segmentation method according to pixel gray scale similarity and the adjacency of the centroid of the salient region, and extracting the interest target in the infrared image;
generating a brightness saliency map of the static image according to an image sequence consisting of a plurality of preprocessed visible light images;
generating a motion saliency map of the visible light image according to an image sequence consisting of a plurality of preprocessed visible light images;
weighting, adding and fusing the brightness saliency map and the motion saliency map according to a preset proportion to obtain a final feature saliency map of the visible light image;
separating the interest target from the background in the characteristic saliency map of the visible light image by a local adaptive threshold segmentation method according to the pixel gray level similarity and the adjacency of the centroid of the salient region, and extracting the interest target in the visible light image;
fusing interest targets in the infrared image and the visible light image by using a preset fusion rule to obtain a final interest target in a target fusion image;
the generating of the brightness saliency map of the still image from the image sequence consisting of the plurality of preprocessed infrared images comprises:
establishing an image pyramid according to each preprocessed infrared image in the image sequence to obtain a plurality of infrared images with different resolutions;
converting and adjusting the infrared images with different resolutions through difference values to obtain a plurality of characteristic difference graphs;
merging the obtained feature difference graphs through a normalized operator to obtain a final brightness saliency graph of the static image;
the generating of the motion saliency map from an image sequence consisting of a plurality of pre-processed infrared images comprises:
obtaining an image motion vector according to an image sequence consisting of a plurality of preprocessed infrared images; the image motion vector includes: intensity differences, spatial consistency differences, and temporal consistency differences;
generating a motion saliency map according to the intensity difference, the spatial consistency difference and the temporal consistency difference;
the gray level image fusion is performed on the corresponding preprocessed infrared image and the visible light image serving as the background by taking the preprocessed visible light image as the background, and the gray level fusion image is obtained by the method comprising the following steps:
selecting a wavelet basis function and the number of decomposition layers, and respectively performing multi-resolution decomposition on the preprocessed visible light image and the preprocessed infrared image to obtain visible light images and infrared images of different scale spaces;
extracting image edge features from low-frequency components of the visible light image and the infrared image of each different scale space;
performing fusion operation on the visible light images and the infrared images in different scale spaces according to a preset fusion rule to obtain multi-resolution expression of the fusion images, and performing wavelet inverse transformation to obtain gray fusion images;
the performing the pseudo-color mapping and labeling of the interest target on the gray-scale fusion image to obtain and output a target pseudo-color fusion image comprises:
and inversely mapping the obtained interest target to the gray level fusion image in HSV space, and performing pseudo-color mapping and labeling on the interest target to obtain a target pseudo-color fusion image.
2. The method of claim 1, wherein:
the predetermined fusion rule is an or rule;
in the or rule, the maximum value of the gray value of the same pixel in the region where the interest target of the corresponding visible light image and the infrared image is located is used as the gray value of the pixel in the target fusion image.
3. The method of claim 1, wherein:
the preset fusion rule is a weighting fusion rule for determining the proportion of the source image according to the gradient and the information entropy.
4. The method of claim 1, wherein pseudo-color mapping the object of interest comprises:
and marking the interest target extracted according to the source image characteristics in the gray level fusion image by using a preset color.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810007446.5A CN108090888B (en) | 2018-01-04 | 2018-01-04 | Fusion detection method of infrared image and visible light image based on visual attention model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810007446.5A CN108090888B (en) | 2018-01-04 | 2018-01-04 | Fusion detection method of infrared image and visible light image based on visual attention model |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108090888A CN108090888A (en) | 2018-05-29 |
| CN108090888B true CN108090888B (en) | 2020-11-13 |
Family
ID=62179971
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810007446.5A Active CN108090888B (en) | 2018-01-04 | 2018-01-04 | Fusion detection method of infrared image and visible light image based on visual attention model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108090888B (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109146904A (en) * | 2018-08-13 | 2019-01-04 | 合肥英睿系统技术有限公司 | The method and apparatus of infrared image object profile is shown in visible images |
| CN113132640B (en) | 2018-08-27 | 2024-01-09 | 深圳市大疆创新科技有限公司 | Image presentation method, image acquisition device and terminal device |
| CN109470664A (en) * | 2018-09-27 | 2019-03-15 | 中国船舶工业系统工程研究院 | A kind of system and recognition methods of quickly contactless identification sea film |
| US11017515B2 (en) | 2019-02-06 | 2021-05-25 | Goodrich Corporation | Thermal image warm-target detection and outline formation |
| CN110322423B (en) * | 2019-04-29 | 2023-03-31 | 天津大学 | Multi-modal image target detection method based on image fusion |
| CN112308102B (en) * | 2019-08-01 | 2022-05-17 | 北京易真学思教育科技有限公司 | Image similarity calculation method, calculation device, and storage medium |
| CN111191574A (en) * | 2019-12-26 | 2020-05-22 | 新绎健康科技有限公司 | Method and device for acquiring viscera partition temperature of facial examination |
| CN111208521B (en) * | 2020-01-14 | 2021-12-07 | 武汉理工大学 | Multi-beam forward-looking sonar underwater obstacle robust detection method |
| CN111325139B (en) * | 2020-02-18 | 2023-08-04 | 浙江大华技术股份有限公司 | Lip language identification method and device |
| CN111715540B (en) * | 2020-04-29 | 2021-02-02 | 江苏旷博智能技术有限公司 | Apparatus and method for sorting coal and gangue |
| CN111833282B (en) * | 2020-06-11 | 2023-08-04 | 毛雅淇 | Image fusion method based on improved DDcGAN model |
| CN112233024B (en) * | 2020-09-27 | 2023-11-03 | 昆明物理研究所 | Medium-long wave double-band infrared image fusion method based on difference characteristic color mapping |
| CN113409232B (en) * | 2021-06-16 | 2023-11-10 | 吉林大学 | Bionic false color image fusion model and method based on croaker visual imaging |
| CN114419312B (en) * | 2022-03-31 | 2022-07-22 | 南京智谱科技有限公司 | Image processing method and device, computing equipment and computer readable storage medium |
| CN115100193B (en) * | 2022-08-23 | 2022-11-25 | 南京天朗防务科技有限公司 | Weak and small target detection and identification method and device based on infrared and visible light images |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1822046A (en) * | 2006-03-30 | 2006-08-23 | 上海电力学院 | Infrared and visible light image fusion method based on regional property fuzzy |
| CN101872473A (en) * | 2010-06-25 | 2010-10-27 | 清华大学 | Multiscale image natural color fusion method and device based on over-segmentation and optimization |
| US8775597B2 (en) * | 2011-05-26 | 2014-07-08 | Eci Telecom Ltd. | Technique for management of communication networks |
| CN104700381A (en) * | 2015-03-13 | 2015-06-10 | 中国电子科技集团公司第二十八研究所 | Infrared and visible light image fusion method based on salient objects |
-
2018
- 2018-01-04 CN CN201810007446.5A patent/CN108090888B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1822046A (en) * | 2006-03-30 | 2006-08-23 | 上海电力学院 | Infrared and visible light image fusion method based on regional property fuzzy |
| CN101872473A (en) * | 2010-06-25 | 2010-10-27 | 清华大学 | Multiscale image natural color fusion method and device based on over-segmentation and optimization |
| US8775597B2 (en) * | 2011-05-26 | 2014-07-08 | Eci Telecom Ltd. | Technique for management of communication networks |
| CN104700381A (en) * | 2015-03-13 | 2015-06-10 | 中国电子科技集团公司第二十八研究所 | Infrared and visible light image fusion method based on salient objects |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108090888A (en) | 2018-05-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108090888B (en) | Fusion detection method of infrared image and visible light image based on visual attention model | |
| Zhu et al. | A fast single image haze removal algorithm using color attenuation prior | |
| CN111104943B (en) | Color image region-of-interest extraction method based on decision-level fusion | |
| Zhu et al. | Single image dehazing using color attenuation prior. | |
| CN107784669A (en) | A kind of method that hot spot extraction and its barycenter determine | |
| CN114519808A (en) | Image fusion method, device and equipment and storage medium | |
| CN104103082A (en) | Image saliency detection method based on region description and priori knowledge | |
| CN113222877B (en) | Infrared and visible light image fusion method and application thereof in airborne photoelectric video | |
| CN104299196A (en) | Image processing device and method and display device | |
| CN106846289A (en) | A kind of infrared light intensity and polarization image fusion method based on conspicuousness migration with details classification | |
| CN114612359A (en) | Visible light and infrared image fusion method based on feature extraction | |
| Asmuni et al. | An improved multiscale retinex algorithm for motion-blurred iris images to minimize the intra-individual variations | |
| Wang et al. | Retinex algorithm on changing scales for haze removal with depth map | |
| Hassan et al. | Enhancement of under-exposed image for object tracking algorithm through homomorphic filtering and mean histogram matching | |
| CN116452447A (en) | Low-illumination high-definition image processing method | |
| Xiaoxu et al. | Image dehazing base on two-peak channel prior | |
| Asmare et al. | Multi-sensor image enhancement and fusion for vision clarity using contourlet transform | |
| Kaur et al. | Salient infrared target and visible image fusion based on morphological segmentation | |
| Shylaja et al. | Feature extraction using marker based watershed segmentation on the human face | |
| Liu et al. | Adaptive depth map-based retinex for image defogging | |
| CN109190633A (en) | A kind of intelligent object identifying system and control method based on deep learning | |
| Singhai et al. | Image enhancement for low-light and hazy conditions using retinex theory and wavelet transform fusion | |
| Owusu-Agyeman et al. | A concise review on Image Dehazing Techniques | |
| Likhith et al. | Development of data driven adaptive edge detectors for image processing | |
| Wang et al. | The research of pseudo-color infrared and visible light image fusion detection methods of dim and small target based on visual attention |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |