CN113947554B

CN113947554B - Multi-focus image fusion method based on NSST and significant information extraction

Info

Publication number: CN113947554B
Application number: CN202010693743.7A
Authority: CN
Inventors: 何小海; 吴剑; 吴晓红; 王正勇; 卿粼波; 吴小强
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2020-07-17
Filing date: 2020-07-17
Publication date: 2023-07-14
Anticipated expiration: 2040-07-17
Also published as: CN113947554A

Abstract

The invention provides a multi-focus image fusion method based on NSST and significant information extraction. Mainly relates to the multi-focus image fusion problem in the image fusion field. First, the source image is decomposed in multiple dimensions and directions by NSST transformation to obtain high and low frequency subbands. Secondly, improving Laplace energy of a local area is adopted for the low-frequency subband coefficient, and low-frequency subband initial fusion weight is constructed, so that a non-local mean value filtering correction fusion rule is increased for correcting the low-frequency initial fusion weight; the high-frequency subband coefficient is subjected to a fusion rule based on the combination of spatial frequency and energy of the correlation coefficient, and then is corrected by a phase consistency fusion rule, so that a high-frequency subband fusion weight is constructed; finally, the final fusion image is obtained through NSST inverse transformation. Because the corresponding weight correction strategies are respectively carried out on the low-frequency sub-bands and the high-frequency sub-bands, the discrimination error rate of the focusing area is reduced. The validity of this method was verified in experiments with multiple sets of differently focused images.

Description

Multi-focus image fusion method based on NSST and significant information extraction

Technical Field

The invention relates to the multi-focus image fusion problem in the field of image fusion, in particular to a multi-focus image fusion method based on NSST and significant information extraction.

Background

The image fusion technology is a big research hotspot belonging to the field of image processing. Information (brightness, color, space, etc.) contained in one image is limited, and thus it is difficult to satisfy a specific application scene only by one image whose information is limited. The images with different emphasis information are fused together according to a certain rule, so that an image with more information and more convenient observation can be obtained. Obviously, the goal of image fusion is to preserve as much useful information as possible while removing some redundant information. The multi-focus image fusion integrates different focus images in the same scene through a certain fusion method, so that the definition of the fused image is higher, and the contained information is more abundant.

The image fusion field is applied to a plurality of pixel-level fusion methods, including fusion based on a spatial domain, fusion based on a transform domain and fusion based on deep learning. The fusion method based on the transform domain is more applied than the other two methods. The transform domain is to obtain intermediate data with different characteristic information by performing some reversible mathematical transformation on the original data information of the image. And obtaining a fused image after reversible transformation by carrying out corresponding fusion rule processing on the intermediate data. Obviously, in this type of method, reversible transformation and fusion rules become critical. The reversible transforms that occur sequentially are pyramid transforms, wavelet transforms, non-downsampled contourlet transforms (NSCT), non-downsampled shear wave transforms (NSST), and the like. The fusion rules are relatively more, and the fusion rules based on spatial frequency, the fusion rules based on energy information, the fusion rules based on guide filtering and the like are included. From a different perspective, many fusion algorithms have been proposed by students. Most fusion algorithms obtain fusion images with the phenomena of low definition, loss of focusing information, blurred focusing edges and the like.

Disclosure of Invention

The invention provides a multi-focus image fusion method based on NSST and significant information extraction. And performing NSST transformation on different focusing images to obtain high and low frequency sub-band coefficients, and processing by adopting different fusion rules and a certain correction rule to finally obtain a fusion image. The invention mainly realizes the aim through the following process steps:

(1) Processing different focusing images by using NSST transformation to obtain high-low frequency sub-band coefficients;

(2) Performing preliminary treatment on the low-frequency subband coefficient obtained in the step (1) by adopting an initial low-frequency fusion rule of improved Laplace energy Sum (SML) to obtain an initial low-frequency fusion weight;

(3) Performing certain error correction on the result in the step (2) by using a low-frequency correction fusion rule extracted by the saliency information;

(4) Performing preliminary treatment on the high-frequency subband coefficient obtained in the step (1) by adopting an initial high-frequency fusion rule based on a correlation coefficient to obtain an initial high-frequency fusion weight;

(5) A series of high-frequency fusion weights are assisted by Phase Consistency (PC) correction rules with different degrees, and discrimination correction is carried out;

(6) And (3) performing NSST inverse transformation on the processing results obtained in the steps (3) and (5) to obtain a fusion result.

Drawings

FIG. 1 is a multi-focus image fusion frame diagram based on NSST and salient information extraction;

Detailed Description

According to the invention, non-local mean filtering (NLMF) is introduced to combine with Guided Filtering (GF) to carry out weighted correction on low-frequency subband coefficients, spatial frequency and energy based on correlation coefficients are combined to form an initial high-frequency weighted fusion rule, and meanwhile, a phase consistency correction strategy is applied to carry out correction judgment on initial high-frequency fusion weights.

The non-local mean filtering correction fusion rule is as follows:

combining a second low-frequency sub-band correction rule block diagram with the attached drawing;

after NSST transformation is carried out on different focused images, the obtained low-frequency subband image loses detail information, and an initial fusion weight obtained based on an improved Laplace energy Sum (SML) initial fusion rule is used for judging whether an error exists, the detail information of a source image is considered, the source image is filtered by non-local mean value filtering, and difference value operation is carried out on the source image and the filtered image to obtain significance information D _l ，

D _l ＝|I _l -I _l ×NMLF| (1)

(1) Wherein: i _l (0 < L < L) is the source image, L represents the number of the source images, and x represents the filtering operation.

Then the detail information of the focusing area is obtained by utilizing the guide filtering,

G _l ＝guidedfilter(I _l ,D _l ,r,eps) (2)

finally, a big strategy is adopted to obtain the low-frequency sub-band correction fusion weight

And carrying out error correction on the initial fusion weight.

The initial high frequency weighted fusion rule is as follows:

the high-frequency subband image contains detail information of a source image, and the invention combines spatial frequency and energy by utilizing a correlation coefficient to form an initial high-frequency fusion rule.

First, an operation of a correlation coefficient (Corr) is defined:

wherein:

respectively representing the high frequency subband images and the images after mean filtering the high frequency subband images, respectively>

Are respectively->

M x N is the taken image block size.

Then defining a spatial frequency and energy (SF_Eng_Corr) calculation formula based on the correlation coefficient as follows:

wherein: sf_corr and eng_corr represent spatial frequency correlation coefficients and energy correlation coefficients, respectively, and for a focused region and an unfocused region of an image, sf_corr values and eng_corr values of the focused region class tend to be larger than those of the unfocused region. By utilizing the point, the two are combined in a weighted way to form SF_Eng_Corr, and then the initial high-frequency fusion weight is obtained by utilizing a big strategy.

The phase consistency correction strategy is as follows:

the initial high frequency fusion weights derived from the correlation coefficients disregard the correlation of the high frequency subband coefficients themselves. For this purpose, the phase consistency PC is used for correction, the procedure being as follows:

the invention adopts a new active measurement rule NAM to obtain high-frequency correction fusion weight:

wherein: PC, LSCM, LE phase consistency, local sharpness transform and local energy, respectively; alpha, beta and gamma are respectively proportional factors.

For the high-frequency subband coefficient, NAM can integrate local energy information, detail edge information and gradient information carried by the coefficient itself together through a certain proportion, thereby being beneficial to distinguishing a focusing region.

In order to verify the effectiveness of the multi-focus image fusion method based on NSST and significant information extraction, a series of comparison experiments are performed. 3 groups of different focused images with the sizes of 512 pixels multiplied by 512 pixels, 640 pixels multiplied by 480 pixels and 512 pixels are selected in the experiment to carry out fusion experiments, and compared with the existing five common algorithms; and simultaneously, adopting six evaluation indexes to quantitatively evaluate. The comparison pictures used are registered, and the experimental results are shown in Table one:

table one average index result

Tab.1 The result of average index

It can be seen from the table that the method of the invention improves the visual definition of the fusion image to 0.9009 and the structural similarity to 0.9945 on the premise of retaining enough fusion information. In other indexes, the standard deviation STD is slightly reduced, and the rest indexes are improved to a certain extent. The algorithm not only effectively reserves the detailed information such as the contour, the texture and the like of the source image, but also has good visual effect in the focusing edge area of the image. The contrast definition of the fusion image is improved to a certain extent, and the fusion effect is ideal. Therefore, the algorithm herein is a viable multi-focus image fusion method.

Claims

1. The multi-focus image fusion method based on NSST and significant information extraction is characterized by comprising the following steps:

2. The method of claim 1, wherein in step (3), a non-local mean filtering correction fusion rule is added to make the low frequency fusion weight more accurate, and the correction fusion rule is as follows:

filtering the source image by utilizing non-local mean filtering, and performing difference value operation on the source image and the filtered image to obtain significance information D _l ：

D _l ＝|I _l -I _l ×NMLF| (1)

(1) Wherein: i _l (0 < L < L) is a source image, L represents the number of the source images, and x represents the filtering operation;

and then, guiding filtering is utilized to obtain detail information of a focusing area:

G _l ＝guidedfilter(I _l ,D _l ,r,eps) (2)

and finally, obtaining the low-frequency sub-band correction fusion weight by adopting a big strategy, and carrying out error correction on the initial fusion weight.

3. The method of claim 1, wherein the spatial frequency and energy based on the correlation coefficient are extracted by applying the initial high frequency fusion weights in step (4), the extraction process is as follows:

for a focused region and an unfocused region of an image, the spatial frequency correlation coefficient value sf_corr and the energy correlation coefficient value (eng_corr) of the focused region tend to be larger than those of the unfocused region; by utilizing the point, the two are weighted and combined to form spatial frequency and energy (SF_Eng_Corr) based on a correlation coefficient, and then an initial high-frequency fusion weight is obtained by utilizing a big strategy;

wherein: sf_corr and eng_corr represent spatial frequency correlation coefficients and energy correlation coefficients, respectively.

4. The method of claim 1, wherein in step (5), phase consistency fusion correction is performed on the initial high-frequency fusion weight, and the high-frequency detail information, the local energy information and the gradient edge information are integrated together by a certain proportion, so that discrimination of a focusing area is facilitated.

5. The method of claim 1, wherein the initial fusion weights are obtained according to different fusion rules for the high-frequency and low-frequency subband coefficients obtained through NSST transformation, and the initial weights are respectively corrected by using correction fusion rules, so that the information quantity and definition of the fused image are improved to a certain extent.