CN103279923A - Partial image fusion processing method based on overlapped region - Google Patents

Abstract

The invention provides a partial image fusion processing method based on an overlapped region. The overlapped region is registered, located and extracted and the overlapped region is processed correspondingly according to different fusion algorithms. According to the partial image fusion processing method based on the overlapped region, fusion processing of two images with overlapped parts is achieved, the universal fusion limit that a traditional fusion algorithm only can be used for whole images of the same size and complete registering is overcome, and the adaptability and the practicality of the fusion algorithm and relevant software are improved.

Description

Local Image Fusion Processing Method Based on Overlapping Area

technical field

The invention relates to image fusion and its application field, in particular to a partial image fusion processing method based on overlapping regions.

Background technique

   Image fusion is an important branch of multi-sensor data fusion. Image fusion is the comprehensive processing of multiple images obtained by different sensors according to a certain algorithm to obtain a new image that meets certain requirements.

At present, the widely used fusion algorithms mainly include simple fusion algorithm, component replacement algorithm, Brovey algorithm, high-pass filtering HPF (high-pass filtering) fusion algorithm, multi-scale and multi-resolution analysis fusion algorithm, etc. However, most of the existing research and algorithm implementations are carried out under certain prerequisites. These prerequisites include: 1) The original images participating in the fusion must be of the same size and have been fully registered; 2) Other Fusion is global fusion, that is to say, the content contained in the two images must be exactly the same, and the fusion area is the entire image area; 3) For the wavelet fusion algorithm and the Contourlet fusion algorithm, in addition to the above conditions, it is also required to participate in the fusion The image size of must be N*N and N must be an integer power of 2. These preconditions are sometimes inconsistent with real application conditions, thus limiting the applicability of image fusion algorithms and related software.

In practical applications, the images involved in fusion are often of different sizes, without registration (there are often translation, rotation and scale changes between images), and images with only partial overlapping regions. For the fusion problem in this case, the traditional fusion processing method cannot complete the corresponding processing at all, and the existing literature and related materials have not given a positive corresponding solution.

Contents of the invention

The purpose of the present invention is to provide a local image fusion processing method based on overlapping regions. The present invention realizes local image fusion processing of overlapping areas by registering the read-in images, positioning and extracting overlapping areas, performing corresponding processing and fusion on overlapping areas, and seamlessly splicing non-overlapping image areas . The invention can achieve better image processing effect and improve the adaptability and practicability of the fusion algorithm. The key steps are to register, locate and extract the overlapping areas, and to deal with the overlapping areas according to different fusion algorithms.

The technical solution of the present invention is a local image fusion processing method based on overlapping regions, which is characterized in that: comprising the following steps:

Step 101: start the local image fusion processing method based on overlapping regions;

Step 102: Import two images with partially overlapping regions, marked as img1, img2;

Step 103: Select a registration algorithm, obtain relevant parameters and judgment images, and perform corresponding processing;

Step 104: Select a fusion algorithm, and perform positioning, extraction and processing on overlapping regions according to the result of step 103 and the selected fusion algorithm to achieve fusion;

Step 105: seamlessly splicing non-overlapping image regions;

Step 106: End the local image fusion processing method based on overlapping regions.

Described step 103 comprises the following steps:

Step 201: Start selecting a registration algorithm;

Step 202: Extract feature points from the two read-in images respectively, and mark the point sets as P1 and P2 respectively;

Step 203: Use the feature descriptor to perform rough matching point pairs of feature points, the result is marked as Q1, and obtain a schematic diagram of the rough matching result of the image, marked as PZ_CD;

Step 204: Perform RANSAC processing on Q1 to obtain a fine matching point pair, the result is marked as Q2, and a schematic diagram of the rough matching result of the image is obtained, marked as PZ_JD;

Step 205: Perform least squares processing on Q2 to obtain a transformation matrix H for registration of two images, and obtain translation parameters, scaling parameters and rotation parameters;

Step 206: According to the size of the two images read in and the transformation matrix H obtained in step 204, find the size of the smallest format after fusing and splicing the two images, and create six blackboard images PZ1, PZ1_PD of this size , PZ1_CH and PZ2, PZ2_PD, PZ2_CH, and initialized to all 0. At the same time, you can also get the offset when img1 is placed in the image, marked as DX, DY;

Step 207: according to DX, DY and transformation matrix H obtained in step 205, respectively place img1 and img2 in images PZ1, PZ1_PD and PZ2, PZ2_PD, and process PZ1_PD and PZ2_PD;

Step 208: Finish selecting the registration algorithm. the

The step 207 includes the following steps:

Step 301: start to place img1, img2 and process PZ1_PD and PZ2_PD;

Step 302: place img1 in images PZ1 and PZ1_PD respectively according to DX and DY obtained in step 206;

Step 303: Scan PZ1_PD row by row and column by row, set all the values in the corresponding area storing img1 in PZ1_PD to 255, and set all the values in other places to 0;

Step 304: Replace the horizontal offset and vertical offset in the conversion matrix H obtained in step 205 with DX and DY respectively, to obtain a new conversion matrix H_XIN;

Step 305: use the obtained new matrix to process img2, and place the transformed img2 in PZ2 and PZ2_PD respectively;

Step 306: Scan PZ2_PD row by row and column by row, set all the values in the corresponding area storing img2 in PZ2_PD to 255, and set all the values in other places to 0;

Step 307: Finish placing img1 and img2 and processing PZ1_PD and PZ2_PD.

Described step 104 comprises the following steps:

Step 401: start selecting a fusion algorithm;

Step 402: Scan the PZ1 image row by row and column by row. If PZ1_PD and PZ2_PD are 255 at the same time, this position belongs to the overlapping area of images PZ1 and PZ2, and store the pixel value of this position in the corresponding position of image PZ1_CH. Also perform similar processing on the image PZ2, and the result is placed in PZ2_CH;

Step 403: Perform corresponding preprocessing on the images PZ1_CH and PZ2_CH according to the selection of different fusion algorithms, and make them complete the fusion process, and the results are stored in Fusion;

Step 404: End selection of fusion algorithm.

The step 403 includes the following steps:

Step 501: Start to preprocess and fuse images PZ1_CH and PZ2_CH according to different fusion algorithms;

Step 502: Select different fusion methods, if the selected fusion method is not wavelet fusion or Contourlet fusion method, directly proceed to step 505, otherwise proceed to step 503;

Step 503: Determine whether the size of the images PZ1_CH and PZ2_CH is N*N and N is an integer power of 2. If not go to step 504, otherwise go to step 505;

Step 504: trim the size of the images PZ1_CH and PZ2_CH, and the size is the smallest integer power of 2 greater than the maximum value of the length and width of the images of PZ1_CH and PZ2_CH. According to the latest size obtained, generate the blackboard images PZ1_CH_XIN, PZ2_CH_XIN of this size, and the initial value is all 0, and put PZ1_CH, PZ2_CH from the 00 position in PZ1_CH_XIN, PZ2_CH_XIN respectively;

Step 505: Perform fusion processing on the images PZ1_CH, PZ2_CH or PZ1_CH_XIN, PZ2_CH_XIN obtained from step 504 at this time, and the fusion result is marked as Fusion;

Step 506: Judging whether it has undergone size trimming, if step 504 has been passed before, then step 507 is still required, otherwise, go directly to step 508;

Step 507: Process the fused image Fusion, extract only a part of the same size as PZ1_CH from position 00, and assign it to the corrected Fusion with the same size as PZ1_CH;

Step 508: End the preprocessing and fusion of images PZ1_CH and PZ2_CH according to different fusion algorithms.

Described step 105 includes the following steps:

Step 601: Start seamless stitching of non-overlapping image regions;

Step 602: Scan the PZ1_PD image row by row and column by row. When the pixel value in PZ1_PD is equal to 0, replace the pixel value at the same position in Fusion with the pixel value at this position in img2;

Step 603: Scan the PZ2_PD image row by row and column by row. When the pixel value in PZ2_PD is equal to 0, replace the pixel value at the same position in Fusion with the pixel value at this position in img1;

Step 604: End the seamless splicing of non-overlapping image regions.

The advantages of the present invention are: it overcomes the limitation that the traditional fusion algorithm can only be used for the same size, complete registration, and global fusion of the entire image, and realizes: 1) different sizes, no registration (there are often translations between images, Rotation and scale change), fusion processing of images with only partial overlapping areas; 2) wavelet and Contourlet fusion processing in the case of irregular overlapping area images (the size is not N*N and N is not an integer power of 2), reaching Good fusion processing effect; 3) Positioning, extraction and fusion of overlapping areas, seamless splicing of non-overlapping image areas.

The invention breaks through the harsh preconditions of conventional fusion processing methods, reduces the requirements for participating in fusion images, and has good adaptability and practicability.

Description of drawings

Figure 1. The main flow chart of the local image fusion processing method based on overlapping regions;

Figure 2 is a flow chart of selecting a registration algorithm, obtaining relevant parameters and judging images, and performing corresponding processing;

Figure 3 is a flow chart for placing img1, img2 and processing PZ1_PD and PZ2_PD;

Fig. 4 selects the fusion algorithm, according to the result of step 103 and the selected fusion algorithm, the overlapping regions are judged, extracted and processed to realize the flow chart of fusion;

Figure 5 is a flowchart of preprocessing and fusion of images PZ1_CH and PZ2_CH according to different fusion algorithms;

Fig. 6 is a flow chart of seamless stitching of non-overlapping image regions.

Detailed ways

The key steps of the local image fusion processing method based on overlapping areas are registration, positioning, and extraction of overlapping areas, and corresponding processing of overlapping areas for different fusion algorithms. Since the two images have partial overlapping areas, we must first register the images to obtain relevant parameters (translation, scaling parameters, rotation parameters, etc.), and then locate and extract the overlapping areas. However, the extracted overlapping regions may be regular images, or images of any size and shape. In addition, some fusion algorithms have special requirements on the size of the image, so we must perform some processing on the overlapping area images to perform local area fusion.

The feature of the local image fusion processing method based on overlapping regions is: firstly, register the two read-in images with partial overlapping regions and obtain relevant parameters and judgment images, and then perform overlapping region positioning, Extraction and fusion (note here: for some fusion algorithms, images in overlapping areas need to be processed before fusion), and finally non-overlapping image areas are seamlessly spliced by judging images. In this way, the partial image fusion processing of the read-in two images with partially overlapping regions is completed.

As shown in Figure 1.

The main flowchart step features are:

Step 101: start the local image fusion processing method based on overlapping regions;

Step 102: Import two images with partially overlapping regions, marked as img1, img2;

Step 103: Select a registration algorithm, obtain relevant parameters and judgment images, and perform corresponding processing;

Step 104: Select a fusion algorithm, and perform positioning, extraction and processing on overlapping regions according to the result of step 103 and the selected fusion algorithm to achieve fusion;

Step 105: seamlessly splicing non-overlapping image regions;

Step 106: End the local image fusion processing method based on overlapping regions;

as shown in picture 2,

Described step 103 comprises the following steps:

Step 201: Start selecting a registration algorithm;

Step 202: Extract feature points from the two read-in images respectively, and mark the point sets as P1 and P2 respectively;

Step 203: Use the feature descriptor to perform rough matching point pairs of feature points, the result is marked as Q1, and obtain a schematic diagram of the rough matching result of the image, marked as PZ_CD;

Step 204: Perform RANSAC processing on Q1 to obtain a fine matching point pair, the result is marked as Q2, and a schematic diagram of the rough matching result of the image is obtained, marked as PZ_JD;

Step 205: Perform least squares processing on Q2 to obtain a transformation matrix H for registration of two images, and obtain translation parameters, scaling parameters and rotation parameters;

Step 206: According to the size of the two images read in and the transformation matrix H obtained in step 204, find the size of the smallest format after the two images are fused and stitched, and create six blackboard images PZ1, PZ1_PD of this size , PZ1_CH and PZ2, PZ2_PD, PZ2_CH, and initialized to all 0. At the same time, you can also get the offset when img1 is placed in the image, marked as DX, DY;

Step 207: according to DX, DY and transformation matrix H obtained in step 205, respectively place img1 and img2 in images PZ1, PZ1_PD and PZ2, PZ2_PD, and process PZ1_PD and PZ2_PD;

Step 208: end the selection registration algorithm;

As shown in Figure 3,

The step 207 includes the following steps:

Step 301: start to place img1, img2 and process PZ1_PD and PZ2_PD;

Step 302: place img1 in images PZ1 and PZ1_PD respectively according to DX and DY obtained in step 206;

Step 303: Scan PZ1_PD row by row and column by row, set all the values in the corresponding area storing img1 in PZ1_PD to 255, and set all the values in other places to 0;

Step 304: Replace the horizontal offset and vertical offset in the conversion matrix H obtained in step 205 with DX and DY respectively, to obtain a new conversion matrix H_XIN;

Step 305: use the obtained new transformation matrix H_XIN to process img2, and place the transformed img2 in PZ2 and PZ2_PD respectively;

Step 306: Scan PZ2_PD row by row and column by row, set all the values in the corresponding area storing img2 in PZ2_PD to 255, and set all the values in other places to 0;

Step 307: finish placing img1 and img2 and process PZ1_PD and PZ2_PD;

As shown in Figure 4,

Described step 104 comprises the following steps:

Step 401: start selecting a fusion algorithm;

Step 402: Scan the PZ1 image row by row and column by row. If PZ1_PD and PZ2_PD are 255 at the same time, this position belongs to the overlapping area of images PZ1 and PZ2, and store the pixel value of this position in the corresponding position of image PZ1_CH. Also perform similar processing on the image PZ2, and the result is placed in PZ2_CH;

Step 403: Perform corresponding preprocessing on the images PZ1_CH and PZ2_CH according to the selection of different fusion algorithms, and make them complete the fusion process, and the results are stored in Fusion;

Step 404: End selection of the fusion algorithm;

As shown in Figure 5,

The step 403 includes the following steps:

Step 501: Start to preprocess and fuse images PZ1_CH and PZ2_CH according to different fusion algorithms;

Step 502: Select different fusion methods, if the selected fusion method is not wavelet fusion or Contourlet fusion method, directly proceed to step 505, otherwise proceed to step 503;

Step 503: Determine whether the size of the images PZ1_CH and PZ2_CH is N*N and N is an integer power of 2. If not go to step 504, otherwise go to step 505;

Step 504: trim the size of the images PZ1_CH and PZ2_CH, and the size is the smallest integer power of 2 greater than the maximum value of the length and width of the images of PZ1_CH and PZ2_CH. According to the latest size obtained, generate the blackboard images PZ1_CH_XIN, PZ2_CH_XIN of this size, and the initial value is all 0, and put PZ1_CH, PZ2_CH from the 00 position in PZ1_CH_XIN, PZ2_CH_XIN respectively;

Step 505: Perform fusion processing on the images PZ1_CH, PZ2_CH or PZ1_CH_XIN, PZ2_CH_XIN obtained from step 504 at this time, and the fusion result is marked as Fusion;

Step 506: Judging whether it has undergone size trimming, if step 504 has been passed before, then step 507 is still required, otherwise, go directly to step 508;

Step 507: Process the fused image Fusion, extract only a part of the same size as PZ1_CH from position 00, and assign it to the corrected Fusion with the same size as PZ1_CH;

Step 508: End the preprocessing and fusion of images PZ1_CH and PZ2_CH according to different fusion algorithms;

As shown in Figure 6,

The step 105 includes the following steps:

Step 601: Start seamless stitching of non-overlapping image regions;

Step 602: Scan the PZ1_PD image row by row and column by row. When the pixel value in PZ1_PD is equal to 0, replace the pixel value at the same position in Fusion with the pixel value at this position in img2;

Step 603: Scan the PZ2_PD image row by row and column by row. When the pixel value in PZ2_PD is equal to 0, replace the pixel value at the same position in Fusion with the pixel value at this position in img1;

Step 604: End the seamless splicing of non-overlapping image areas;

The parts that are not described in detail in this embodiment belong to well-known common means in this industry, and will not be described here one by one.

Claims (6)

1. based on topography's method for amalgamation processing of overlapping region, it is characterized in that: comprise the steps:

Step 101: beginning is based on topography's method for amalgamation processing of overlapping region;

Step 102: import two width of cloth overlap the zone image, be labeled as img1, img2;

Step 103: select registration Algorithm, obtain correlation parameter and process decision chart picture, and carry out respective handling;

Step 104: select blending algorithm, result and selected blending algorithm according to step 103 position, extract and handle the overlapping region, realize merging;

Step 105: carry out seamless spliced to non-overlapped image-region;

Step 106: finish the topography's method for amalgamation processing based on the overlapping region.

2. the topography's method for amalgamation processing based on the overlapping region according to claim 1, it is characterized in that: described step 103 comprises the steps:

Step 201: begin to select registration Algorithm;

Step 202: respectively two width of cloth images that read in are carried out feature point extraction, its point set is labeled as P1, P2 respectively;

Step 203: the use characteristic descriptor carries out that the thick match point of unique point is right, and result queue is Q1, and obtains the thick matching result synoptic diagram of image, is labeled as PZ_CD;

Step 204: Q1 is carried out RANSAC handle that to obtain smart match point right, result queue is Q2, and obtains the thick matching result synoptic diagram of image, is labeled as PZ_JD;

Step 205: Q2 is carried out the transition matrix H that Least Square in Processing obtains two width of cloth image registrations, obtain translational movement parameter, scaling parameter and rotation parameter;

Step 206: according to the size of read in two width of cloth images and the transition matrix H that in step 204, obtains, obtain the size with the spliced minimum breadth of two width of cloth image co-registration, and create blackboard image PZ1, PZ1_PD, PZ1_CH and PZ2, PZ2_PD, the PZ2_CH that six width of cloth should size, and be initialized as complete 0; Side-play amount in the time of simultaneously can also obtaining being placed on img1 in this image is labeled as DX, DY;

Step 207: according to gained DX in the step 205, DY and transition matrix H are placed on img1, img2 respectively among image PZ1, PZ1_PD and PZ2, the PZ2_PD respectively, and PZ1_PD and PZ2_PD are handled;

Step 208: finish to select registration Algorithm.

3. the topography's method for amalgamation processing based on the overlapping region according to claim 2, it is characterized in that: described step 207 comprises the steps:

Step 301: begin to place img1, img2 and PZ1_PD and PZ2_PD are handled;

Step 302: according to gained DX in the step 206, DY is placed on img1 respectively among image PZ1 and the PZ1_PD;

Step 303: PZ1_PD is scanned line by line, the value of depositing the corresponding region of img1 among the PZ1_PD all is arranged to 255, other local values all are set to 0;

Step 304: use DX respectively, DY goes to replace horizontal offset and the vertical offset among the transition matrix H of gained in step 205, obtains new transition matrix H_XIN;

Step 305: utilize the new matrix of gained that img2 is handled, and the img2 after the conversion is placed on respectively among PZ2 and the PZ2_PD;

Step 306: PZ2_PD is scanned line by line, the value of depositing the corresponding region of img2 among the PZ2_PD all is arranged to 255, other local values all are set to 0;

Step 307: end is placed img1, img2 and PZ1_PD and PZ2_PD is handled.

4. the topography's method for amalgamation processing based on the overlapping region according to claim 1, it is characterized in that: described step 104 comprises the steps:

Step 401: begin to select blending algorithm;

Step 402: the PZ1 image is scanned line by line, if PZ1_PD and PZ2_PD are 255 simultaneously, then this position belongs to the overlapping region of image PZ1 and PZ2, and the pixel value of this position is existed on the correspondence position of image PZ1_CH;

To image PZ2, carry out similar processing equally, its result is placed among the PZ2_CH;

Step 403: according to the different blending algorithm of selection image PZ1_CH and PZ2_CH are carried out corresponding pre-service, and make it finish fusion process, its result exists among the Fusion;

Step 404: finish to select blending algorithm.

5. the topography's method for amalgamation processing based on the overlapping region according to claim 4, it is characterized in that: described step 403 comprises the steps:

Step 501: beginning is carried out pre-service and fusion according to different blending algorithms to image PZ1_CH and PZ2_CH;

Step 502: select different fusion methods, do not merge or the Contourlet fusion method if the fusion method of choosing is not small echo, directly carry out step 505, otherwise carry out step 503;

Step 503: judge that image PZ1_CH, PZ2_CH image size are that N*N and N are 2 integral number power;

If not carry out step 504, otherwise carry out step 505;

Step 504: image PZ1_CH, PZ2_CH size are repaired, and its size is 2 integral number power greater than PZ1_CH, the peaked minimum of PZ2_CH image length and width;

According to resulting up-to-date size, generate this big or small blackboard image PZ1_CH_XIN, PZ2_CH_XIN, and initial value is complete 0, and PZ1_CH, PZ2_CH are placed on PZ1_CH_XIN, the PZ2_CH_XIN from 00 position respectively;

Step 505: image PZ1_CH, PZ2_CH or the PZ1_CH_XIN that obtains from step 504, PZ2_CH_XIN to this moment, carry out fusion treatment, fusion results is labeled as Fusion;

Step 506: judge whether through big deseaming, if pass through step 504 before this, then also need carry out step 507, otherwise directly carry out 508;

Step 507: the image Fusion after merging is handled, only extract a size part identical with the PZ1_CH size since 00 position, assignment is given among the Fusion after big or small identical with PZ1_CH being corrected;

Step 508: finish according to different blending algorithms image PZ1_CH and PZ2_CH to be carried out pre-service and fusion.

6. the topography's method for amalgamation processing based on the overlapping region according to claim 1, it is characterized in that: described step 105 comprises the steps:

Step 601: begin to carry out seamless spliced to non-overlapped image-region;

Step 602: the PZ1_PD image is scanned line by line, when pixel value equals 0 among the PZ1_PD, remove to replace the pixel value of same position among the Fusion with the pixel value of this position among the img2;

Step 603: the PZ2_PD image is scanned line by line, when pixel value equals 0 among the PZ2_PD, remove to replace the pixel value of same position among the Fusion with the pixel value of this position among the img1;

Step 604: finish to carry out seamless spliced to non-overlapped image-region.

Images