CN113298823B

CN113298823B - Image fusion method and device

Info

Publication number: CN113298823B
Application number: CN202110552484.0A
Authority: CN
Inventors: 刘金伟; 吴鹏志
Original assignee: Xi'an Ruisi Shuzhi Technology Co ltd
Current assignee: Xi'an Ruisi Shuzhi Technology Co ltd
Priority date: 2021-05-20
Filing date: 2021-05-20
Publication date: 2024-03-15
Anticipated expiration: 2041-05-20
Also published as: CN113298823A

Abstract

The embodiment of the application provides an image fusion method and device, wherein the method comprises the following steps: obtaining N images, wherein the N images are obtained by scaling the same original image, the scaling ratios of the N images are different, and N is a numerical value larger than 1; dividing each image by taking the preset pixel number as a dividing window to obtain a divided image block aiming at each image; and carrying out image fusion based on the segmented image blocks to obtain fused image blocks. According to the embodiment of the application, the local characteristic information and the global characteristic information can be fused together, so that more information can be input to the neural network at one time, and the interpretation effect of the neural network is improved.

Description

Image fusion method and device

Technical Field

The application belongs to the technical field of image processing, and particularly relates to an image fusion method and device.

Background

In remote sensing images, larger-sized images are very common, and in addition, the pixels of the images of rivers, lakes, trees, etc. that we want to interpret are also larger. The pixel size of the image is generally larger than the input field of view (e.g., 400×400 pixels) of the neural network, and the image is generally reduced to 400×400 pixels, which results in losing a large amount of useful information, resulting in poor interpretation of the neural network.

Disclosure of Invention

The embodiment of the application aims to provide an image fusion method and device for solving the problem that an existing neural network is poor in interpretation effect.

In a first aspect, an embodiment of the present application provides an image fusion method, including:

obtaining N images, wherein the N images are obtained by scaling the same original image, the scaling ratios of the N images are different, and N is a numerical value larger than 1;

dividing each image by taking the preset pixel number as a dividing window to obtain a divided image block aiming at each image;

and carrying out image fusion based on the segmented image blocks to obtain fused image blocks.

Optionally, the image fusion is performed based on the segmented tiles to obtain fused tiles, which includes at least one of the following:

fusing the segmented image blocks of different images in the N images to obtain the fused image block;

fusing different segmentation blocks of the same image in the N images to obtain the fused block;

and fusing the segmented blocks of different images and the segmented blocks of the same image in the N images to obtain the fused block.

Optionally, fusing the segmented tiles of different images in the N images to obtain the fused tile, including:

acquiring a corresponding relation table between segmented image blocks of at least two images in the N images;

and fusing the segmentation blocks with the corresponding relation in the at least two images based on the corresponding relation table.

Optionally, the obtaining a correspondence table between segmented tiles of at least two images in the N images includes:

acquiring block indexes of segmented blocks of each image in the at least two images under a two-dimensional coordinate axis, wherein initial indexes of the block indexes corresponding to the at least two images are the same;

acquiring a second block index corresponding to a first block index in the first image in the second image based on the scaling of the N images; the proportional relationship between the second tile index and the first tile index is a proportional relationship between the second image and the first image, and the first image and the second image are any images in the at least two images.

And acquiring a first segmentation block corresponding to the first block index and a second segmentation block corresponding to the second block index, and establishing a corresponding relation between the first segmentation block and the second segmentation block.

Optionally, fusing different segmented tiles of the same image in the N images to obtain the fused tile includes:

acquiring at least two segmented tiles in a target image, wherein the at least two segmented tiles have a preset position relationship in the image, and the target image is any image in N images;

and fusing the at least two segmentation blocks to obtain the fused block.

Optionally, the fusing the segmented tiles of different images and the segmented tiles of the same image in the N images to obtain the fused tile includes:

fusing at least two segmentation blocks with a preset position relationship in the target image to obtain a semi-fused block;

obtaining a segmented block with a corresponding relation with a target segmented block in the target image in at least one image except the target image of the N images; the target split tile is one of the at least two split tiles;

and fusing the segmented image blocks with the corresponding relation with the target segmented image blocks with the semi-fused image blocks to obtain the fused image blocks.

Optionally, the image fusion is performed based on the segmented image blocks to obtain a fused image block, which includes any one of the following:

fusing the segmented image blocks along the RGB channel direction to obtain the fused image blocks;

weighting each divided block based on a preset weight value for each image, and fusing the weighted divided blocks along the RGB channel direction to obtain the fused block;

and weighting and adding the segmented blocks based on a preset weight value for each image to obtain the fused block.

Optionally, the preset pixel number is smaller than or equal to the input pixel number corresponding to the input field of view of the preset neural network.

Optionally, the image fusion is performed based on the segmented image blocks to obtain fused image blocks, including:

training the preset neural network through the fused blocks to obtain a trained neural network model.

In a second aspect, an embodiment of the present application provides an image fusion apparatus, including:

the first acquisition module is used for acquiring N images, wherein the N images are images obtained by scaling the same original image, the scaling ratios of the N images are different, and N is a numerical value larger than 1;

the second acquisition module is used for dividing each image by taking the preset pixel number as a dividing window to obtain a divided image block aiming at each image;

and the third acquisition module is used for carrying out image fusion based on the segmented image blocks to obtain fused image blocks.

According to the image fusion method and device provided by the application, N images are obtained by scaling the same original image, scaling proportions of the N images are different, then the preset pixel number is used as a segmentation window, each image is segmented to obtain segmented image blocks aiming at each image, and finally image fusion is carried out based on the segmented image blocks to obtain fused image blocks; the scaling ratio based on N images is different, so that when the images are segmented based on the same preset pixel number as a segmentation window, the image information of the segmented image blocks segmented by each image is different, the segmented image blocks in the image with reduced scale can contain more global image information, when the images are fused based on the segmented image blocks, different image information can be fused together, when the fused image blocks are used for interpretation, surrounding environment information can be perceived, the problem that only the segmented image is used for reducing the image pixels, and error interpretation results are generated according to local information is avoided, and the interpretation effect is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present application, and that other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.

Fig. 1 is a schematic flow chart of an image fusion method in an embodiment of the present application;

FIG. 2 is one of the image schematic diagrams in the embodiment of the present application;

FIG. 3 is a second image diagram of an embodiment of the present application;

FIG. 4 is a third image diagram of an embodiment of the present application;

FIG. 5 is a block diagram illustrating a block position according to an embodiment of the present disclosure;

FIG. 6 is a second diagram illustrating the locations of the divided tiles according to the embodiment of the present application;

FIG. 7 is a third diagram illustrating the position of a segment in accordance with the embodiments of the present invention;

FIG. 8 is a fourth image schematic diagram in an embodiment of the present application;

FIG. 9 is a plot of a miou trend in an embodiment of the present application;

FIG. 10 is a graph of loss trend in an embodiment of the present application;

FIG. 11 is a schematic diagram of module components of an image fusion apparatus according to an embodiment of the present application;

fig. 12 is a schematic structural diagram of an electronic device in an embodiment of the present application.

Detailed Description

Technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application are within the scope of the protection of the present application.

Specifically, in the prior art, when an image is interpreted by using a neural network model, if the image is slightly larger than the input field of view of the neural network, for example, the input field of view of the neural network is 400×400 pixels, and the image size is 1000×1000 pixels, the image is typically reduced to 400×400 pixels, so that the image can be input into the neural network model. However, this approach may lose a large amount of useful information, thereby reducing the interpretation effect.

Based on this, the image can be segmented into tiles that fit the input field of view, but image segmentation suffers from the problem of "blind-man-like", i.e. interpreting the target by means of local information, often leading to erroneous conclusions due to limited information.

In view of this, the present application provides an image fusion method to fuse local features and more global features in one tile, so as to avoid the problem of poor interpretation effect when interpreting the target only by local information, and improve the accuracy of interpretation.

As shown in fig. 1, the method for image fusion according to the embodiment of the present application includes the following steps:

step 101: n images are acquired.

The N images are obtained by scaling the same original image, the scaling ratios of the N images are different, and N is a numerical value larger than 1.

That is, an image pyramid of N layers may be constructed first in the present embodiment.

An image pyramid is one of the multi-scale representations of an image, an efficient but conceptually simple structure that interprets images in multiple resolutions. A pyramid of one image is a series of image sets that are arranged in a pyramid shape with progressively lower resolution and that are derived from the same original image.

It should be noted that, the value of N and the scaling of each image may be set according to actual requirements, which is not limited herein.

For example, as shown in fig. 2-4, the value of N may be 3 and the scaling may be 1, 0.5, and 0.25 in that order. The scaling ratio of the image shown in fig. 2 is 1, that is, the image shown in fig. 2 is an original image; the image shown in fig. 3 has a scaling ratio of 0.5, that is, the image shown in fig. 3 is an image obtained by 1/2 times of the image shown in fig. 2; the image shown in fig. 4 has a scaling factor of 0.25, i.e., the image shown in fig. 4 is an image obtained by 1/4 magnification of the image shown in fig. 3.

Step 102: and dividing each image by taking the preset pixel number as a dividing window to obtain a divided image block aiming at each image.

Specifically, in this embodiment, a dividing window may be preset, where the size of the dividing window is a preset number of pixels, and then each image is divided by the dividing window to obtain a divided block for each image.

That is, in each image, the size of one divided tile is the size of one divided window.

The image is segmented according to the segmentation window until the maximum number of segments can be segmented, i.e. the number of segmented tiles contained in each image is not limited here.

For example, assuming that the number of pixels of an image is 2000×2000 for each image in fig. 2 to 4, if the preset number of pixels is 500×500, the images in fig. 2, 3, and 4 are divided by using the 500×500 number of pixels as a division window until the maximum number of blocks that can be cut, respectively, to obtain divided tiles for each image.

In this way, the same preset pixel number is used as a segmentation window, and the image information contained in each segmentation block in different images is different due to different scaling of the N images. At this time, it can be obtained that a segmented image block in the image shown in fig. 2 can obtain local information; one segmented block in the image shown in fig. 3 can obtain more global information than one segmented block in fig. 2; a segmented tile in the image of fig. 4 can yield more global information.

In addition, the preset number of pixels may be less than or equal to the number of input pixels corresponding to the input field of view of the preset neural network.

In this way, when dividing each image by taking the preset pixel number as a dividing window to obtain divided blocks for each image, each divided block can be input into a preset neural network.

Step 103: and (5) carrying out image fusion based on the segmented image blocks to obtain fused image blocks.

After obtaining the segmented blocks corresponding to each image, image fusion can be performed based on the segmented blocks to obtain fused blocks.

Therefore, as the images with different scaling ratios are segmented based on the same preset pixel number as the segmentation window, the image information of the segmented image blocks segmented by each image is different, and the segmented image blocks in the image with reduced scale can contain more global image information, so that when the images are fused based on the segmented image blocks, different image information can be fused together, the surrounding environment information can be perceived when the fused image blocks are interpreted, the problem that the image pixels are only segmented to reduce the image pixels and the error interpretation result is generated according to the local information is avoided, and the interpretation effect is improved.

Optionally, in this embodiment, the preset pixel number is less than or equal to the input pixel number corresponding to the input field of view of the preset neural network.

In addition, after the image fusion is carried out based on the segmented image blocks to obtain fused image blocks, the preset neural network can be trained through the fused image blocks to obtain a trained neural network model.

Since the input field of view of the neural network is smaller than the actual size of the image, this results in the neural network being able to process only a portion of the image at a time, while the neural network is less effective in interpretation due to the existence of the "blind image" problem. The segmented image blocks are fused to obtain the fused image blocks, and then the fused image blocks are used for training the preset neural network, and as the fused image blocks contain more image information, more image information can be input into the neural network at one time, the blind man image problem is reduced, and the interpretation effect of the neural network is improved; in addition, the image fusion method and device realize fusion of the images before the images are input into the neural network, and do not need to fuse the feature images in the neural network, so that the convenience of image fusion is improved.

Of course, after the neural network model is obtained, in practical application, the image to be interpreted may be fused by the image fusion method in this embodiment, and the fused image block is sent to the neural network model for interpretation, so as to obtain an interpretation result, so as to ensure accuracy of the interpretation result.

Optionally, in this embodiment, when performing image fusion based on the segmented tiles to obtain the fused tiles, at least one of the following manners may be included:

the method comprises the following steps: and fusing the segmentation blocks of different images in the N images to obtain the fused block.

In this way, the segmented tiles of different images in the N images may be fused to obtain a fused tile.

Specifically, the different image of the N images may be at least two images of the N images, and may be any image.

The fusing the segmented blocks of different images in the N images to obtain the fused block may include the following steps:

and fusing the segmented image blocks with the corresponding relation in the at least two images based on the corresponding relation table to obtain fused image blocks.

That is, a correspondence table between the divided tiles of at least two images may be established, alternatively, a correspondence table between the divided tiles of all the images may be established; and then, fusing the segmented image blocks with the corresponding relation in at least two images based on the established corresponding relation table to obtain fused image blocks.

Optionally, when acquiring the correspondence table between the segmented tiles of at least two images in the N images, the method may include the following steps:

acquiring block indexes of segmented blocks of each image in the at least two images under a two-dimensional coordinate axis, wherein initial block indexes corresponding to the at least two images are the same;

Specifically, the starting tile index corresponding to at least two images may be 0.

Of course, since the unit of tile is one, the tile index is an integer.

Because there is a scaling between the N images and all the images are located in the same two-dimensional coordinate axis, the image information of the divided tiles corresponding to the same tile index is different. In addition, the tile indexes of the divided tiles having the same image information between different images also have a proportional relationship, and in this case, the present embodiment may consider that the tile indexes having a proportional relationship have a correspondence relationship between the images.

Then, the divided tiles corresponding to the tile indexes with the corresponding relationship can be obtained, and the corresponding relationship is established.

Furthermore, it should be noted that one divided tile in the first image may correspond to a plurality of divided tiles in the second image, and of course, it may also be that a plurality of divided tiles in the first image correspond to one divided tile in the second image, which is determined by a proportional relationship between the first image and the second image. For example, if the reduction ratio of the first image is large, one of the divided blocks in the first image corresponds to a plurality of divided blocks in the second image, whereas if the reduction ratio of the second image is large, a plurality of divided blocks in the first image corresponds to one of the divided blocks in the second image. For example, assuming that the ratio of the first image to the second image is 1:2, and that a tile index in the first image is (1, 1), and that the tile index in the corresponding second image is (2, 2), a correspondence between the split tile corresponding to the tile index (2, 2) and the split tile corresponding to the tile index (1, 1) may be established; in addition, assuming that a tile index in the second image is (1, 1), the tile index corresponding to the first image is (1/2 ), and at this time, since the tile with the tile index of (1/2 ) is the tile with the tile index of (1, 1), a corresponding relationship between the segmented tile corresponding to the tile index (1, 1) in the second image and the segmented tile corresponding to the tile index (1, 1) in the first image is also established. Based on this, that is, one divided tile in the first image corresponds to four divided tiles in the second image.

The above is illustrated by the following figures 2 to 4.

If the tile index of the image in fig. 2 is (i, j), the tile index corresponding to the tile index (i, j) in fig. 3 is (i/2, j/2), and the tile index corresponding to the tile index (i, j) in fig. 4 is (i/4,j/4), then the corresponding relationship between the segmented tile corresponding to the tile index (i, j) in fig. 2, the segmented tile corresponding to the tile index (i/2, j/2) in fig. 3, and the segmented tile corresponding to the tile index (i/4,j/4) in fig. 4 may be established, so as to perform image fusion, and obtain the fused tile.

In this way, the segmented image blocks among different images are fused, and the segmented image blocks in the image based on the reduced scale have more global image information, so that the fused image blocks also have more image information, and the interpretation effect when the fused image blocks are interpreted is improved.

The two modes are as follows: and fusing different segmentation blocks of the same image in the N images to obtain the fused block.

In this way, different segmentation blocks in the same image can be fused to obtain a fused image.

Wherein, when fusing different segmentation blocks of the same image in the N images to obtain the fused block, the method may include the following steps:

acquiring at least two segmented tiles in a target image, wherein the at least two segmented tiles have a preset position relationship in the target image, and the target image is any image in N images;

and fusing the at least two segmentation blocks to obtain the fused block.

Specifically, the preset positional relationship may be a diagonal positional relationship, a cross positional relationship, an annular positional relationship, or the like, which is not particularly limited herein.

At least two segmentation blocks with preset position relations are fused, so that when the blocks are interpreted after fusion, surrounding environment information can be perceived, and the interpretation effect is improved.

As an example, a plurality of divided tiles having a diagonal position relationship may be shown in fig. 5, in which fig. 5, an image is divided, and a small square represents one divided tile, and the embodiment may combine the plurality of divided tiles (divided tiles A, B, C, D and E) having a diagonal position relationship in fig. 5; as shown in fig. 6, an image is segmented, and a small square represents one segmented block, and in this embodiment, the segmented blocks (segmented blocks A, B, C, D and E) having the cross-shaped positional relationship in fig. 6 may be fused; as shown in fig. 7, the image is divided, and one small square represents one divided block, and in this embodiment, the divided blocks (divided blocks A, B, C, D and E) having the annular position relationship in fig. 7 can be fused.

In addition, because different segmentation blocks in the same image have different image information, when the different segmentation blocks in the same image are fused, the fused blocks can obtain more global information, so that the interpretation effect when the fused blocks are interpreted is improved.

The three modes are as follows: and fusing the segmented blocks of different images and the segmented blocks of the same image in the N images to obtain the fused block.

In this way, one mode and two modes thereof can be combined, that is, the divided blocks of different images and the divided blocks of the same image are fused, so as to obtain a fused block.

Optionally, when fusing the segmented tiles of different images and the segmented tiles of the same image in the N images to obtain the fused tiles, the method may include the following steps:

Specifically, any image in the N images is used as a target image, and at least two segmentation blocks with a preset position relationship in the target image are fused by utilizing the two modes to obtain a semi-fusion block; then, by utilizing one mode, the segmented image blocks with corresponding relation in other images are obtained, and the obtained segmented image blocks are fused with the semi-fusion image blocks to obtain fused image blocks.

For example, as an example, the split tile a and the split tile B in fig. 2 may be fused to obtain a semi-fused tile, and based on that the split tile B has a corresponding relationship with the split tile C in fig. 3, the semi-fused tile may be fused with the split tile C in fig. 3 to obtain a fused tile.

In this way, the embodiment of the application can realize fusion of the segmentation blocks in any mode, so that more global information is obtained.

Furthermore, optionally, in this embodiment, image fusion is performed based on the segmented tiles to obtain fused tiles, which includes any one of the following modes:

the method comprises the following steps: and fusing the segmented image blocks along the RGB channel direction to obtain the fused image blocks.

The RGB image has three channels R, G and B, and in this way, the split tiles can be fused along the RGB channel direction, so as to obtain a fused tile.

The two modes are as follows: and carrying out weighting operation on each divided block based on a preset weight value aiming at each image, and fusing the weighted divided blocks along the RGB channel direction to obtain the fused block.

In this manner, a corresponding weight value may be set for each of the N images, then, when image fusion is performed, weighting operation is performed on the segmented tiles to be fused based on the weight values, and then, the weighted segmented tiles are fused along the RGB channel direction, so as to obtain a fused tile.

It should be noted that, the weight value corresponding to each image is a constant between [0,1 ].

The three modes are as follows: and weighting and adding the segmented blocks based on a preset weight value for each image to obtain the fused block.

In this manner, a corresponding weight value may be set for each of the N images, and then, when the images are fused, the split tiles to be fused are weighted based on the weight values and then added, and the addition operation is performed, so that the obtained image is still a 3-channel image, but the manner does not require the neural network to perform adaptation.

In this way, fusion among the partitioned tiles is achieved in any mode, so that the fused tiles contain more global information.

The embodiments of the present application are illustrated below by way of illustrative examples.

As shown in fig. 8, a remote sensing image is obtained, wherein the image includes mountain land, woodland, river channel, etc., the resolution is 2 meters, the color depth is 16, and the size is 12547 x 9315. And constructing two-scale image pyramids for the image, namely an original image and an image scaled by 0.5 times, respectively, and comparing the results of network training by using the original image and network training by using the fused image blocks to obtain an average intersection ratio (moiou) trend chart and a loss (loss) trend chart respectively.

Referring to fig. 9, in fig. 9, c3_miou represents a miou trend curve corresponding to the training of the original image, and c6_miou represents a miou trend curve corresponding to the training of the fused image block; referring to fig. 10, c3_loss represents the loss trend curve corresponding to the training of the original image, and c6_loss represents the loss trend curve corresponding to the training of the fused block. As can be seen from fig. 9 and 10, the miou of the fused tile is 21% higher than the original image, and the loss convergence is also better.

It should be noted that, in the image fusion method provided in the embodiment of the present application, the execution subject may be an image fusion device, or a control module in the image fusion device for executing the image fusion method. In the embodiment of the present application, an image fusion device is described by taking an example of an image fusion method performed by the image fusion device.

As shown in fig. 11, the image fusion apparatus includes:

a first obtaining module 1101, configured to obtain N images, where the N images are all images obtained by scaling the same original image, scaling ratios of the N images are different, and N is a numerical value greater than 1;

the second obtaining module 1102 is configured to divide each image with a preset number of pixels as a division window, so as to obtain a divided block for each image;

the third obtaining module 1103 is configured to perform image fusion based on the segmented image blocks, so as to obtain a fused image block.

The device acquires N images through a first acquisition module 1101, wherein the N images are images obtained by scaling the same original image, the scaling ratios of the N images are different, N is a numerical value larger than 1, for each image, the second acquisition module 1102 is used for dividing each image by taking the preset pixel number as a dividing window to obtain divided blocks for each image, and finally the third acquisition module 803 is used for carrying out image fusion based on the divided blocks to obtain fused blocks; the scaling ratio based on N images is different, so that when the images are segmented based on the same preset pixel number as a segmentation window, the image information of the segmented image blocks segmented by each image is different, the segmented image blocks in the image with reduced scale can contain more global image information, when the images are fused based on the segmented image blocks, different image information can be fused together, when the fused image blocks are used for interpretation, surrounding environment information can be perceived, the problem that only the segmented image is used for reducing the image pixels, and error interpretation results are generated according to local information is avoided, and the interpretation effect is improved.

It should be noted that, in order to avoid repetition, the image fusion device provided in the above embodiment can implement all the method steps and beneficial effects of the above image fusion method embodiment, and in this embodiment, the same method steps and beneficial effects as those in the above method embodiment are not repeated.

The embodiment of the present application further provides an electronic device, which is configured to execute the image fusion method according to the embodiment of the present application, based on the same technical concept, and fig. 12 is a schematic structural diagram of an electronic device for implementing each embodiment of the present application. The electronic devices may be configured or configured with a relatively large difference, and may include a processor (processor) 1210, a communication interface (Communications Interface) 1220, a memory (memory) 1230, and a communication bus 1240, where the processor 1210, the communication interface 1220, and the memory 1230 perform communication with each other through the communication bus 1240. Processor 1210 may invoke computer programs stored in memory 1230 and executable on processor 1210 to perform the steps of:

and fusing the segmented image blocks with the corresponding relation in the at least two images based on the corresponding relation table to obtain the fused image blocks.

and fusing the at least two segmentation blocks to obtain the fused block.

Optionally, after the image fusion is performed based on the segmented image blocks to obtain the fused image blocks, the method further includes:

The embodiment of the present application further provides a readable storage medium, on which a program or an instruction is stored, where the program or the instruction implements each process of the above embodiment of the image fusion method when executed by a processor, and the same technical effects can be achieved, so that repetition is avoided, and no further description is given here.

Wherein the processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium such as a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to performing the functions in the order shown or discussed, but may also include performing the functions in a substantially simultaneous manner or in an opposite order depending on the functions involved, e.g., the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solutions of the present application may be embodied essentially or in a part contributing to the prior art in the form of a computer software product stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk), comprising several instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the methods described in the embodiments of the present application.

The embodiments of the present application have been described above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, which are merely illustrative and not restrictive, and many forms may be made by those of ordinary skill in the art without departing from the spirit of the present application and the scope of the claims, which are also within the protection of the present application.

Claims

1. An image fusion method, comprising:

image fusion is carried out based on the segmented image blocks, so that fused image blocks are obtained;

the image fusion is performed based on the segmented image blocks to obtain fused image blocks, which comprise:

fusing the segmented blocks of different images in the N images to obtain the fused block, wherein the fusing block comprises the following steps:

based on the corresponding relation table, fusing the segmented image blocks with the corresponding relation in the at least two images to obtain the fused image blocks;

the obtaining a correspondence table between the segmented tiles of at least two images in the N images includes:

acquiring a second block index corresponding to a first block index in the first image in the second image based on the scaling of the N images; the proportional relation between the second block index and the first block index is the proportional relation between the second image and the first image, and the first image and the second image are any images in the at least two images;

2. The image fusion method according to claim 1, wherein the image fusion is performed based on the segmented tiles to obtain fused tiles, which comprises any one of the following:

3. The image fusion method of claim 1, wherein the preset number of pixels is less than or equal to the number of input pixels corresponding to the input field of view of the preset neural network.

4. The method of image fusion according to claim 3, wherein after the image fusion based on the segmented tiles, the method further comprises:

5. An image fusion apparatus, comprising:

the third acquisition module is used for carrying out image fusion based on the segmented image blocks to obtain fused image blocks;