CN113962844A

CN113962844A - Image fusion method, storage medium and terminal device

Info

Publication number: CN113962844A
Application number: CN202010697989.1A
Authority: CN
Inventors: 郑加章
Original assignee: Wuhan TCL Group Industrial Research Institute Co Ltd
Current assignee: Wuhan TCL Group Industrial Research Institute Co Ltd
Priority date: 2020-07-20
Filing date: 2020-07-20
Publication date: 2022-01-21

Abstract

The application discloses an image fusion method, a storage medium and a terminal device, wherein the method divides a plurality of acquired image frames into a first image group and a second image group based on exposure; respectively determining a first weight map corresponding to the first image group and a second weight map corresponding to the second image group; and finally, fusing the image frames to obtain an output image based on the first weight map and the second weight map which are obtained through determination. Thus, a first weight map corresponding to the first image group and a second weight map corresponding to the second image group are determined, and a plurality of image frames are fused according to the fusion weight maps corresponding to the first weight map and the second weight map, so that the image frames with different exposure levels correspond to different weight maps, and the respective effects are improved through the weight maps corresponding to the image frames with different exposure levels, thereby improving the image effect of the output image obtained by fusion.

Description

Image fusion method, storage medium and terminal device

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image fusion method, a storage medium, and a terminal device.

Background

At present, in order to obtain a clear and natural image at highlight and shadow, the image is generally determined by a multi-exposure fusion technology. The multi-exposure fusion technique is to generally acquire multi-frame multi-exposure images and fuse the multi-frame multi-exposure images to obtain an output image. However, the conventional multi-exposure fusion technology has the problem of poor image effect of fused output images.

Disclosure of Invention

The technical problem to be solved by the present application is to provide an image fusion method, a storage medium, and a terminal device, aiming at the deficiencies of the prior art.

In order to solve the above technical problem, a first aspect of an embodiment of the present application provides an image fusion method, where the method includes:

acquiring a plurality of image frames;

dividing a number of image frames into a first image group and a second image group based on exposure;

determining a first weight map corresponding to the first image group and a second weight map corresponding to the second image group;

and fusing the image frames to obtain an output image based on the determined first weight map and the second weight map.

According to the image fusion method, the similarity of the image content of each image frame in the image frames meets a preset condition.

The image fusion method comprises the steps that a plurality of image frames are original image data, and for each first image in a first image group, the exposure amount corresponding to the first image is smaller than or equal to a preset exposure amount; for each of the second images, the exposure amount corresponding to the second image is greater than the preset exposure amount.

The image fusion method, wherein the determining of the first weight map corresponding to the first image group specifically includes:

for each first image in the first image group, determining a first gray-scale image corresponding to the first image, and determining a first pixel point weight corresponding to each pixel point in the first image based on the first gray-scale image to obtain a first weight matrix corresponding to the first image;

and determining a first weight map corresponding to the first image group based on a first weight matrix corresponding to all the first images in the first image group respectively.

The image fusion method includes, for each first image in the first image group, determining a first grayscale map corresponding to the first image, and determining a first pixel weight corresponding to each pixel in the first image based on the first grayscale map, so as to obtain a first weight matrix corresponding to the first image:

for each first image in the first image group, determining a first gray scale map corresponding to the first image;

determining a first distance between each pixel point in the first gray scale image and a first preset expected value based on a preset first Gaussian curve;

and taking the determined first distance corresponding to each pixel point as a first pixel point weight corresponding to each pixel point to obtain a first weight matrix corresponding to the first image.

The image fusion method, wherein after determining a first gray scale map corresponding to each first image in the first image group, the method further comprises:

and normalizing the first gray-scale image, and taking the normalized first gray-scale image as a first gray-scale image corresponding to the first image.

The image fusion method, wherein the determining of the second weight map corresponding to the second image group specifically includes:

for each second image in the second image group, determining a second gray scale map corresponding to the second image; determining a mean matrix corresponding to the second image based on the second gray scale map, and determining a second weight matrix corresponding to the second image based on the mean matrix and the second gray scale map;

and determining a second weight map corresponding to the second image group based on a second weight matrix corresponding to all the second images in the second image group respectively.

The image fusion method, wherein the determining a second weight matrix corresponding to the second image based on the mean matrix and the second grayscale map specifically includes:

determining a second distance between each pixel point in the mean value matrix and a second expected value based on a preset second Gaussian curve, and taking each determined second distance as a second pixel point weight corresponding to each pixel point to obtain a first temporary weight matrix;

determining a second temporary weight matrix corresponding to the second image based on a preset third Gaussian curve and the second gray-scale map;

and determining a second weight matrix corresponding to the second image based on the first temporary weight matrix and the second temporary weight matrix.

The image fusion method, wherein the determining a second temporary weight matrix corresponding to the second image based on a preset third gaussian curve and the second gray scale map specifically includes:

determining a guide filter map corresponding to the second image based on the second gray map;

and determining a third distance between each pixel point in the guide filter graph and a third expected value based on a preset third Gaussian curve, and taking each determined third distance as a third pixel point weight corresponding to each pixel point to obtain a second temporary weight matrix.

According to the image fusion method, a plurality of image frames are original image data; the acquiring of the plurality of image frames specifically includes:

acquiring a plurality of image frames;

for each image frame, carrying out channel separation on the image frame according to the color sequence corresponding to the image frame to obtain a multi-channel image;

and taking the obtained multiple channel images as multiple image frames.

The image fusion method, wherein the fusing the image frames to obtain the output image based on the first weight map and the second weight map obtained by the determination specifically includes:

fusing each color channel of each image frame based on the first weight map and the second weight map which are obtained through determination to obtain a fused image;

and converting the fused image into a single-channel image according to a color sequence, and taking the single-channel image as an output image.

The image fusion method, wherein the fusing, based on the first weight map and the second weight map obtained by the determination, each color channel of each image frame to obtain a fused image specifically includes:

selecting a target image frame from the image frames, and taking other image frames except the target image frame from the image frames as reference image frames;

for each color channel in the target image frame, respectively determining a reference color channel corresponding to the color channel in each reference image frame, wherein the channel number of each reference color channel in the corresponding reference image frame is the same as the channel number of the color channel in the target image frame;

based on the first weight map and the second weight map, carrying out weighting processing on the color channel and each reference color channel to obtain a fusion channel corresponding to the color channel;

and determining fusion images corresponding to the plurality of image frames based on all the fusion channels obtained by determination.

A second aspect of embodiments of the present application provides a computer-readable storage medium storing one or more programs, which are executable by one or more processors to implement steps in an image fusion method as described in any one of the above.

A second aspect of the embodiments of the present application provides a terminal device, which includes: a processor, a memory, and a communication bus; the memory has stored thereon a computer readable program executable by the processor;

the communication bus realizes connection communication between the processor and the memory;

the processor, when executing the computer readable program, implements the steps in the image fusion method as described in any of the above.

Has the advantages that: compared with the prior art, the image fusion method, the storage medium and the terminal device are provided, and the method divides a plurality of acquired image frames into a first image group and a second image group based on exposure; respectively determining a first weight map corresponding to the first image group and a second weight map corresponding to the second image group; and finally, fusing the image frames to obtain an output image based on the first weight map and the second weight map which are obtained through determination. Thus, a first weight map corresponding to the first image group and a second weight map corresponding to the second image group are determined, and a plurality of image frames are fused according to the fusion weight maps corresponding to the first weight map and the second weight map, so that the image frames with different exposure levels correspond to different weight maps, and the respective effects are improved through the weight maps corresponding to the image frames with different exposure levels, thereby improving the image effect of the output image obtained by fusion.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without any inventive work.

Fig. 1 is a flowchart of an image fusion method provided in the present application.

Fig. 2 is an exemplary diagram of an image directly captured by the image capturing device.

Fig. 3 is an exemplary diagram of an output image obtained by fusing through the image fusion method provided by the present application.

Fig. 4 is a schematic structural diagram of a terminal device provided in the present application.

Detailed Description

The present application provides an image fusion method, a storage medium, and a terminal device, and in order to make the purpose, technical solution, and effect of the present application clearer and clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. As used herein, the term "and/or" includes all or any element and all combinations of one or more of the associated listed items.

It will be understood by those within the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The inventor finds that in a night scene environment, in order to obtain an image clear at highlight and shadow, a multi-frame multi-exposure image is generally acquired and then processed by a fusion technology to obtain an image clear at highlight and shadow. The image generated by single exposure cannot compress a highlight area and a highlight shadow area, the multi-exposure image contains image data of the same scene with different exposure amounts, the long-exposure image can be used for brightening the shadow area, the short-exposure image can be used for compressing the highlight area, and the highlight and shadow clear images can be obtained by fusing the multi-exposure image.

However, when determining an image by using a multi-exposure fusion technique, it is necessary to determine a fusion weight map corresponding to multiple image frames, where the fusion weight map is generally obtained based on quality evaluation of local image features, such as calculating the fusion weight map by using contrast, saturation, and exposure of the image, extracting luminance component weights of different image layers after performing detail decomposition on the image, and the like. The weight matrixes corresponding to the image frames in the fusion weight map are determined by the same method, and the suppression effect of the short-exposure image on the high-light area and the brightening effect of the long-exposure image on the shadow area cannot be embodied, so that the image effect of the output image obtained by fusion is poor.

In order to solve the above problem, in the embodiment of the present application, the image frames are divided into a first image group and a second image group by acquiring based on exposure; respectively determining a first weight map corresponding to the first image group and a second weight map corresponding to the second image group; and finally, fusing the image frames to obtain an output image based on the first weight map and the second weight map which are obtained through determination. Thus, a first weight map corresponding to the first image group and a second weight map corresponding to the second image group are determined, and a plurality of image frames are fused according to the fusion weight maps corresponding to the first weight map and the second weight map, so that the image frames with different exposure levels correspond to different weight maps, and the respective effects are improved through the weight maps corresponding to the image frames with different exposure levels, thereby improving the image effect of the output image obtained by fusion.

The following further describes the content of the application by describing the embodiments with reference to the attached drawings.

The present embodiment provides an image fusion method, as shown in fig. 1, the method including:

s10, acquiring a plurality of image frames.

Specifically, the image frames may be images acquired by an electronic device (e.g., a smart phone or the like) using an image fusion method, and the image frames may be original images acquired by an image acquisition device, where the image acquisition device may be configured in the electronic device operating the image fusion method, or may be configured in another external device, and the acquired original images are sent to the electronic device operating the image fusion method by the external device. In a possible implementation manner of this embodiment, the image capturing device is configured to an electronic device running an image fusion method, so that after the electronic device captures a plurality of image frames, the plurality of image frames can be directly fused, and after the electronic device obtains the plurality of image frames, the electronic device can directly output fused images corresponding to the plurality of image frames, thereby improving the real-time performance of the fusion of the plurality of image frames.

Further, each image frame in the plurality of image frames corresponds to each other, where each image frame corresponds to an image with the same size, and any two image frames in the plurality of image frames are respectively marked as a first image frame and a second image frame, and the first image frame and the second image frame correspond to the same image scene. The first image frame and the second image frame correspond to the same image scene, which means that the similarity between the image content carried by the first image frame and the image content carried by the second image frame meets a preset condition. It can be understood that the image size of the first image frame is the same as the image size of the second image frame, and when the first image frame and the second image frame are overlapped, the coverage rate of the object carried by the first image frame to the object corresponding to the first image frame in the second image frame satisfies the preset condition. The preset condition is a preset threshold, the similarity meeting the preset condition means that the similarity reaches the preset threshold, the coverage rate meeting the preset condition means that the coverage rate reaches the preset threshold, and the preset threshold is preset, for example, 99%.

Further, in an implementation manner of this embodiment, the plurality of image frames include at least two image frames, which are respectively recorded as an image frame a and an image frame B, where an exposure amount corresponding to the image frame a is different from an exposure amount corresponding to the image frame B, for example, the exposure amount corresponding to the image frame a is EV-8, and the exposure amount corresponding to the image frame B is EV +12, where EV (exposure values) is used to reflect an exposure amount, and the exposure amount is used to reflect an incident light amount of an aperture when the image is captured, that is, an amount of light entering the aperture in a process from a shutter opening to a shutter closing; and when the sensitivity was ISO 100, the aperture ratio was F1, and the exposure time was 1 second, the exposure amount was defined as 0. Further, EV-N denotes an exposure amount reduction N steps (e.g., a shutter time reduction of 1/N or a diaphragm reduction of 1/N), EV + N; the exposure amount is increased by N steps (the shutter time is increased by N times or the aperture is increased by N steps), wherein N is a positive integer. In one possible implementation manner of the embodiment, the exposure parameters (such as aperture factor and sensitivity) corresponding to a plurality of image frames are the same, and the image frames with different exposure amounts in the plurality of image frames are generated by adjusting the exposure time duration corresponding to the image frames, for example, for the image frame a with the exposure amount of EV-8 and the image frame B with the exposure amount of EV +12, the exposure time duration corresponding to the image frame a is smaller than the exposure time duration corresponding to the image frame B, wherein the exposure time duration refers to the time interval from the shutter opening to the closing, the aperture blade of the camera lens can leave the influence of the object on the film to leave the image in the time interval, and when the exposure time duration of the camera is long, the light entering the aperture is much; when the exposure time of the image pickup device is short, the light entering the aperture is small.

Based on this, in one implementation manner of the embodiment, the image frames may be a plurality of image frames with different exposure amounts obtained by shooting the same shooting scene for a plurality of times by an imaging device configured with the same exposure parameter, and the exposure amount of a partial image frame existing in the plurality of image frames with different exposure amounts is smaller than a preset exposure amount and is recorded as an underexposure image frame, and the exposure amount of the partial image frame is larger than the preset exposure amount and is recorded as an overexposure image frame, where the preset exposure amount may be 0, and the preset exposure amount is an exposure amount when the sensitivity is ISO 100, the aperture coefficient is F1, and the exposure time is 1 second. Of course, there may be a partial image frame in the plurality of image frames with different exposure amounts, and the exposure amount of the partial image frame is equal to the preset exposure amount and is recorded as a normal exposure image frame.

For example, the following steps are carried out: the image acquisition equipment comprises a plurality of image frames, a plurality of image acquisition equipment and a control system, wherein the image frames are acquired by the image acquisition equipment through different exposure time lengths, the image frames are 5 image frames with different exposure amounts, and the 5 image frames comprise a normal exposure image frame, an underexposure image frame and an overexposure image frame. The specific exposure of 5 frame image frames may be: EV-24, EV-16, EV-8, EV0, and EV +12, i.e., 5 frame image frames comprise 3 underexposed image frames, 1 normal exposed image frame, and 1 overexposed image frame.

Further, in order to improve the fusion speed of the image frames, in an implementation manner of this embodiment, the image frames are RAW image data, where the RAW image data may be RAW data obtained by converting a captured light source signal into a digital signal by a CMOS (Complementary Metal-Oxide-Semiconductor) or CCD (Charge Coupled Device) image sensor, and the RAW image data is unprocessed and compressed image data. In the embodiment, image details are prevented from being discarded in the image processing and compression processes, so that each image frame contains each image detail in the fusion process, and the quality of the output image obtained by fusion can be improved. In addition, the original image data RAW is used as the image frame, and the image frame does not need to be decoded in the fusion process, so that the time required by fusion of a plurality of image frames can be reduced, and the real-time performance of image fusion is improved.

S20, dividing a plurality of image frames into a first image group and a second image group based on exposure.

Specifically, the exposure amount is used to reflect the amount of light entering the diaphragm when an image is captured, that is, the amount of light entering the diaphragm during the shutter opening to closing. The time occupied by the process from the opening to the closing of the shutter is exposure time duration, and when the exposure parameters are fixed, the longer the exposure time duration is, the more the light quantity entering the aperture is, namely the greater the exposure quantity is; conversely, the shorter the exposure period, the smaller the amount of light entering the diaphragm, i.e., the smaller the exposure amount.

Further, the first image group includes at least one image frame, and the second image group includes at least one image frame, where the image frame in the first image group is referred to as a first image, and the image frame in the second image group is referred to as a second image. Each first image in the first image group is one image frame in a plurality of image frames, each second image in the second image group is one image frame in a plurality of image frames, and for any first image in the first image group, no second image which is the same as the first image exists in the second image group; similarly, for any second image in the second image group, the first image which is the same as the second image does not exist in the first image group. It is understood that the union of the first image set and the second image set is the image frames, and the intersection of the first image set and the second image set is an empty set.

Further, in one implementation of the embodiment, the first image group and the second image group are divided based on an exposure amount threshold. Correspondingly, the dividing the image frames into the first image group and the second image group based on the exposure amount may specifically include: for each frame of a plurality of image frames, comparing the exposure corresponding to the image frame with a preset exposure threshold; if the exposure corresponding to the image frame is larger than the preset exposure, adding the image frame to a second image group; and if the exposure amount corresponding to the image frame is less than or equal to the preset exposure amount, adding the image frame to the first image group, thereby obtaining a first image group and a second image group. Furthermore, as can be known from the dividing process of the first image group and the second image group, for each first image in the first image group, the exposure amount corresponding to the first image is less than or equal to the preset exposure amount; for each of the second images, the exposure amount corresponding to the second image is greater than the preset exposure amount.

Further, the preset exposure amount may be 0, and is an exposure amount when the sensitivity is ISO 100, the aperture coefficient is F1, and the exposure time is 1 second, which is referred to herein as a normal exposure amount; at least one first image exists in the first image group, the exposure amount of the first image is smaller than the preset exposure amount, the first image is called an underexposure image, and the highlight area can be suppressed through the underexposure image. In addition, the exposure amount of the second image in the second image group is greater than the preset exposure amount, the second image is referred to as an overexposed image, and the overexposed image can be used for brightening a shadow region, so that the highlight region and the shadow region can be clear by fusing a plurality of image frames including the underexposed image and the overexposed image, and the image effect of the output image obtained by fusion is improved.

S30, determining a first weight map corresponding to the first image group and a second weight map corresponding to the second image group.

Specifically, the first weight map is used for reflecting the weight of each first image in the first image group in the process of fusing a plurality of image frames; the second weight map is used for reflecting the weight of each second image in the second image group in the fusion process of a plurality of image frames. The first weighted image comprises a plurality of image channels, the number of the channels of the plurality of image channels is the same as the number of the first images contained in the first image group, and the plurality of image channels are in one-to-one correspondence with the plurality of first images in the first image group. For example, the first image group includes 3 first images, which are respectively a first image a, a first image B, and a first image C, and then the first weight image is a 3-channel image, which is respectively an image channel 1, an image channel 2, and an image channel 3; then first image a corresponds to image channel 1, first image B corresponds to image channel 2, and first image C corresponds to image channel 3.

For each first image, the image channel corresponding to the first image is used to reflect the weight of the first image in the process of fusing a plurality of image frames, and it can be understood that the image size of the first image is the same as the image size corresponding to the image channel, and for each pixel point in the image channel, the pixel value is a weight value corresponding to a target pixel point in the first image, and the pixel position of the target pixel point in the first image is the same as the pixel position of the pixel point in the image channel. For example, for the first image a, the image channel corresponding to the first weighted image is image channel 1, and the pixel value corresponding to the pixel point (10,20) in the image channel 1 is the weight corresponding to the pixel point (10,20) in the first image a.

Further, the second weighted image includes a plurality of image channels, the number of the channels of the plurality of image channels is the same as the number of the second images included in the second image group, and the plurality of image channels are in one-to-one correspondence with the plurality of second images in the second image group. For example, the second image group includes 3 second images, which are respectively a second image a, a second image B, and a second image C, and then the second weighted image is a 3-channel image, which is respectively an image channel 1, an image channel 2, and an image channel 3; then second image a corresponds to image channel 1, second image B corresponds to image channel 2, and second image C corresponds to image channel 3.

Further, for each second image, the image channel corresponding to the second image is used to reflect the weight of the second image in the process of fusing the plurality of image frames, and it can be understood that the image size of the second image is the same as the image size corresponding to the image channel, and for each pixel point in the image channel, the pixel value is a weight value corresponding to a target pixel point in the second image, and the pixel position of the target pixel point in the second image is the same as the pixel position of the pixel point in the image channel. For example, for the second image a, the image channel corresponding to the second weighted image is image channel 1, and the pixel value corresponding to the pixel point (30,20) in the image channel 1 is the weight corresponding to the pixel point (30,20) in the second image a.

Further, in an implementation manner of this embodiment, the determining the first weight map corresponding to the first image group specifically includes:

Specifically, a Gray Scale Image (Gray Scale Image or Gray Scale Image) is also called a Gray Scale Image, and is an Image represented by Gray Scale. The gray scale is formed by dividing the white and the black into a plurality of levels (e.g., 256, etc.) according to a logarithmic relationship, wherein each of the plurality of levels is a gray scale. In addition, since the original image data is single-channel data, before determining the first grayscale corresponding to the first image, color channel separation needs to be performed on the original image data to obtain multi-channel original image data, where the color of each pixel in each channel in the multi-channel original image data is the same.

Based on this, in an implementation manner of this embodiment, the acquiring the plurality of image frames specifically includes:

acquiring a plurality of image frames;

and taking the obtained multiple channel images as multiple image frames.

Specifically, the color sequence is a color arrangement sequence of each pixel in the image frame, and performing channel separation on the image frame according to the color sequence corresponding to the image frame refers to performing color channel separation on image frame data according to the color sequence of each pixel in the image frame. For example, as shown in fig. 2, the image frames are original image data of H × W × 1, and the image frame color order is RGBG, where H denotes the height of the first image, W denotes the width of the first image, and 1 denotes the color channel data of the first image. The image frames may be separated by color channels to generate a multi-channel image of H/2W 4, where H/2 represents the height of the first image after color channel separation, W/2 represents the width of the first image after color channel separation, and 4 represents the color channel data of the first image after color channel separation. The 4 color channels are respectively marked as a first color channel 1, a second color channel 2, a third color channel 3 and a fourth color channel 4, wherein the first color channel stores R pixel points, the second color channel stores G pixel points, the third color channel stores B pixel points, and the fourth color channel stores G pixel points; the R pixel represents a red pixel, the G pixel represents a green pixel, and the B pixel represents a blue pixel. In addition, after each image frame is divided into the multi-channel images respectively corresponding to the image frames, the multi-channel images are used as the image frames, so that each image frame is a four-channel image, and the color of the pixel point of each channel in the four-channel image is the same.

Further, after the four channels of image frames are acquired, for each image frame, averaging the 4 channels of RGBG data of the image frame to generate a gray map corresponding to the image frame. The process of averaging the 4 channels RGBG data of the image frame may be as follows: and for each pixel point in the R channel, respectively calculating the average value of the pixel value of each target pixel point and the pixel value of the pixel point by using the reference pixel point corresponding to the pixel point in the G channel, the G channel and the B channel, and taking the average value as the pixel value of the target pixel point in the gray-scale image. The pixel position of each reference pixel point in the corresponding channel is the same as the pixel position of the pixel point in the R channel, and the pixel position of the target pixel point in the gray-scale image is the same as the pixel position of the pixel point in the R channel. For example, the first color channel is an R channel, the second color channel is a G channel, the third color channel is a B channel, and the fourth color channel is a G channel, where the pixel point (10,15) in the first color channel has a red pixel value of 60; the pixel point (10,15) in the second color channel has a green pixel value of 60, the pixel point (10,15) in the third color channel has a blue pixel value of 55, and the pixel point (10,15) in the fourth color channel has a green pixel value of 65, so that the gray value of the pixel point (10,15) in the gray map is (60+60+55+65)/4, which is 60.

In an implementation manner of this embodiment, after the first grayscale map corresponding to each first image is acquired, normalization processing may be performed on the first grayscale map. Correspondingly, after determining the first gray scale map corresponding to each first image in the first image group, the method further includes: and normalizing the first gray-scale image, and taking the normalized first gray-scale image as a first gray-scale image corresponding to the first image. The normalization process is to divide the original image by the maximum value of the stored image, for example, if the maximum number of bits of the device stored image is 10 bits, then the maximum value of the divided stored image is 1023.

Further, after the first gray scale map corresponding to each first image is obtained, the first gray scale map corresponding to each first image may be processed by using an average filter (for example, a filter kernel is 9 × 9, etc.) to obtain each filtered gray scale map. After obtaining each filtered grayscale image, for each filtered first grayscale image, determining, based on the first grayscale image, a first weight matrix corresponding to the grayscale image, and accordingly, in an implementation manner of the embodiment, for each first image in the first image group, determining a first grayscale image corresponding to the first image, and determining, based on the first grayscale image, a first pixel weight corresponding to each pixel point in the first image, so as to obtain a first weight matrix corresponding to the first image specifically includes:

Specifically, the first gaussian curve is preset and is used for determining a gaussian distance of each pixel point in the first gray scale map. The first preset expected value is preset and used for representing expected Gaussian distances corresponding to the pixel points, and for each pixel point, the first distance corresponding to the pixel point is the distance between the Gaussian distance corresponding to the pixel point and the first preset expected value. The calculation formula of the gaussian distance may be:

wherein, W_kA k-th weight matrix, where k is 1, 2.. M, and M is the number of first images in the first image group; p_ij,kThe pixel value of a pixel point with coordinates i, j in the kth gray-scale image is shown; σ is the standard deviation, e.g., 0.18, etc.; u is a first desired value; the first expected value can be determined according to the exposure of the first image corresponding to the first gray scale map.

Further, after the first distances corresponding to the pixels are determined, the first distances are used as first pixel point weights corresponding to the pixels, so that the first pixel point weights corresponding to the pixels in the first gray-scale image are obtained. In addition, after obtaining a first pixel point weight value corresponding to each pixel in the first gray scale image, taking each first pixel point weight value as an element value of a target element corresponding to the pixel in the first weight matrix, where a position of the target element in the first weight matrix corresponds to a position of the pixel in the gray scale image (e.g., a position of the target element in the first weight matrix is (35,45), and a position of the pixel in the gray scale image is (35, 45)); and finally, taking the first weight matrix as a first weight matrix corresponding to the first image.

For example, the following steps are carried out: the first image group comprises 4 first images which are respectively a first image A, a first image B, a first image C and a first image D, wherein the exposure amount of the first image A is EV-24, the exposure amount of the first image B is EV-16, the exposure amount of the first image C is EV-8 and the exposure amount of the first image D is EV-0, so that the expected value corresponding to the first image A is 0.6, the expected value corresponding to the first image B is 0.45, the expected value corresponding to the first image C is 0.3 and the expected value corresponding to the first image D is 0.15.

Further, the determining process of the first weight map may be to arrange the first weight matrixes according to a channel direction to obtain the first weight map. It is understood that the first weight map is a multi-channel image, the number of channels of the first weight map is the same as the number of images of the first image in the first image group, and each image channel in the first weight map corresponds to one of several first weight matrices, and the first weight matrices corresponding to the image channels are different from each other. In one implementation of the embodiment, in order to quickly determine the weight matrix corresponding to each first image, when the first weight map is generated, the weight matrices may be sorted according to the shooting time of the corresponding first image, for example, according to the ascending order of the shooting time, or according to the descending order of the shooting time. Here, the description is made in ascending order of the shooting time, then, for any two adjacent channels in the first weight image, the channel a and the channel B, if the channel number of the channel a is smaller than the channel number of the channel B, the shooting time of the first image corresponding to the channel a is earlier than that of the first image corresponding to the channel B, and the first image a is adjacent to the first image B.

For example, the following steps are carried out: the first image group comprises 3 image frames which are respectively a first image frame, a second image frame and a third image frame, the shooting time corresponding to the first image frame is 11 points and 55 minutes 25 seconds, the shooting time corresponding to the second image frame is 11 points and 55 minutes 26 seconds, the shooting time corresponding to the third image frame is 11 points and 55 minutes 28 seconds, then the channel number of the weight matrix corresponding to the first image frame in the first weight map is 0, the channel number of the second image frame in the corresponding weight matrix in the first weight map is 1, and the channel number of the third image frame in the corresponding weight matrix in the first weight map is 1.

Further, in an implementation manner of this embodiment, the determining the second weight map corresponding to the second image group specifically includes:

Specifically, the grayscale map (Gray Scale Image or Gray Scale Image) is also called a grayscale map, and is an Image represented by grayscale. The gray scale is formed by dividing the white and the black into a plurality of levels (e.g., 256, etc.) according to a logarithmic relationship, wherein each of the plurality of levels is a gray scale. In addition, since the original image data is single-channel data, before determining the grayscale corresponding to the first image, color channel separation needs to be performed on the original image data to obtain multi-channel original image data, where the color of each pixel point in each channel in the multi-channel original image data is the same. The process of determining the second grayscale map corresponding to each second image in the second image group is the same as the process of determining the first grayscale map corresponding to each first image in the first image group, and it is not limited to this description, and specific reference may be made to the process of determining the first grayscale map corresponding to the first image. It is to be noted that, before determining the second grayscale images corresponding to the second images, color channel separation needs to be performed on the second images to convert the second images into multi-channel images, where a process of performing color channel separation on the second images is the same as a process of performing color channel separation on the first images, which is also not described herein, and reference may be specifically made to a process of performing color channel separation on the first images.

Further, all the element values in the mean matrix are equal and equal to the mean value of the pixel values of all the pixel points in the gray level image. Firstly, creating a two-dimensional matrix, wherein the matrix dimension of the two-dimensional matrix is the same as the image size of the gray-scale image, each element in the two-dimensional matrix corresponds to a target pixel point in the gray-scale image, and each element in the two-dimensional matrix is used for reflecting the pixel point weight value of the corresponding target pixel point; wherein, the position of the target pixel point in the gray scale corresponds to the position of the element in the two-dimensional matrix, for example, the position of the element in the two-dimensional matrix is (30,40), then the pixel position of the target pixel point corresponding to the element in the gray scale map is (30, 40); secondly, after the two-dimensional matrix is determined, setting each element value in the two-dimensional matrix as a preset value (for example, 1) and calculating the mean value of the pixel values of all pixel points in the gray-scale image; and finally, replacing the element value of each target element in the two-dimensional matrix by the calculated mean value to obtain a mean value matrix. For example, if the element value of each element in the two-dimensional matrix is 1, and the average pixel value of all the pixel points in the grayscale is 5, the two-dimensional matrix and the average pixel value may be subjected to a dot product operation, so that the element value of each element in the two-dimensional matrix is 5. Of course, when the element value in the two-dimensional matrix is a non-1 value (e.g., 3, etc.), the element value of each element in the two-dimensional matrix may be replaced with an average pixel value (e.g., 5, etc.), so that the element value of each element in the two-dimensional matrix is the average pixel value (e.g., 5, etc.).

Further, in an implementation manner of this embodiment, the determining, based on the mean matrix and the second grayscale map, a second weight matrix corresponding to the second image specifically includes:

Specifically, the second gaussian curve is preset and is used for determining the gaussian distance of each pixel point in the mean matrix. The second preset expected value is preset and used for representing expected Gaussian distances corresponding to the pixel points, and for each pixel point, the second distance corresponding to the pixel point is the distance between the Gaussian distance corresponding to the pixel point and the first preset expected value. The calculation formula of the gaussian distance may be:

wherein, W_kThe k-th transformed mean matrix is k 1,2, and N is the number of second images in the second image group; p_ij,kThe pixel value of a pixel point with coordinates i, j in the mean matrix before the kth conversion is obtained; σ is the standard deviation, e.g., 018, etc.; u is a second desired Gaussian distance; wherein the second expected gaussian distance may be determined according to the exposure of the second image corresponding to the mean matrix. For example, if the exposure amount of the second image a is EV +12, the expected value corresponding to the second image a is 0.05. It is of course worth mentioning that each second image corresponds to a mean matrix, whichThe element values of each element in the mean matrix are equal, so that when the element values in the mean matrix are converted into weight values through the gaussian distance, a target element can be selected from the mean matrix, the weight value corresponding to the target element is calculated through a gaussian distance formula, and then the original values of all the elements in the mean matrix are set as the weight values, so that the converted mean matrix is obtained. Therefore, the calculation steps of mean matrix conversion can be reduced, the acquisition speed of the converted mean matrix is improved, and the speed of image fusion is improved.

Further, in an implementation manner of this embodiment, the determining, based on a preset third gaussian curve and the second grayscale map, a second temporary weight matrix corresponding to the second image specifically includes:

Specifically, the guiding filter map is obtained by guiding filtering the second gray scale map, where the guiding filter may be a guiding filter with a half-value of 21 and a regularization parameter of 0.12. The third gaussian curve is preset, the third expected value is preset, and based on the preset third gaussian curve, the determination of the third distance between each pixel point in the guide filter map and the third expected value is the same as the determination of the first distance based on the first gaussian curve, which is not described here. In addition, after the second grayscale map corresponding to each second image is acquired, normalization processing may be performed on the second grayscale map. Correspondingly, after determining the second gray scale map corresponding to each second image in the second image group, the method further includes: and normalizing the second gray scale image, and taking the normalized second gray scale image as a second gray scale image corresponding to the second image. The normalization process is to divide the original image by the maximum value of the stored image, for example, if the maximum number of bits of the device stored image is 10 bits, then the maximum value of the divided stored image is 1023.

Further, after the first temporary weight matrix and the second temporary weight matrix are obtained, the matrix dimension of the first temporary weight matrix is the same as the matrix dimension of the second temporary weight matrix and is equal to the image size of the second image, wherein the matrix dimension refers to the number of rows and the number of columns of the matrix. For example, the image size of the second image is 224 x 224, then the matrix dimension of the first temporary weight matrix is 224 rows, 224 columns, and the matrix dimension of the second temporary weight matrix is 224 rows, 224 columns. Based on this, after the first temporary weight matrix and the second temporary weight matrix are obtained, the second weight matrix corresponding to the gray scale map can be determined through the matrix dot product operation of the first temporary weight matrix and the second temporary weight matrix, and the second weight matrix of the second image corresponding to the second gray scale map can be obtained.

Further, after second weight matrixes corresponding to the second gray scale maps are obtained, the second weight matrixes are arranged according to the channel direction to obtain second weight maps. It is to be understood that the second weight map is a multi-channel image, and the number of channels of the second weight map is the same as the number of images of the second image in the second image group. In one implementation of the embodiment, in order to quickly determine the second weight matrix corresponding to each second image, when the second weight map is generated, each second weight matrix may be sorted according to the shooting time of the corresponding first image, for example, according to the ascending order of the shooting time, or according to the descending order of the shooting time. To explain in ascending order of shooting time, for any two adjacent channels in the second weighted image, if the channel number of the channel a is smaller than that of the channel B, the shooting time of the second image a corresponding to the channel a is earlier than that of the second image B corresponding to the channel B, and the second image a is adjacent to the second image B.

And S40, fusing the image frames to obtain an output image based on the first weight map and the second weight map which are obtained through determination.

Specifically, the output image is a frame image frame obtained by fusing a plurality of image frames, wherein the output image can be taken as a shot image output by the image acquisition device. After the first weight map and the second weight map are obtained, the first weight map includes a weight matrix corresponding to each first image, and the second weight map includes a weight matrix corresponding to each second image, so that the first weight map and the second weight map can be spliced according to a channel direction to obtain the weight map, wherein the weight map includes a weight matrix corresponding to each image frame in a plurality of image frames. In addition, when the first weight map and the second weight map are merged, the first weight map may be located before the second weight map or may be located after the second weight map according to the channel direction.

For example, the following steps are carried out: the first weight image is a 4-channel image, the channel numbers are 0,1,2 and 3 respectively, the second weight image is a single-channel image, and the channel number is 0; then the weight map obtained by splicing the first weight map and the second weight map is a 5-channel image; when the second weight map is positioned behind the first weight map according to the channel direction, the channel with the channel number of 0 of the second weight image is changed into the channel with the channel number of 4 in the weight map; when the second weight map is located before the first weight map according to the channel direction, the channel with the channel number of 0 in the first weight map is changed into the channel with the channel number of 1 in the weight map; the channel with the channel number 1 in the first weight map is changed into the channel with the channel number 2 in the weight map; the channel with the channel number 2 in the first weight map is changed into the channel with the channel number 3 in the weight map; the channel with channel number 3 in the first weight map becomes the channel with channel number 4 in the weight map.

Based on this, in an implementation manner of this embodiment, the fusing, based on the first weight map and the second weight map obtained by the determination, the color channels of each image frame to obtain a fused image specifically includes:

Specifically, the target image frame is one of a plurality of image frames, and the reference image frame is all image frames except the target image frame in the plurality of image frames, where the target image frame may be any one of the plurality of image frames, or may be the image frame positioned at the top in the shooting order in the plurality of image frames, or may be the image frame positioned at the bottom in the shooting order in the plurality of image frames. For example, the image frames include an image frame a, an image frame B, and an image frame C, the image frame a is taken as a target image frame, and then the image frame B and the image frame C are reference image frames.

Further, the color channels included in the target image frame correspond to the color channels included in the reference image frames one-to-one, so that for each color channel in the target image frame, one reference color channel exists in each reference image frame and corresponds to the color channel, wherein the correspondence means that for each reference color channel, the channel number of the reference color channel in the reference image frame to which the reference color channel belongs is the same as the channel number of the color channel in the target image frame. For example, the image frames include an image frame a, an image frame B and an image frame C, the image frame a is a target image frame, the image frame B and the image frame C are reference image frames, the image frame a includes four color channels, the image frame B and the image frame C both include four color channels, and the channel numbers of the four color channels in the image frame a are 0,1,2 and 3, respectively; the channel numbers of the four color channels in the image frame B and the image frame C are also 0,1,2, and 3, respectively, so that the reference color channel a with the channel number of 0 in the image frame B and the reference color channel B with the channel number of 0 in the image frame C are both the reference color channels corresponding to the color channel a with the channel number of 0 in the image frame a.

Further, before weighting the color channel and each reference color channel based on the first weight map and the second weight map, the first weight map and the second weight map need to be spliced according to a channel direction to obtain the weight map, wherein the weight map includes a plurality of image channels, the number of the image channels is the same as that of the image frames, the channels correspond to the image frames one by one, and each channel is used for reflecting a weight coefficient of the corresponding image frame in a fusion process of the image frames. For example, the image frames include 5 image frames, the weight map is 5-channel images, and the 5 channels respectively correspond to the 5 image frames one by one.

Further, after the weight map is obtained, each pixel of the weight map is subjected to normalization operation, and the normalized weight map is taken as the weight map, wherein the normalization operation formula is as follows:

wherein the content of the first and second substances,

is the pixel value with coordinates i, j, W in the normalized k-th weight map_ij,kThe image weight is a pixel value with coordinates i, j in the k-th weight map, M is the number of first images in the first image group, N is the number of second images in the second image group, and M + N is the number of images of a plurality of image frames.

Further, after the normalized weight map, performing a matrix dot product operation on each image frame and its corresponding normalized weight matrix (image channel) to update the pixel value of each pixel in each image frame, for example, image frame a includes pixel (0,0), pixel (0,1), pixel (1,1) and pixel (1,0), and image frame a corresponds to a weight matrix including a₀₀，a₀₁，a₁₁And a₁₀Pixel points (0,0) and a₀₀The updated pixel value corresponding to the pixel point (0,0) is the pixel value before updating corresponding to the pixel point (0,0) and a₀₀The product of (a), i.e., the pixel value before update corresponding to pixel (0,0) is 50, a₀₀If the value is 0.5, the updated pixel value corresponding to the pixel (0,0) is 0.5 × 50 — 25; similarly, the updated pixel value corresponding to the pixel point (0,1) is the pixel value before updating corresponding to the pixel point (0,1) and a₀₁The updated pixel value corresponding to the pixel point (1,1) is the pixel value before update corresponding to the pixel point (1,1) and a₁₁The product of (a); the updated pixel value corresponding to the pixel point (1,0) is the pixel value before updating corresponding to the pixel point (1,0) and a₁₀The product of (a).

Further, after each updated image frame is obtained, the pixel values of the corresponding pixel positions in each updated image frame are added, and the added pixel value is used as the pixel value of the pixel position in the output image to obtain the output. For example, the image frames comprise an image frame a and an image frame B, and the image scale of an output image obtained by fusing the image frame a and the image frame B is the same as that of the image frame a, wherein the image scale of the image frame a is the same as that of the image frame B; and the pixel value of any pixel point (i, j) in the output image is equal to the pixel value of the pixel point (i, j) in the image frame A and the pixel value of the pixel point (i, j) in the image frame B.

Further, in one implementation manner of this embodiment, since color channel separation is performed on each image frame of the image frames after the image frames are acquired. Therefore, when the output image is acquired, the output image is the original image data after color channel separation. In this way, when a plurality of image frames are merged, the channel images for the color channels in the respective image frames are merged. Correspondingly, the fusing the image frames to obtain the output image based on the first weight map and the second weight map obtained by the determination specifically includes:

Specifically, after the first weight map and the second weight map are obtained, the weight map for fusion is determined based on the first weight map and the second weight map, and the obtaining process of the weight map is already described in the foregoing implementation manner, and is not described here again. After the weight map is obtained, each weight matrix a (image channel) in the weight map corresponds to one image frame a, and each color channel in the image frame a corresponds to the weight map. It is understood that the weighting matrix corresponding to the color channels of the image frame a is the weighting matrix a, for example, the image frame a includes a first color channel (R channel), a second color channel (G channel), a third color channel (B channel), and a fourth color channel (G channel), and then the first color channel (R channel), the second color channel (G channel), the third color channel (B channel), and the fourth color channel (G channel) all correspond to the weighting matrix a.

Based on this, the fusion of the color channels of the image frames to obtain the fused image may be to fuse the color channels with the channel number of 0 of each image frame, fuse the color channels with the channel number of 1 of each image frame, fuse the color channels with the channel number of 2 of each image frame, fuse the color channels with the channel number of 3 of each image frame, and fuse the color channels corresponding to each channel number to obtain the color channel of the fused image to obtain the fused image. In addition, after the fused image is acquired, the fused image is subjected to channel rearrangement according to the color sequence of the image frames, and an output image is obtained. For example, the fused image is a multi-channel image in which the R channel, the G channel 1, the B channel, and the G channel 2 are respectively provided in order of channel numbers, and the image size of the fused image is H/2 × W/2 × 4, where H/2 represents the height of the fused image, W/2 represents the width of the fused image, and 4 represents the fused image; creating a reference image of H W1, wherein H represents the height of the reference image, W represents the width of the reference image, and 1 represents the number of color channels of the reference image; and adding the pixel points of the color channels in the fused image into the reference image according to the sequence of the fused image according to the channel numbers, the sequence of the pixel points in the R channel, the sequence of the pixel points in the G channel 1, the sequence of the pixel points in the B channel and the sequence of the pixel points in the G channel 2 to obtain an output image.

For example, the following steps are carried out: the fused image is respectively an R channel, a G channel 1, a B channel and a G channel 2 according to the sequence of channel numbers, wherein the R channel comprises pixel points (0,0)_R、(0,1)_R、(1,0)_RAnd (1,1)_RG channel 1 includes pixel points (0,0)_G1、(0,1)_G1、(1,0)_G1And (1,1)_G1And the B channel comprises pixel points (0,0)_B、(0,1)_B、(1,0)_BAnd (1,1)_BG channel 2 includes pixel points (0,0)_G2、(0,1)_G2、(1,0)_G2And (1,1)_G2Then, the pixel points in the first line of the output image are arranged as follows: (0,0)_R、(0,0)_G1、(0,0)_B、(0,0)_G2(ii) a Second action (0,1)_G1、(0,1)_R、(0,1)_B、(0,1)_G2(ii) a Third action (1,0)_B、(1,0)_G2、(1,0)_R、(1,0)_G1Fourth action (1,1)_G2、(1,1)_R、(1,1)_G1、(1,1)_B。

In summary, the present application provides an image fusion method, a storage medium, and a terminal device, in which a plurality of image frames are obtained and divided into a first image group and a second image group based on an exposure amount; respectively determining a first weight map corresponding to the first image group and a second weight map corresponding to the second image group; and finally, fusing the image frames to obtain an output image based on the first weight map and the second weight map which are obtained through determination. Thus, a first weight map corresponding to the first image group and a second weight map corresponding to the second image group are determined, and a plurality of image frames are fused according to the fusion weight maps corresponding to the first weight map and the second weight map, so that the image frames with different exposure levels correspond to different weight maps, and the respective effects are improved through the weight maps corresponding to the image frames with different exposure levels, thereby improving the image effect of the output image obtained by fusion. For example, as shown in fig. 2, an image of a shooting scene directly shot by an imaging device is shown, and as shown in fig. 3, an image obtained by fusing a plurality of image frames corresponding to the shooting scene according to the image fusion method of the present embodiment is shown, as can be seen from fig. 2 and 3, the definition of the text in fig. 3 is higher than that in fig. 2.

Based on the image fusion method described above, the present embodiment provides a computer-readable storage medium storing one or more programs, which are executable by one or more processors to implement the steps in the image fusion method described above.

Based on the image fusion method, the present application further provides a terminal device, as shown in fig. 4, which includes at least one processor (processor) 20; a display screen 21; and a memory (memory)22, and may further include a communication Interface (Communications Interface)23 and a bus 24. The processor 20, the display 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a user guidance interface preset in the initial setting mode. The communication interface 23 may transmit information. The processor 20 may call logic instructions in the memory 22 to perform the methods in the embodiments described above.

Furthermore, the logic instructions in the memory 22 may be implemented in software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product.

The memory 22, which is a computer-readable storage medium, may be configured to store a software program, a computer-executable program, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes the functional application and data processing, i.e. implements the method in the above-described embodiments, by executing the software program, instructions or modules stored in the memory 22.

The memory 22 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal device, and the like. Further, the memory 22 may include a high speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.

In addition, the specific processes loaded and executed by the storage medium and the instruction processors in the terminal device are described in detail in the method, and are not stated herein.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims

1. An image fusion method, characterized in that the method comprises:

acquiring a plurality of image frames;

2. The image fusion method of claim 1, wherein the similarity of the image content of each of the image frames satisfies a predetermined condition.

3. The image fusion method according to claim 1, wherein the image frames are raw image data, and for each first image in the first image group, the exposure amount corresponding to the first image is less than or equal to a preset exposure amount; for each of the second images, the exposure amount corresponding to the second image is greater than the preset exposure amount.

4. The image fusion method according to any one of claims 1 to 3, wherein the determining the first weight map corresponding to the first image group specifically comprises:

5. The image fusion method according to claim 4, wherein the determining, for each first image in the first image group, a first gray-scale map corresponding to the first image, and determining, based on the first gray-scale map, a weight of a first pixel point corresponding to each pixel point in the first image to obtain a first weight matrix corresponding to the first image specifically comprises:

6. The image fusion method of claim 5, wherein after determining the first gray-scale map corresponding to each first image in the first image group, the method further comprises:

7. The image fusion method according to any one of claims 1 to 3, wherein the determining the second weight map corresponding to the second image group specifically comprises:

8. The image fusion method according to claim 7, wherein the determining a second weight matrix corresponding to the second image based on the mean matrix and the second gray-scale map specifically comprises:

9. The image fusion method according to claim 8, wherein the determining, based on a preset third gaussian curve and the second gray scale map, a second temporary weight matrix corresponding to the second image specifically includes:

10. The image fusion method according to any one of claims 1-3, wherein the image frames are raw image data; the acquiring of the plurality of image frames specifically includes:

acquiring a plurality of image frames;

and taking the obtained multiple channel images as multiple image frames.

11. The image fusion method according to claim 10, wherein the fusing the image frames to obtain the output image based on the determined first weight map and the second weight map specifically comprises:

12. The image fusion method according to claim 11, wherein the fusing the color channels of the image frames to obtain the fused image based on the determined first weight map and the second weight map specifically comprises:

13. A computer-readable storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the steps in the image fusion method according to any one of claims 1 to 12.

14. A terminal device, comprising: a processor, a memory, and a communication bus; the memory has stored thereon a computer readable program executable by the processor;

the processor, when executing the computer readable program, implements the steps in the image fusion method of any of claims 1-12.