CN115272155A - Image synthesis method, image synthesis device, computer equipment and storage medium - Google Patents

Abstract

The application relates to an image synthesis method, an image synthesis device, a computer device, a storage medium and a computer program product. The method comprises the steps of obtaining a group of first images to be synthesized including one frame of low exposure level and second images to be synthesized including at least two frames of high exposure levels, determining an overexposed region in the high-brightness images obtained after the at least two frames of images to be synthesized are fused, comparing the at least two frames of second images to be synthesized to determine a non-moving region in the overexposed region of the high-brightness images, determining the regions to be synthesized in the first images to be synthesized based on the non-moving region, replacing the non-moving region in the overexposed region of the high-brightness images with the images of the regions to be synthesized, and obtaining the synthesized images. Compared with the traditional mode of synthesizing multi-frame images by taking the image with high brightness as the reference, the scheme replaces the non-motion area of the over-exposure area with the image with the low exposure level during synthesis, does not replace the motion area of the over-exposure area, prevents the ghost phenomenon from occurring in the synthesized image and improves the image definition.

Description

Image synthesis method, device, computer equipment and storage medium

technical field

The present application relates to the technical field of image processing, in particular to an image synthesis method, device, computer equipment, storage medium and computer program product.

Background technique

With the development of computer technology, people have been able to shoot images through mobile devices such as mobile phones and cameras. The shooting of HDR (High dynamic range, high dynamic range) images has become the mainstream. HDR images are images obtained by synthesizing multiple frames of images. . There may be images of moving objects in one or more frames in multiple frames of images. The current method of synthesizing multiple frames of images is usually to synthesize multiple frames of images based on a frame of image with high brightness. However, due to the presence of moving objects in multiple frames of images, and the brightness difference between these images with moving objects and other images, direct synthesis will lead to ghosting in the image, resulting in a decrease in the clarity of image synthesis .

Therefore, the current image synthesis method has the defect that the synthesis definition is not high.

Contents of the invention

Based on this, it is necessary to provide an image synthesis method, device, computer equipment, computer readable storage medium and computer program product capable of improving the definition of image synthesis in order to address the above technical problems.

In a first aspect, the present application provides an image synthesis method, the method comprising:

Acquire a group of images to be synthesized of the same object; the group of images to be synthesized includes one frame of the first image to be synthesized at the first exposure level and at least two frames of the second image to be synthesized at the second exposure level; the first The exposure level is lower than the second exposure level; the at least two frames of second images to be synthesized include images with different exposure durations, and one of the frames of images is consistent with the image structure of the first image to be synthesized;

fusing the at least two frames of second images to be synthesized to obtain a high-brightness image, and determining an overexposed area in the high-brightness image;

According to the comparison result of the at least two frames of the second image to be synthesized, determine a non-moving area in the overexposed area of the high-brightness image; no moving object exists in the non-moving area;

Determining a region to be synthesized in the first image to be synthesized according to a matching result of the first image to be synthesized and a non-moving area in the overexposed area;

The non-moving area in the overexposed area of the high-brightness image is replaced with the image of the area to be combined, and a combined image of the same object is obtained according to the replaced high-brightness image.

In one of the embodiments, the acquisition of a group of images to be synthesized of the same object includes:

Acquiring a first original image with a first exposure duration of the target object, processing the first original image based on the first exposure level and the second exposure level respectively, to obtain a first sensitivity and a first exposure duration at the first exposure level The first image to be synthesized, and the short-exposure image of the first sensitivity and the first exposure time under the second exposure level;

Acquiring at least one frame of the second original image for the second exposure time of the target object, processing the at least one frame of the second original image based on the second exposure level, and obtaining the second sensitivity and the second exposure level at the second exposure level. At least one frame of long-exposure images with an exposure duration; the first sensitivity is greater than the second sensitivity; the brightness of the short-exposure image is consistent with that of the long-exposure image;

The group of images to be combined is obtained according to the first image to be combined, the short exposure image and the at least one frame of long exposure image.

In one of the embodiments, the fusing of the at least two frames of second images to be synthesized to obtain a high-brightness image includes:

Determine the non-overexposed area in the at least two frames of the second image to be synthesized, and determine the moving area in the non-overexposed area according to the comparison result of the at least two frames of the second image to be synthesized; the moving area presence of moving objects;

replacing the image of the moving area in the non-overexposed area with the image of the corresponding area in the short exposure image;

replacing the image of the non-moving area in the non-overexposed area with the image of the corresponding area in the long exposure image; there is no moving object in the non-moving area in the non-overexposed area;

According to the fusion result of the replaced image of the moving area in the non-overexposure area and the image of the non-moving area in the non-overexposure area, a fused high-brightness image is obtained.

In one of the embodiments, the determining the non-moving area in the overexposed area of the high-brightness image according to the comparison result of the at least two frames of the second image to be synthesized includes:

Acquiring the pixel coincidence degree of the image of the overexposed area in the at least two frames of the second image to be synthesized;

According to the area where the pixel coincidence degree of the image of the overexposure area of the short exposure image and the image of the overexposure area of the long exposure image is greater than a preset coincidence degree threshold, determine the non-exposed area in the overexposure area of the high brightness image. mobile area.

In one of the embodiments, the determining the area to be synthesized in the first image to be synthesized according to the matching result of the first image to be synthesized and the non-moving area in the overexposed area includes:

matching the first image to be synthesized with the non-moving area in the overexposed area to obtain the non-moving area in the first image to be synthesized;

Obtaining the area to be synthesized according to the non-moving area in the first image to be synthesized.

In one of the embodiments, the determining the overexposure area in the high-brightness image includes:

Acquiring areas in the fused high-brightness image whose brightness is greater than a preset brightness threshold, to obtain an overexposed area in the high-brightness image.

In one of the embodiments, the obtaining the synthesized image of the same object according to the replaced high-brightness image includes:

At least one of sharpening, brightness adjustment, and smoothing and denoising is performed on the replaced high-brightness image to obtain a composite image of the same object whose image definition is greater than a preset definition threshold.

In a second aspect, the present application provides an image synthesis device, the device comprising:

An acquisition module, configured to acquire a group of images to be synthesized of the same object; the group of images to be synthesized includes one frame of the first image to be synthesized at the first exposure level and at least two frames of the second image to be synthesized at the second exposure level ; The first exposure level is smaller than the second exposure level; The at least two frames of the second image to be synthesized include images with different exposure durations, and one frame of the image is consistent with the image structure of the first image to be synthesized ;

A fusion module, configured to fuse the at least two frames of the second images to be synthesized to obtain a high-brightness image, and determine an overexposed area in the high-brightness image;

The first determination module is configured to determine a non-moving area in the overexposed area of the high-brightness image according to the comparison result of the at least two frames of the second image to be synthesized; there is no moving object in the non-moving area;

The second determination module is configured to determine the area to be synthesized in the first image to be synthesized according to the matching result of the first image to be synthesized and the non-moving area in the overexposed area;

A synthesis module, configured to replace the non-moving area in the overexposed area of the high-brightness image with the image of the area to be combined, and obtain a combined image of the same object according to the replaced high-brightness image.

In a third aspect, the present application provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

In a fourth aspect, the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

The image synthesis method, device, computer equipment, storage medium, and computer program product described above, by acquiring a group of the same object including one frame of the first image to be synthesized at a low exposure level and at least two frames of the second image to be synthesized at a high exposure level, And fusing at least two frames of images to be synthesized with high exposure levels, determining the overexposed area in the fused high brightness image, and determining the non The moving area, based on the non-moving area, determines the area to be synthesized in the first image to be synthesized, replaces the non-moving area in the overexposed area of the high-brightness image with the image of the area to be synthesized, and based on the replaced high-brightness image, obtains A composited image of the same object. Compared with the traditional way of synthesizing multiple frames of images based on one frame of high brightness image, this scheme replaces the non-moving area of the over-exposed area with an image of low exposure level during synthesis, and the motion of the over-exposed area Areas are not replaced to prevent ghosting in composite images and improve image clarity.

Description of drawings

Fig. 1 is a schematic flow chart of an image synthesis method in an embodiment;

Fig. 2 is a schematic flow chart of the step of acquiring images to be synthesized in one embodiment;

Fig. 3 is a schematic flow chart of the fusion step in an embodiment;

Fig. 4 is a structural block diagram of an image synthesis device in an embodiment;

Figure 5 is an internal block diagram of a computer device in one embodiment.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

In one embodiment, as shown in FIG. 1 , an image synthesis method is provided. In this embodiment, the method is applied to a terminal for illustration. It can be understood that the method can also be applied to a server, or to a A system including a terminal and a server, and realized through interaction between the terminal and the server, includes the following steps:

Step S202, acquiring a group of images to be synthesized of the same object; a group of images to be synthesized includes one frame of the first image to be synthesized at the first exposure level and at least two frames of the second image to be synthesized at the second exposure level; the first exposure The level is smaller than the second exposure level; at least two frames of the second image to be synthesized include images with different exposure time lengths, and one frame of the image is consistent with the image structure of the first image to be synthesized.

Wherein, the images to be synthesized may be images that need to be synthesized, and the synthesis process may be a synthesis of HDR images, that is, the terminal may obtain a group of images to be synthesized, and synthesize a group of images to be synthesized into an HDR image. Wherein, the above-mentioned acquired group of images to be synthesized may be images for the same object. For example, the terminal uses the image acquisition device set in the terminal to shoot the same object once, and can obtain multiple images taken this time as a group of images. Image to be synthesized. Wherein, a group of images to be synthesized includes at least three frames of images, and there are images with different exposure levels in the at least three frames of images. For example, the above group of images to be combined may include one frame of the first image to be combined under the first exposure level and at least two frames of the second image to be combined under the second exposure level. Wherein the first exposure level is lower than the second exposure level, that is, the exposure level of the first image to be combined is lower than the exposure level of the second image to be combined, so that the brightness of the first image to be combined is lower than the brightness of the second image to be combined. Specifically, the first exposure level can be called ev-, and the second exposure level can be called ev0; where the ev value is called the exposure value, which reflects the combination of aperture size and shutter speed. Accurate exposure needs to be combined with exposure value and exposure compensation amount. Exposure compensation amount is represented by +3, +2, +1, 0, -1, -2, -3, etc. Increase the exposure based on the exposure amount, "-" means reduce the exposure, and the corresponding number is the level of compensation exposure. For example, ev- can indicate a negative exposure level, and ev0 can indicate a 0th exposure level.

Wherein, the aforementioned group of images to be synthesized may also include images with different exposure durations, for example, a group of images to be synthesized may include images to be synthesized with a short exposure duration and images to be synthesized with a long exposure duration. Specifically, the first image to be combined at the first exposure level may be a short-exposure image, and the second image to be combined at the second exposure level includes a short-exposure image and a long-exposure image. Moreover, one of the at least two frames of the second image to be synthesized is consistent with the image structure of the first image to be synthesized. Specifically, it may be the second short-exposure image to be synthesized, and the terminal may generate the above-mentioned first image to be synthesized and the above-mentioned short-exposure second image to be synthesized from the same source, that is, the first image to be synthesized and the second short-exposure image to be synthesized come from the same For the original image, the terminal respectively obtains the first image to be synthesized and the second short-exposure image to be synthesized through different image processing processes, so that the image content and image structure of the first image to be synthesized and the second short-exposure image to be synthesized are consistent.

Step S204, fusing at least two frames of second images to be synthesized to obtain a high-brightness image, and determining an overexposed area in the high-brightness image.

Wherein, after the terminal obtains the aforementioned group of images to be synthesized, it may first synthesize the above-mentioned at least two frames of the second images to be synthesized, that is, the terminal may fuse at least two frames of the second images to be synthesized at the exposure level of ev0. The brightness of the represented exposure level is greater than the ev-exposure level, so after the fusion of at least two frames of the second image to be synthesized, a high-brightness image may be obtained. Wherein, the brightness of each frame of the second image to be synthesized in the at least two frames of the second image to be synthesized may be consistent. The above fusion process may be performed on the non-overexposed area in the second image to be synthesized. For example, the mobile area of the terminal in the non-overexposed area uses the image of this area in the second image to be synthesized to make the moving object as far as possible. Still, in the non-moving area in the non-overexposed area, use the image of this area in the second image to be synthesized by long exposure to make the image quality as clear as possible.

After the terminal fuses the at least two frames of the second image to be synthesized with respect to the non-overexposed area, it may perform corresponding processing on the overexposed area in the second image to be synthesized. The terminal may first determine the overexposed area in the high-brightness image. For example, in one embodiment, determining the overexposed area in the high-brightness image includes: obtaining an area in the fused high-brightness image whose brightness is greater than a preset brightness threshold, to obtain the overexposed area in the high-brightness image. In this embodiment, the terminal may determine whether the area is an overexposed area according to the brightness of each area in the high-brightness image. For example, the terminal can compare the brightness of each pixel in the fused high-brightness image with a preset brightness threshold, so that the terminal can obtain the areas in the high-brightness image whose brightness is greater than the preset brightness threshold, and the terminal can compare the brightness of these pixels with the brightness greater than the preset brightness threshold The area of the image is regarded as the overexposed area in the high-brightness image.

Step S206, according to the comparison result of at least two frames of the second image to be synthesized, determine the non-moving area in the overexposed area of the high-brightness image; no moving object exists in the non-moving area.

Wherein, the second image to be combined may be an image with higher brightness than the first image to be combined. And the brightness of the at least two frames of the second images to be synthesized may be consistent. Therefore, the terminal can compare at least two frames of the second image to be synthesized, and determine the non-moving region in the overexposed region in the high-brightness image based on the comparison result of the comparison. Wherein, the non-moving area represents that there is no moving object in this area, that is, a state where there is no motion blur. Wherein, there are short-exposure images and long-exposure images in the above-mentioned at least two frames of the second images to be synthesized, since the brightness of the short-exposure images in the above-mentioned at least two frames of the second images to be synthesized is consistent with the brightness of the long-exposure images, and the images to be synthesized They are all for the same object, so the terminal can compare the above short exposure image with the long exposure image, and when the terminal detects an image inconsistency between the short exposure image and the long exposure image, it can consider that there is a moving object in the area, and the terminal These areas where moving objects exist may be regarded as moving areas, and furthermore, the terminal may regard areas where images in the short-exposure image and the long-exposure image are the same as non-moving areas, indicating that there is no moving object in this area. Therefore, the non-moving area in the overexposed area in the high-brightness image can be obtained by mapping the comparison result back to the above-mentioned high-brightness image.

Step S208, according to the matching result of the first image to be synthesized and the non-moving area in the overexposed area, determine the area to be synthesized in the first image to be synthesized.

Wherein, the non-moving area in the over-exposed area may be the non-moving area in the over-exposed area in the high-brightness image obtained by the terminal after comparing the short-exposure image and the long-exposure image in the at least two frames of the second image to be synthesized. area. Wherein, the first image to be synthesized and the short-exposure image are images obtained through different processes based on the same original image, so the image content and structure of the first image to be synthesized and the short-exposure image are consistent. Therefore, the non-moving area in the over-exposed area determined by the terminal based on the short-exposure image and the long-exposure image may be consistent with the non-moving area in the first image to be synthesized. The terminal may match the non-moving area in the overexposed area of the above-mentioned high-brightness image with the first image to be synthesized, for example, the matching may be based on the position of the non-moving area in the over-exposed area in the high-brightness image, so that the terminal The area to be combined in the first image to be combined may be determined according to the matching result. Wherein, the images in the area to be synthesized may be images used for image replacement.

Step S210, replacing the non-moving area in the overexposed area of the high-brightness image with the image of the area to be combined, and obtaining a combined image of the same object according to the replaced high-brightness image.

Wherein, after the terminal determines the area to be synthesized in the first image to be synthesized, it may replace the non-moving area in the overexposed area of the high-brightness image with an image of the area to be synthesized, and the first image to be synthesized may be a For a low-level image, that is, an image without overexposure, after the terminal replaces the image in the non-moving area in the overexposed area of the above-mentioned high-brightness image with the image of the area to be synthesized, the overexposure state of the replaced area can be removed. Improve image clarity.

After the terminal replaces the non-moving area in the overexposed area of the high-brightness image with the image of the area to be combined, it can obtain the combined image of the same object based on the replaced high-brightness image. Wherein, the terminal may post-process the replaced high-brightness image to improve the transparency and clarity of the replaced high-brightness image. For example, in one embodiment, obtaining a composite image of the same object according to the replaced high-brightness image includes: performing at least one of sharpening, brightness adjustment, and smoothing and denoising on the replaced high-brightness image A processing is performed to obtain a synthesized image of the same object whose image sharpness is greater than a preset sharpness threshold. In this embodiment, the terminal may adjust the replaced high-brightness image accordingly to improve the transparency and brightness of the image. For example, the terminal may perform at least one of sharpening, brightness adjustment, and smoothing and denoising processing on the replaced high-brightness image, so that the terminal may improve the transparency and clarity of the replaced high-brightness image. After the terminal performs the above adjustments on the replaced high-brightness image, a synthesized image based on the above group of images to be synthesized can be obtained. Since the first image to be synthesized using ev-exposure level is used in the non-moving area of the overexposed area Replace the image in the corresponding area, and maintain the overexposure for the moving area in the overexposed area, so as to ensure that the synthesized image will not cause ghosting due to the overlapping of moving objects and stationary objects.

In the above image synthesis method, by acquiring a group of the same object including one frame of the first image to be synthesized with a low exposure level and at least two frames of the second image to be synthesized with a high exposure level, and fusing at least two frames of the high exposure level to be synthesized image, determine the overexposed area in the fused high-brightness image, and determine the non-moving area in the over-exposed area of the high-brightness image according to the comparison results of at least two frames of the second image to be synthesized, and determine the first based on the non-moving area For the area to be synthesized in the image to be synthesized, replace the non-moving area in the overexposed area of the high-brightness image with the image of the area to be synthesized, and obtain a synthesized image of the same object based on the replaced high-brightness image. Compared with the traditional way of synthesizing multiple frames of images based on one frame of high brightness image, this scheme replaces the non-moving area of the over-exposed area with an image of low exposure level during synthesis, and the motion of the over-exposed area Areas are not replaced to prevent ghosting in composite images and improve image clarity.

In one embodiment, as shown in FIG. 2 , FIG. 2 is a schematic flowchart of a step of acquiring images to be synthesized in one embodiment. The above-mentioned acquisition of a group of images to be synthesized of the same object includes: Step S302: Acquire the first original image of the first exposure time for the target object, process the first original image based on the first exposure level and the second exposure level respectively, and obtain the second The first image to be synthesized with the first sensitivity and the first exposure time at an exposure level, and the short-exposure image with the first sensitivity and the first exposure time at the second exposure level; Step S304: Acquire the target object At least one frame of the second original image of the second exposure time, at least one frame of the second original image is processed based on the second exposure level to obtain at least one frame of long-exposure image of the second sensitivity and the second exposure time under the second exposure level ; the first sensitivity is greater than the second sensitivity; the brightness of the short-exposure image is consistent with that of the long-exposure image; Step S306: Obtain a group of images to be synthesized according to the first image to be synthesized, the short-exposure image and at least one frame of the long-exposure image.

In this embodiment, the terminal can acquire a group of images to be synthesized for the same object through an image acquisition device. For example, the terminal captures multiple frames of images of the same object through the camera to obtain a group of images to be synthesized. In the aforementioned group of images to be combined acquired by the terminal, there may be multiple frames of images with different shooting parameters. Wherein, the above-mentioned same object may be the target object. The terminal may acquire the first original image of the target object under the first exposure duration. And acquire at least one frame of second original image for the above-mentioned target object under the second exposure duration. Wherein, the above-mentioned first exposure duration is shorter than the above-mentioned second exposure duration, and both the above-mentioned first original image and the second original image may be a raw format image, and the raw format image contains camera sensor data information necessary for creating a visible image. The terminal may perform a corresponding processing flow on the above original images to obtain the first image to be combined and the second image to be combined. Specifically, the above-mentioned processing may be ISP (Image Signal Processor, image signal processing) processing of raw format images, and its main function is to perform post-processing on the signal output by the front-end image sensor. Can better restore the details of the scene.

For the above-mentioned first original image, the terminal may perform different ISP processing on the first original image to obtain multiple images to be synthesized under different exposure levels. Wherein, the above-mentioned first original image has a corresponding first exposure time and first sensitivity (ISO). The terminal can process the above-mentioned first original image based on the first exposure level, so that the terminal can obtain the first image to be synthesized with the first sensitivity and the first exposure time under the first exposure level; the terminal can also process the first original image based on the second exposure level The above-mentioned first original image, so that the terminal can obtain a short-exposure image with the first sensitivity and the first exposure time under the second exposure level. That is, the terminal may obtain images to be synthesized with different exposure levels through different processes based on a first original image. Specifically, the first image to be synthesized may be an image of ev-exposure level, the short-exposure image may be an image of ev0 exposure level, and the exposure duration and sensitivity of the first image to be synthesized and the short-exposure image may be the same. That is, the first image to be synthesized and the short-exposure image can be generated from the same source, so that the image content and image structure of the first image to be synthesized and the short-exposure image are consistent, which facilitates subsequent determination of the region to be synthesized.

For the second original image, the terminal may process at least one frame of the second original image based on the second exposure level to obtain at least one frame of long-exposure image with a second sensitivity and a second exposure time under the second exposure level. Since the exposure time of the second original image is longer, there may be a moving area caused by the moving object. Wherein, in order to make the brightness of the short-exposure image consistent with the brightness of the long-exposure image, the terminal can make the first sensitivity greater than the second sensitivity, so that the terminal can obtain a short exposure based on a shorter exposure time and a larger first sensitivity. image, and obtain a long-exposure image based on a longer exposure time and a smaller second sensitivity, so that the short-exposure image with consistent brightness can be compared with the long-exposure image to determine the moving area. Wherein, the above at least one frame of the long exposure image may be an image of the same exposure level as the above short exposure image, for example, both are at the exposure level of ev0, so that the terminal can obtain a combination of the short exposure image and the long exposure image with the same brightness. Specifically, when combining images, the terminal may acquire one ev- first image to be combined and N ev0 second images to be combined, where N≥2. The above N ev0s may include images to be combined with short exposure time and high ISO, and may also include images to be combined with long exposure time and low ISO, so that the brightness of the second images to be combined in the above N ev0s is consistent. And the image content and image structure of the first image to be synthesized of ev- is consistent with the second image to be synthesized of ev0 with a short exposure time. After acquiring the first image to be synthesized, the short exposure image and the long exposure image, the terminal may obtain the above-mentioned group of images to be synthesized for the same object based on these images, and a group of images to be synthesized includes at least three frames of images.

Through this embodiment, based on the same original image, the terminal can obtain multiple images to be synthesized with consistent image content and structure under different exposure levels, so that the terminal can correspond to The moving area of the image to be synthesized with low brightness is returned to improve the accuracy of image synthesis. Moreover, the terminal generates a short-exposure image and a long-exposure image with consistent brightness, and compares the short-exposure image and the long-exposure image to determine the moving area in the image, and then can perform image synthesis based on the moving area, improving the clarity of image synthesis.

In one embodiment, as shown in FIG. 3 , FIG. 3 is a schematic flowchart of the fusion step in one embodiment. The above fusion of at least two frames of the second images to be synthesized to obtain a high-brightness image includes: step S402: determining the non-overexposed areas in the at least two frames of the second images to be synthesized, and according to the comparison results of the at least two frames of the second images to be synthesized, Determine the moving area in the non-overexposed area; there is a moving object in the moving area; step S404: replace the image of the moving area in the non-overexposed area with the image of the corresponding area in the short exposure image; step S406: replace the image in the non-overexposed area The image of the non-moving area is replaced with the image of the corresponding area in the long exposure image; there is no moving object in the non-moving area in the non-overexposed area; Step S408: According to the image and the non-moving area in the non-overexposed area after replacement The fusion result of the image of the non-moving area in the over-exposed area is obtained to obtain a fused high-brightness image.

In this embodiment, the terminal may fuse at least two frames of the second images to be synthesized at the second exposure level to obtain a high-brightness image. For example, the terminal may first determine the non-overexposed areas in the above-mentioned at least two frames of the second images to be synthesized. Specifically, the terminal may detect the brightness of each pixel in each of the second images to be synthesized, and reduce the brightness to a value lower than the preset brightness threshold. The area is regarded as the non-overexposed area. The terminal may also determine the moving area in the non-overexposed area according to the comparison result of at least two frames of the second image to be synthesized. Wherein, the moving area indicates that there is a moving object in the area, and a motion blurred area is generated. Specifically, the terminal may compare the at least two frames of the second images to be combined to obtain the pixel coincidence degree, and use the region in the non-overexposure area whose pixel coincidence degree is lower than the preset coincidence degree threshold as the movement in the non-overexposure area. area, after the terminal determines the moving area in the non-overexposure area, other areas in the non-overexposure area can be determined as non-moving areas.

The at least two frames of the second image to be synthesized include a short-frame exposure image and at least one long-exposure image, and the terminal may use the short-exposure image as a reference frame to replace the image of the moving area in the non-overexposed area with the short-exposure image The image of the corresponding area in the image, and the image of the non-moving area in the non-overexposed area is replaced by the image of the corresponding area in the long exposure image. Specifically, when the short-exposure image with the exposure level of ev0 is used as the reference frame, for the non-overexposed area in the image, if there is a moving area, the terminal can use the image of the corresponding area in the short-exposure image at this position to ensure that objects in motion It is frozen and clearly presented; the remaining non-moving areas in the non-overexposed area can use the image of the corresponding area in the long-exposure image to ensure clear quality. The terminal determines replacement images for each area in the second image to be synthesized, and can obtain a fused high-brightness image according to the fusion result of the image of the moving area in the non-overexposed area and the image of the non-moving area in the non-overexposed area.

Through this embodiment, the terminal can use the image of the corresponding area in the short exposure image for the moving area in the non-overexposed area of the second image to be synthesized, and use the image of the corresponding area in the long exposure image for the non-moving area in the non-overexposed area. image, thereby improving the definition of the fused high-brightness image, and further improving the definition of the final composite image.

In one embodiment, according to the comparison results of at least two frames of the second image to be synthesized, determining the non-moving area in the overexposed area of the high-brightness image includes: acquiring the at least two frames of the overexposed area in the second image to be synthesized The pixel coincidence degree of the image; according to the pixel coincidence degree of the image of the overexposure region of the short exposure image and the image of the overexposure region of the long exposure image is greater than the preset coincidence degree threshold, determine the non- mobile area.

In this embodiment, the terminal may determine the non-moving area in the overexposed area in the at least two frames of the second image to be synthesized by comparison. For example, the terminal may compare the pixels of the at least two frames of the second images to be synthesized, and acquire the pixel coincidence degree of the overexposed area in the at least two frames of the second images to be synthesized, and the terminal may acquire the overexposure The pixel coincidence degree of the image of the region and the image of the overexposed region in the long exposure image is treated with a preset coincidence degree threshold, and the non-moving region in the overexposed region in the high brightness image is determined according to these regions. Specifically, the above-mentioned at least two frames of second images to be synthesized include a short-exposure image and a long-exposure image, and the brightness of the short-exposure image and the long-exposure image are consistent, so the overexposure area determined based on the short-exposure image and the long-exposure image will be Consistent with the overexposed area of the fused high-brightness image. Therefore, the terminal can determine the non-moving area in the above-mentioned over-exposure area by comparing the short-exposure image and the long-exposure image, while other areas in the over-exposure area are determined as moving areas in the over-exposure area. The moving area indicates that there is a moving object in the area, and the non-moving area indicates that there is no moving object in the area.

Through this embodiment, the terminal can determine the non-moving area of the overexposed area in the high-brightness image based on the pixel overlap of the image. Therefore, the terminal can perform image replacement based on the non-moving area, thereby improving the clarity of image synthesis.

In one embodiment, according to the matching result of the first image to be synthesized and the non-moving area in the overexposed area, determining the area to be synthesized in the first image to be synthesized includes: combining the first image to be synthesized with the non-moving area in the overexposed area The non-moving area in the first image to be synthesized is matched to obtain the non-moving area in the first image to be synthesized; the area to be synthesized is obtained according to the non-moving area in the first image to be synthesized.

In this embodiment, the terminal may determine the non-moving areas in the first image to be combined to replace the non-moving areas in the over-exposed areas by corresponding the determined non-moving areas in the over-exposed areas back to the first image to be combined. area. Since the first image to be synthesized and the short-exposure image are derived from the same original image, and their image structure and image content are consistent, the non-moving area in the over-exposed area determined by the terminal based on the short-exposure image can be compared with the first corresponding to the image to be synthesized. The terminal may match the first image to be synthesized with the determined non-moving area in the overexposed area to obtain the non-moving area in the first image to be synthesized, so that the terminal may base on the non-moving area in the first image to be synthesized Get the area to be synthesized. Wherein, the image in the area to be synthesized can be used to replace the image in the non-moving area of the overexposed area in the high-brightness image. When you want to use the ev-frame image for the overexposed area in the image, you can decide whether to use it according to whether the position belongs to the moving area, so that the ev-frame image can be accurately restored to the synthesized image to avoid ghosting.

Through this embodiment, the terminal can identify the moving area and the non-moving area of the overexposed area in the high-brightness image, and based on the corresponding relationship between the short-exposure image and the first image to be combined, determine in the first image to be combined that the above The location of the non-moving area in the exposure area corresponds to the area to be combined, so that the terminal can implement image synthesis based on the images in the area to be combined, which improves the clarity of image synthesis.

It should be understood that although the steps in the flow charts involved in the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flow charts involved in the above-mentioned embodiments may include multiple steps or stages, and these steps or stages are not necessarily executed at the same time, but may be performed at different times For execution, the execution order of these steps or stages is not necessarily performed sequentially, but may be executed in turn or alternately with other steps or at least a part of steps or stages in other steps.

Based on the same inventive concept, an embodiment of the present application further provides an image synthesis device for implementing the above-mentioned image synthesis method. The solution to the problem provided by the device is similar to the implementation described in the above method, so the specific limitations in one or more embodiments of the image synthesis device provided below can refer to the definition of the image synthesis method above, I won't repeat them here.

In one embodiment, as shown in FIG. 4 , an image synthesis device is provided, including: an acquisition module 500, a fusion module 502, a first determination module 504, a second determination module 506, and a synthesis module 508, wherein:

The acquiring module 500 is configured to acquire a group of images to be synthesized of the same object; a group of images to be synthesized includes one frame of the first image to be synthesized at the first exposure level and at least two frames of the second image to be synthesized at the second exposure level; The first exposure level is lower than the second exposure level; at least two frames of the second image to be synthesized include images with different exposure durations, and one frame of the image is consistent with the image structure of the first image to be synthesized.

The fusion module 502 is configured to fuse at least two frames of second images to be synthesized to obtain a high-brightness image, and determine an overexposed area in the high-brightness image.

The first determination module 504 is configured to determine a non-moving area in the overexposed area of the high-brightness image according to the comparison result of at least two frames of the second image to be synthesized; there is no moving object in the non-moving area.

The second determination module 506 is configured to determine the area to be synthesized in the first image to be synthesized according to the matching result of the first image to be synthesized and the non-moving area in the overexposed area.

The synthesis module 508 is configured to replace the non-moving area in the overexposed area of the high-brightness image with the image of the area to be combined, and obtain a combined image of the same object according to the replaced high-brightness image.

In one embodiment, the above acquisition module 500 is specifically configured to acquire the first original image of the first exposure duration of the target object, process the first original image based on the first exposure level and the second exposure level respectively, and obtain the first exposure The first image to be synthesized with the first sensitivity and the first exposure time under the level, and the short exposure image with the first sensitivity and the first exposure time under the second exposure level; obtain the second exposure time for the target object At least one frame of the second original image, at least one frame of the second original image is processed based on the second exposure level to obtain at least one frame of long-exposure image with the second sensitivity and the second exposure time under the second exposure level; the first sensitivity greater than the second sensitivity; the brightness of the short-exposure image is consistent with that of the long-exposure image; and a group of images to be synthesized is obtained according to the first image to be synthesized, the short-exposure image and at least one frame of the long-exposure image.

In one embodiment, the above fusion module 502 is specifically configured to determine the non-overexposed area in at least two frames of the second image to be synthesized, and determine the non-overexposed area in the non-overexposed area according to the comparison result of the at least two frames of the second image to be synthesized. There are moving objects in the moving area; replace the image of the moving area in the non-overexposed area with the image of the corresponding area in the short exposure image; replace the image of the non-moving area in the non-overexposed area with the image in the long exposure image The image of the corresponding area; there is no moving object in the non-moving area in the non-overexposed area; according to the fusion result of the image of the moving area in the replaced non-overexposed area and the image of the non-moving area in the non-overexposed area, get The fused high-brightness image.

In one embodiment, the above-mentioned first determination module 504 is specifically configured to obtain the pixel overlap of the overexposed area images in at least two frames of the second image to be synthesized; In the overexposed area of the image, an area where the overlapping degree of pixels of the image is greater than a preset overlapping degree threshold is determined as a non-moving area in the overexposed area of the high-brightness image.

In one embodiment, the above-mentioned second determination module 506 is specifically configured to match the first image to be synthesized with the non-moving area in the overexposed area to obtain the non-moving area in the first image to be synthesized; according to the first image to be synthesized The non-moving area in the image is the area to be synthesized.

In one embodiment, the above-mentioned fusion module 502 is specifically configured to obtain regions in the fused high-brightness image whose brightness is greater than a preset brightness threshold, and obtain overexposed regions in the high-brightness image.

In one embodiment, the above synthesis module 508 is specifically configured to perform at least one of sharpening, brightness adjustment, and smoothing and denoising on the replaced high-brightness image to obtain the same The composited image of the object.

Each module in the above-mentioned image synthesizing device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure may be as shown in FIG. 5 . The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, mobile cellular network, NFC (Near Field Communication) or other technologies. When the computer program is executed by a processor, an image synthesis method is realized. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the casing of the computer device , and can also be an external keyboard, touchpad, or mouse.

Those skilled in the art can understand that the structure shown in Figure 5 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation to the computer equipment on which the solution of this application is applied. The specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, where a computer program is stored in the memory, and the processor implements the above image synthesis method when executing the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above image synthesis method is implemented.

In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above image synthesis method is realized.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all Information and data authorized by the user or fully authorized by all parties.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any reference to storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile and volatile storage. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive variable memory (ReRAM), magnetic variable memory (Magnetoresistive Random Access Memory, MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (Phase Change Memory, PCM), graphene memory, etc. The volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory. As an illustration and not a limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM). The databases involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a blockchain-based distributed database, etc., but is not limited thereto. The processors involved in the various embodiments provided by this application can be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., and are not limited to this.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be determined by the appended claims.

Claims (11)

1. An image synthesis method, characterized in that the method comprises:

acquiring a group of images to be synthesized of the same object; the group of images to be synthesized comprises a first image to be synthesized of a frame with a first exposure level and at least two second images to be synthesized of a frame with a second exposure level; the first exposure level is less than the second exposure level; the at least two frames of second images to be synthesized comprise images with different exposure time lengths, and one frame of image is consistent with the image structure of the first image to be synthesized;

fusing the at least two frames of second images to be synthesized to obtain a high-brightness image, and determining an overexposure area in the high-brightness image;

determining a non-moving area in an overexposed area of the high-brightness image according to a comparison result of the at least two frames of second images to be synthesized; the non-moving area is free of moving objects;

determining a region to be synthesized in the first image to be synthesized according to a matching result of the first image to be synthesized and a non-moving region in the overexposure region;

and replacing the non-moving area in the overexposure area of the high-brightness image with the image of the area to be synthesized, and obtaining a synthesized image of the same object according to the replaced high-brightness image.

2. The method according to claim 1, wherein the acquiring a set of images to be synthesized of the same object comprises:

acquiring a first original image of a first exposure time for a target object, and processing the first original image respectively based on a first exposure level and a second exposure level to obtain a first to-be-synthesized image of a first sensitivity and the first exposure time at the first exposure level and a short-exposure image of the first sensitivity and the first exposure time at the second exposure level;

acquiring at least one frame of second original image of a second exposure duration aiming at the target object, and processing the at least one frame of second original image based on a second exposure level to obtain at least one frame of long exposure image of a second sensitivity and the second exposure duration under the second exposure level; the first sensitivity is greater than the second sensitivity; the short exposure image and the long exposure image have the same brightness;

and obtaining the group of images to be synthesized according to the first image to be synthesized, the short exposure image and the at least one frame long exposure image.

3. The method according to claim 2, wherein the fusing the at least two frames of the second image to be synthesized to obtain a high-brightness image comprises:

determining a non-overexposed area in the at least two frames of second images to be synthesized;

determining a moving area in the non-overexposure area according to a comparison result of the at least two frames of second images to be synthesized; moving objects exist in the moving area;

replacing the image of the moving area in the non-overexposure area with the image of the corresponding area in the short-exposure image;

replacing the image of the non-moving area in the non-overexposure area with the image of the corresponding area in the long-exposure image; a non-moving region in the non-overexposure region is free of moving objects;

and obtaining a fused high-brightness image according to the fusion result of the replaced image of the moving area in the non-overexposure area and the image of the non-moving area in the non-overexposure area.

4. The method according to claim 2, wherein the determining a non-moving region in the overexposed region of the high brightness image according to the comparison result of the at least two frames of second images to be synthesized comprises:

acquiring the pixel coincidence degree of the images of the overexposure areas in the at least two frames of second images to be synthesized;

and determining a non-moving area in the overexposure area of the high-brightness image according to an area in which the pixel overlapping degree of the image of the overexposure area of the short-exposure image and the image of the overexposure area of the long-exposure image is greater than a preset overlapping degree threshold value.

5. The method according to claim 1, wherein determining the region to be synthesized in the first image to be synthesized according to the matching result of the first image to be synthesized and the non-moving region in the overexposed region comprises:

matching the first image to be synthesized with a non-moving area in the overexposure area to obtain the non-moving area in the first image to be synthesized;

and obtaining a region to be synthesized according to the non-moving region in the first image to be synthesized.

6. The method of claim 1, wherein determining an overexposed region in a high brightness image comprises:

and obtaining a region with the brightness larger than a preset brightness threshold value in the fused high-brightness image to obtain an overexposed region in the high-brightness image.

7. The method according to any one of claims 1 to 6, wherein obtaining a synthesized image of the same object from the replaced high-brightness image comprises:

and performing at least one of sharpening, brightness adjustment and smooth denoising on the replaced high-brightness image to obtain a synthesized image of the same object with the image definition larger than a preset definition threshold.

8. An image synthesizing apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring a group of images to be synthesized of the same object; the group of images to be synthesized comprises a first image to be synthesized of a frame with a first exposure level and at least two second images to be synthesized of a frame with a second exposure level; the first exposure level is less than the second exposure level; the at least two frames of second images to be synthesized comprise images with different exposure durations, and one frame of image is consistent with the image structure of the first image to be synthesized;

the fusion module is used for fusing the at least two frames of second images to be synthesized to obtain a high-brightness image and determining an overexposed area in the high-brightness image;

a first determining module, configured to determine a non-moving region in an overexposed region of the high-brightness image according to a comparison result of the at least two frames of second images to be synthesized; the non-moving area is free of moving objects;

a second determining module, configured to determine a region to be synthesized in the first image to be synthesized according to a matching result between the first image to be synthesized and a non-moving region in the overexposed region;

and the synthesis module is used for replacing the non-moving area in the overexposure area of the high-brightness image with the image of the area to be synthesized, and obtaining a synthesized image of the same object according to the replaced high-brightness image.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 7.

11. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 7 when executed by a processor.

Images