CN111429369A - High dynamic range image generation method and device, electronic equipment and storage medium - Google Patents

High dynamic range image generation method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111429369A
CN111429369A CN202010183368.1A CN202010183368A CN111429369A CN 111429369 A CN111429369 A CN 111429369A CN 202010183368 A CN202010183368 A CN 202010183368A CN 111429369 A CN111429369 A CN 111429369A
Authority
CN
China
Prior art keywords
image
ghost
pixel
images
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010183368.1A
Other languages
Chinese (zh)
Other versions
CN111429369B (en
Inventor
王涛
李绪琴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Megvii Technology Co Ltd
Original Assignee
Beijing Megvii Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Megvii Technology Co Ltd filed Critical Beijing Megvii Technology Co Ltd
Priority to CN202010183368.1A priority Critical patent/CN111429369B/en
Publication of CN111429369A publication Critical patent/CN111429369A/en
Application granted granted Critical
Publication of CN111429369B publication Critical patent/CN111429369B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20208—High dynamic range [HDR] image processing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Facsimile Image Signal Circuits (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

The embodiment of the application provides a method and a device for generating a high dynamic range image, wherein the method comprises the following steps: acquiring a reference image and at least one other image, and adjusting the brightness of the other images through histogram matching for each other image; generating candidate ghost mask images corresponding to other images based on the difference between the other images after brightness adjustment and corresponding pixels of the reference image; setting highlight relevant parts in ghost mask areas of candidate ghost mask images corresponding to other images as non-ghost mask areas to obtain final ghost mask images corresponding to other images; a high dynamic range image is generated based on the final ghost mask image and the plurality of images for each of the other images. The high-light area is determined as a ghost area, and the situation that the brightness of the high-light area in the high-dynamic-range image is abnormal, such as the high-light suppression situation that the brightness of the high-light area is dark, is avoided, and the quality of the generated high-dynamic-range image is improved.

Description

High dynamic range image generation method and device, electronic equipment and storage medium
Technical Field
The present application relates to the field of image processing, and in particular, to a method and an apparatus for generating a high dynamic range image, an electronic device, and a storage medium.
Background
The High Dynamic Range (HDR) fusion algorithm is used for fusing images with different exposure quantities of the same scene shot by a camera to obtain a high dynamic range image, so that the quality of the image shot in a night scene is greatly improved. When a moving object appears in a shot scene, a high dynamic range image obtained by fusion may generate a ghost, which may also be referred to as an artifact, and the ghost in the high dynamic range image needs to be removed.
At present, the commonly adopted method for removing the ghost image is as follows: adjusting the brightness of the images with different exposure amounts to the same brightness, calculating the brightness difference between the pixels at the same position of the images with different exposure amounts, comparing the brightness difference between the pixels at the same position of the images with different exposure amounts with a threshold value, and if the brightness difference between the pixels at the same position of the images with different exposure amounts is greater than the threshold value, the same position belongs to a ghost area. After determining each position belonging to the ghost region, the ghost region of the high dynamic range image is determined, and then, the ghost in the high dynamic range image is removed.
However, the difference in luminance between a plurality of pixels at the same position in the highlight region of an image from different exposure amounts is generally large, resulting in that the highlight region is erroneously determined as a ghost region according to the result of comparison of the difference in luminance between a plurality of pixels at the same position in the highlight region of an image from different exposure amounts with a threshold value. However, the high-light area has no ghost, and the high-light area is erroneously determined as belonging to the ghost area, so that the brightness of the pixels in the high-light area is adjusted in a ghost-removing manner, and when the ghost is removed, the brightness abnormality of the high-light area in the high-dynamic-range image occurs, for example, a highlight suppression situation occurs due to brightness darkening.
Disclosure of Invention
In order to overcome the problems in the related art, the present application provides a high dynamic range image generation method, apparatus, electronic device, and storage medium.
According to a first aspect of embodiments of the present application, there is provided a high dynamic range image generation method, including:
acquiring a plurality of images for generating a high dynamic range image, the plurality of images comprising: a reference image, at least one other image;
for each other image, adjusting the brightness of the other image through histogram matching; generating a candidate ghost mask image corresponding to the other image based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image; setting a highlight relevant part in a ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area to obtain a final ghost mask image corresponding to the other image, wherein the highlight relevant part corresponds to highlight areas of the reference image and the image with the maximum exposure in the other image;
a high dynamic range image is generated based on the final ghost mask image and the plurality of images for each of the other images.
According to a second aspect of embodiments of the present application, there is provided a high dynamic range image generating apparatus including:
an acquisition unit configured to acquire a plurality of images for generating a high dynamic range image, the plurality of images including: a reference image, at least one other image;
a processing unit configured to adjust, for each of the other images, the brightness of the other image by histogram matching; generating a candidate ghost mask image corresponding to the other image based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image; setting a highlight relevant part in a ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area to obtain a final ghost mask image corresponding to the other image, wherein the highlight relevant part corresponds to highlight areas of the reference image and the image with the maximum exposure in the other image;
a generating unit configured to generate a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images.
The high dynamic range image generation method and device provided by the application acquire a plurality of images for generating a high dynamic range image, wherein the plurality of images comprise: a reference image, at least one other image; for each other image, adjusting the brightness of the reference image or the brightness of the other images through histogram matching; generating a candidate ghost mask image corresponding to the other image based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image; setting a highlight relevant part in a ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area to obtain a final ghost mask image corresponding to the other image, wherein the highlight relevant part corresponds to highlight areas of the reference image and the image with the maximum exposure in the other image; and fusing the plurality of images based on the final ghost mask image corresponding to each other image to obtain a high dynamic range image. The method and the device have the advantages that the highlight related part in the ghost mask area of the candidate ghost mask image is set to be the non-ghost mask area to obtain the final ghost mask image, so that the actual ghost area in the candidate image is accurately determined by the final ghost mask image, the highlight area in the candidate image cannot be determined to be the ghost area, the brightness of the pixels in the highlight area in the candidate image cannot be adjusted in a mode of removing the ghost, and the condition that the brightness of the highlight area in the high dynamic range image is abnormal, such as the highlight suppression condition that the brightness of the highlight area is dark, is avoided.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
FIG. 1 illustrates a flow diagram of one embodiment of a high dynamic range image generation method provided herein;
FIG. 2 is a flow chart diagram illustrating a high dynamic range image generation method provided by the present application;
fig. 3 is a block diagram showing a structure of a high dynamic range image generating apparatus provided in the present application;
fig. 4 shows a block diagram of an electronic device provided in the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the associated invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the related invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
FIG. 1 shows a flow diagram of one embodiment of a high dynamic range method provided herein, the method comprising:
step 101, acquiring a plurality of images for generating a high dynamic range image.
In the present application, a plurality of images for generating a high dynamic range image may be selected from all images of the same scene that are continuously captured. The plurality of images for generating a high dynamic range image includes: a reference picture, at least one other picture.
In the present application, for each of a plurality of images used to generate a high dynamic range image, the exposure level of the image refers to the exposure amount employed at the time of shooting the image. The exposure amount of each of the plurality of images used to generate the high dynamic range image is different.
For example, the plurality of images for generating the high dynamic range image are two images for generating the high dynamic range image among all images continuously taken. Two images for generating a high dynamic range image include: overexposed images, underexposed images, the exposure of overexposed images being greater than the exposure of underexposed images. The overexposed image is a reference image, and the underexposed image is another image.
For another example, the plurality of images are images for generating a high dynamic range image of three of all images continuously captured. Three images for generating a high dynamic range image include: underexposed images, normally exposed images, overexposed images, the exposure of overexposed images being greater than the exposure of normally exposed images, the exposure of normally exposed images being greater than the exposure of underexposed images. The normally exposed image is a reference image, and the underexposed image and the overexposed image are the exposure of other images.
Step 102, generating a final ghost mask image corresponding to each other image.
In this application, in order to generate a final ghost mask image corresponding to each other image, for each other image, the final ghost mask image corresponding to the other image may be obtained by: adjusting the brightness of the other images through histogram matching; generating a candidate ghost mask image corresponding to the other image based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image; and setting a highlight relevant part in a ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area to obtain a final ghost mask image corresponding to the other image, wherein the highlight relevant part in the ghost mask area of the candidate ghost mask image corresponding to the other image corresponds to the highlight area of the reference image and the image with the maximum exposure in the other image.
For each other image, the brightness of the other image is adjusted through histogram matching, so that the brightness of the reference image and the brightness of the other image can reach the same brightness level.
In other words, the brightness of the reference image is maintained, and the brightness of each of the other images is adjusted by targeting the brightness of the reference image, so that the brightness of the plurality of images is adjusted to the same standard after the brightness of each of the other images is adjusted.
In this application, for each other image, after the brightness of the other image is adjusted, the brightness of the other image, that is, the brightness of the pixel in the other image changes, and the other image may be referred to as the brightness-adjusted other image to indicate that the other image has undergone brightness adjustment.
For each other image, when the candidate ghost mask image corresponding to the other image is generated based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image, a brightness difference image corresponding to the other image may be generated, where the brightness difference image corresponding to the other image describes the difference between the other image and the corresponding pixel of the reference image; and generating a candidate ghost mask image corresponding to the other image based on the brightness difference image corresponding to the other image.
For each other image, the pixel value of the pixel in the brightness difference image corresponding to the other image is the brightness difference between the brightness of the first pixel in the reference image and the brightness of the second pixel corresponding to the first pixel in the other image.
Each pixel in the reference image may be referred to as a first pixel. Each pixel in any one of the other images may be referred to as a second pixel.
For each first pixel in the reference image, the first pixel corresponds to a second pixel in the other image.
For each first pixel, the position of the first pixel in the reference image is the same as the position of the second pixel corresponding to the first pixel in the other images.
In this application, for each other image, a candidate ghost mask image corresponding to the other image may be generated based on the luminance difference image corresponding to the other image.
For each of the other images, when the exposure amount of the reference image is greater than the exposure amount of the other image, the luminance of the first pixel in the reference image may be subtracted by the luminance of the second pixel corresponding to the first pixel in the other image to obtain a luminance difference between the luminance of the first pixel and the luminance of the second pixel corresponding to the first pixel in the other image. When the exposure amount of the other image is larger than the exposure amount of the reference image, the luminance of the second pixel in the other image corresponding to the first pixel in the reference image may be subtracted from the luminance of the first pixel in the reference image to obtain a luminance difference between the luminance of the first pixel and the luminance of the second pixel in the other image corresponding to the first pixel. Thus, it is ensured that the brightness difference values are all greater than zero. It is also possible to directly perform subtraction calculation for two luminances used for calculating a luminance difference value regardless of the exposure amount, and in the case where the obtained calculation result is negative, the absolute value of the calculation result, that is, the magnitude of the difference between the luminance of the first pixel in the reference image and the luminance of the second pixel corresponding to the first pixel in the other image is taken as the luminance difference value.
For each other image, after calculating the difference between the reference image and the corresponding pixel of the other image, a luminance difference image corresponding to the other image may be generated, where the pixel value of each pixel in the luminance difference image corresponding to the other image is the luminance difference value between the first pixel and the second pixel at the corresponding position.
It should be understood that the luminance of the second pixel corresponding to the first pixel in the other image described above refers to the luminance of the second pixel corresponding to the first pixel in the other image after the luminance adjustment is performed on the other image.
In this application, for each other image, a candidate ghost mask image corresponding to the other image may be generated based on the luminance difference image corresponding to the other image.
When the candidate ghost mask image corresponding to the other image is generated based on the luminance difference image corresponding to the other image, it may be determined whether a pixel value of each pixel in the luminance difference image corresponding to the other image is greater than a luminance difference threshold, where the pixel value of the pixel is an absolute value of a luminance difference between luminance of a first pixel corresponding to the pixel and luminance of a second pixel corresponding to the first pixel after luminance adjustment. The position of the first pixel corresponding to the pixel in the reference image is the same as the position of the pixel in the luminance difference image corresponding to the other image. When the pixel value of a pixel in the brightness difference image corresponding to the other image is greater than the brightness difference threshold, the pixel value of the pixel may be maintained. When the pixel value of a pixel in the luminance difference image corresponding to the other image is less than or equal to the luminance difference threshold, the pixel value of the pixel may be set to 0.
For each other image, after determining to maintain the pixel value of the pixel or set the pixel value of the pixel to 0 for each pixel in the luminance difference image corresponding to the other image, a candidate ghost mask image corresponding to the other image may be obtained.
For each other image, the candidate ghost mask image corresponding to the other image includes: ghost mask regions, non-ghost mask regions.
For each other image, the pixel value of each pixel in the ghost mask area of the candidate ghost mask image corresponding to the other image is a value other than 0.
For each other image, the pixel value of each pixel in the non-ghost-mask region of the candidate ghost-mask image corresponding to the other image is 0.
In some embodiments, for each other image, generating a candidate ghost mask image corresponding to the other image based on the difference between the brightness-adjusted other image and the corresponding pixel of the reference image comprises: calculating a normalized pixel value of each pixel associated with the ghost area in the brightness difference image corresponding to the other image, wherein the pixel associated with the ghost area is a pixel of which the pixel value is greater than a brightness difference threshold value; setting the pixel value of each pixel associated with the ghost area as the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel which is not associated with the ghost area in the brightness difference image corresponding to the other image as 0 to obtain a candidate ghost mask image corresponding to the other image, wherein the pixel which is not associated with the ghost area is the pixel of which the pixel value is less than or equal to the brightness difference threshold value.
For each other image, the pixels of the luminance difference image corresponding to the other image whose pixel values are greater than the luminance difference threshold value may be referred to as pixels associated with the ghost region, and the pixels of the luminance difference image corresponding to the other image whose pixel values are less than or equal to the luminance difference threshold value may be referred to as pixels not associated with the ghost region.
For each of the other images, when calculating the normalized pixel value of each pixel associated with the ghost region in the luminance difference image corresponding to the other image, for each pixel associated with the ghost region in the luminance difference image corresponding to the other image, the pixel value of the pixel associated with the ghost region may be divided by the maximum value of the pixel values of the pixels in the luminance difference image corresponding to the other image, and the obtained result may be taken as the normalized pixel value of the pixel associated with the ghost region.
For each other image, after setting the pixel value of each pixel associated with the ghost area in the brightness difference image corresponding to the other image as the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel not associated with the ghost area as 0, obtaining a candidate ghost mask image corresponding to the other image.
For each other image, the candidate ghost mask image corresponding to the other image includes a ghost mask region and a non-ghost mask region. The pixels in the ghost mask area of the candidate ghost mask image corresponding to the other image are pixels associated with the ghost area, and the pixels in the non-ghost mask area of the candidate ghost mask image corresponding to the other image are pixels not associated with the ghost area.
The pixel value of each pixel associated with the ghost area in the ghost mask area of the candidate ghost mask image corresponding to the other image is the normalized pixel value of the pixel associated with the ghost area. Each pixel value of a pixel of the candidate ghost mask image corresponding to the other image, which is not associated with the ghost area, is 0.
In some embodiments, for each of the other images, calculating a normalized pixel value for each pixel associated with the ghost region in the luminance difference image corresponding to the other image comprises: for each pixel associated with the ghost area in the brightness difference image corresponding to the other image, determining a pixel value of the pixel associated with the ghost area for normalization based on whether the pixel value of the pixel associated with the ghost area is greater than a preset brightness difference upper limit value; and performing normalization processing on the pixel value used for normalization of each pixel associated with the ghost area to obtain a normalized pixel value of each pixel associated with the ghost area.
In order to calculate the normalized pixel value of each pixel associated with the ghost region in the luminance difference image corresponding to the other image, a preset luminance difference upper limit value may be preset.
For each other image, regarding each pixel associated with the ghost area in the brightness difference image corresponding to the other image, when the pixel value of the pixel associated with the ghost area is less than or equal to the preset brightness difference upper limit value, the pixel value of the pixel is taken as the pixel value of the pixel for normalization, and when the pixel value of the pixel associated with the ghost area is greater than the preset brightness difference upper limit value, the preset brightness difference upper limit value is taken as the pixel value of the pixel for normalization.
For each other image, when normalization processing is performed on the pixel value for normalization of each pixel associated with the ghost area, for each pixel associated with the ghost area, the pixel value for normalization of the pixel is divided by the preset brightness difference value upper limit value to obtain a normalized pixel value of the pixel.
For example, the brightness difference threshold is 20, and the preset brightness difference upper limit value is 100.
For each other image, the pixel having a pixel value less than or equal to 20 in the luminance difference image corresponding to the other image is a pixel not associated with the ghost region.
For each other image, regarding a pixel associated with a ghost area, each pixel value of which in the brightness difference image corresponding to the other image is greater than 20, when the pixel value of the pixel associated with the ghost area is less than or equal to 100, the pixel value of the pixel is taken as the pixel value of the pixel for normalization, and when the pixel value of the pixel associated with the ghost area is greater than 100, 100 is taken as the pixel value of the pixel for normalization.
For each of the other images, when the normalization processing is performed on the pixel value for normalization of each of the pixels associated with the ghost area, the pixel value for normalization of each of the pixels associated with the ghost area is divided by 100 to obtain a normalized pixel value of the pixel.
In this application, for each other image, a highlight-related part in a ghost mask region of a candidate ghost mask image corresponding to the other image may be set as a non-ghost mask region, so as to obtain a final ghost mask image corresponding to the other image, where the highlight-related part in the ghost mask region of the candidate ghost mask image corresponding to the other image corresponds to a highlight region of a reference image and an image with the largest exposure amount in the other image.
For each of a plurality of images used to generate a high dynamic range image, a highlight region of the image may be detected in advance. For each of a plurality of images used to generate a high dynamic range image, the luminance of each pixel in a highlight region of the image is greater than a highlight luminance threshold, e.g., 200.
For each other image, when the reference image and the image with the largest exposure amount in the other images are the reference image, that is, the exposure amount of the reference image is larger than that of the other images, the highlight relevant part in the ghost mask area of the candidate ghost mask image corresponding to the other images corresponds to the highlight area of the reference image, the candidate ghost mask image corresponding to the other images has the same size as the reference image, and the position of the highlight relevant part in the ghost mask area of the candidate ghost mask image corresponding to the other images in the candidate ghost mask image corresponding to the other images is the same as the position of the highlight area in the reference image.
For example, the plurality of images for generating the high dynamic range image are two images for generating the high dynamic range image among all images continuously taken. Two images for generating a high dynamic range image include: the image processing method comprises the steps of overexposure images and underexposure images, wherein the exposure amount of the overexposure images is larger than that of the underexposure images, the overexposure images are reference images, and the underexposure images are other images.
The highlight-related part in the ghost mask area of the candidate ghost mask image corresponding to the underexposed image corresponds to the highlight area of the overexposed image, and the position of the highlight-related part in the ghost mask area of the candidate ghost mask image corresponding to the underexposed image in the candidate ghost mask image corresponding to the overexposed image is the same as the position of the highlight area in the overexposed image.
For each other image, when the reference image and the image with the largest exposure amount in the other images are the other images, that is, the exposure amount of the other images is larger than that of the reference image, the highlight relevant part in the ghost mask region of the candidate ghost mask image corresponding to the other images corresponds to the highlight region of the other images, and the position of the highlight relevant part in the ghost mask region of the candidate ghost mask image corresponding to the other images in the candidate ghost mask image corresponding to the other images is the same as the position of the highlight region of the other images in the other images.
For each other image, when the highlight-related part in the ghost mask area of the candidate ghost mask image corresponding to the other image is set as the non-ghost mask area, the pixel values of all the pixels of the highlight-related part in the ghost mask area of the candidate ghost mask image corresponding to the other image may be directly set to 0, so that all the pixels of the highlight-related part in the ghost mask area of the candidate ghost mask image corresponding to the other image become pixels which are not related to the ghost area, and the highlight-related part in the ghost mask area of the candidate ghost mask image corresponding to the other image becomes the non-ghost mask area, thereby obtaining a final ghost mask image corresponding to the other image.
The ghost mask region of the final ghost mask image corresponding to the other image is the remaining part of the ghost mask region of the candidate ghost mask image corresponding to the other image except the highlight relevant part.
The non-ghost mask region of the final ghost mask image corresponding to the other image is composed of the non-ghost mask region of the candidate ghost mask image corresponding to the other image and the highlight correlation portion in the ghost mask region of the candidate ghost mask image corresponding to the other image.
In some embodiments, for each other image, performing noise pixel removal processing on the candidate ghost mask image corresponding to the other image to obtain a candidate ghost mask image corresponding to the other image after noise processing; setting highlight relevant parts in ghost mask areas of candidate ghost mask images corresponding to other images as non-ghost mask areas, wherein the highlight relevant parts comprise: and setting highlight relevant parts in the ghost mask areas of the candidate ghost mask images corresponding to the other images after the noise processing as non-ghost mask areas.
In this application, before setting a highlight relevant part in a ghost mask area of a candidate ghost mask image corresponding to another image as a non-ghost mask area, for each other image, noise pixel removal processing may be performed on the candidate ghost mask image corresponding to the other image, so as to obtain a candidate ghost mask image corresponding to the other image after the noise processing.
For each other image, after the candidate ghost mask image corresponding to the other image is subjected to the noise pixel removal processing, the candidate ghost mask image corresponding to the other image may be referred to as a candidate ghost mask image corresponding to the other image after the noise processing, so as to indicate that the candidate ghost mask image corresponding to the other image is subjected to the noise processing.
Noisy pixels may also be referred to as isolated points. For each other image, the performing noise pixel removal processing on the candidate ghost mask image corresponding to the other image may include: and performing erosion operation such as erosion operation of 7 × 7 and expansion operation such as expansion operation of 11 × 11 on the candidate ghost mask image corresponding to the other image.
Step 103, generating a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each other image.
In the application, when a plurality of images are fused based on the final ghost mask image corresponding to each other image to obtain a high dynamic range image, the plurality of images can be fused by using a direct fusion method of different exposure value images of the existing high dynamic range fusion algorithm to obtain a candidate image.
After the candidate image is obtained, a ghost area of the candidate image may be determined based on the final ghost mask image corresponding to each of the other images.
For each other image, the size of the final ghost mask image corresponding to the other image is the same as the size of the candidate image, the ghost mask region of the final ghost mask image corresponding to the other image corresponds to a region of the candidate image, the position of the ghost mask region of the final ghost mask image corresponding to the other image in the final ghost mask image corresponding to the other image is the same as the position of the region corresponding to the ghost mask region in the candidate image, and the region of the candidate image corresponding to the ghost mask region of the final ghost mask image corresponding to the other image is the ghost region of the candidate image or a part of the ghost region of the candidate image.
For the ghost mask region of the final ghost mask image corresponding to each of the other images, a region in the candidate image corresponding to the ghost mask region may be determined. After determining the regions of each of the candidate images corresponding to the ghost mask regions of the final ghost mask image corresponding to the other images, the ghost regions of the candidate images may be determined.
After the ghost region of the candidate image is accurately determined, pixels in the ghost region of the candidate image may be replaced with pixels of a corresponding position in one of the plurality of images, for example, pixels in the ghost region of the candidate image may be replaced with corresponding pixels in a reference image of the plurality of images, thereby eliminating ghosting in the candidate image, so that the ghost region of the candidate image is changed into a non-ghost region corresponding to the ghost region of the candidate image, resulting in a high dynamic range image without ghosting, the luminance of a pixel in the non-ghost region in the high dynamic range image being the luminance of the pixel in the image from which it came.
The pixels of the respective positions in one of the plurality of images are from a region of the image corresponding to a ghost region of the candidate image. The position of the region of the image corresponding to the ghost region of the candidate image in the image is the same as the position of the ghost region of the candidate image in the candidate image.
When replacing a pixel in a ghost region of a candidate image with a pixel of a corresponding position in one image, for each pixel in the ghost region of the candidate image, replacing the pixel with a pixel in the one image corresponding to the pixel, the position of the pixel in the one image in the region of the one image corresponding to the ghost region being the same as the position of the pixel in the ghost region.
When the existing high dynamic range fusion algorithm used for fusing the plurality of images is a high dynamic range fusion algorithm for fusing the plurality of images based on the weight for fusion of each pixel in each image, the existing high dynamic range fusion algorithm calculates the weight for fusion of each pixel in each image.
In the present application, for each of the other images, for each of pixels associated with a ghost region in the ghost mask region of the final ghost mask image corresponding to the other image, the pixel associated with the ghost region corresponding to one pixel in the other image, the position of the pixel associated with the ghost region in the ghost mask region being the same as the position of the pixel in the other image corresponding thereto, the pixel value of the pixel associated with the ghost region is divided by the maximum value of the pixel values of the pixels in the ghost mask region, and according to the obtained result, the weight for fusion of the pixel corresponding to the ghost region is reduced, for example, the obtained result is multiplied by the weight for fusion of the pixel corresponding to the ghost region calculated by the existing high dynamic range fusion algorithm, the final weight for fusion of the pixel corresponding to the pixel associated with the ghost region is obtained.
When the pixel value of each pixel associated with the ghost region in the final ghost mask image corresponding to each other image is the normalized pixel value of each pixel associated with the ghost region for each other image, for each other image, for each pixel associated with a ghost region in the ghost mask region of the corresponding final ghost mask image for that other image, reducing the weight for fusion of the pixel corresponding to the pixel associated with the ghost region according to the normalized pixel value of the pixel associated with the ghost region, for example, the normalized pixel value of the pixel associated with the ghost region is multiplied by the weight for fusion of the pixel corresponding to the pixel associated with the ghost region calculated by the existing high dynamic range fusion algorithm, to obtain the final weight for fusion of the pixel corresponding to the pixel associated with the ghost region.
In some embodiments, for each other image, performing mean filtering processing on the final ghost mask image corresponding to the other image to obtain a final ghost mask image corresponding to the other filtered image; generating a high dynamic range image based on the final ghost mask image and the plurality of images for each of the other images, comprising: and generating a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each filtered other image.
In this application, before generating the high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images, the final ghost mask image corresponding to the other image may be subjected to mean filtering processing for each of the other images to obtain the final ghost mask image corresponding to the other filtered image.
For each other image, after performing the mean filtering process on the final ghost mask image corresponding to the other image, the candidate ghost mask image corresponding to the other image may be referred to as the filtered final ghost mask image corresponding to the other image, so as to indicate that the final ghost mask image corresponding to the other image is subjected to the mean filtering process.
In this application, for each other image, the final ghost mask image corresponding to the other image is subjected to the mean filtering process, and the pixel value of each pixel associated with the ghost area in the ghost mask area of the final ghost mask image corresponding to the other image may be adjusted.
Since the pixel values of the pixels associated with the ghost region in the ghost mask region of the final ghost mask image corresponding to the other image are associated with the weights for fusion of the pixels corresponding to the ghost region in the other image, adjusting the pixel values of the pixels associated with the ghost region in the ghost mask region of the final ghost mask image corresponding to the other image by the mean filtering process is equivalent to adjusting the weights for fusion of the pixels corresponding to the pixels associated with the ghost region in the other image by the mean filtering process. Thus, details of the edge of the candidate image in the ghost area obtained after the fusion of the plurality of images can be made natural.
In some embodiments, generating the high dynamic range image based on the final ghost mask image and the plurality of images for each of the other images comprises: fusing the plurality of images by using a preset high dynamic range fusion algorithm to obtain a candidate image; determining a ghost area of the candidate image based on the final ghost mask image corresponding to each other image; and replacing pixels in the ghost area of the candidate image with pixels in an area corresponding to the ghost area of the candidate image in the target image to obtain the high dynamic range image, wherein the target image is the image with the largest exposure amount in the plurality of images.
The high dynamic range fusion algorithm is preset to be the existing high dynamic range fusion algorithm. The multiple images can be fused by using a preset high dynamic range fusion algorithm based on the final ghost mask image corresponding to each other image to obtain a candidate image. Then, a ghost area of the candidate image is determined based on the final ghost mask image corresponding to each of the other images. And finally, replacing the pixels in the ghost area of the candidate image by the pixels in the area corresponding to the ghost area in the target image, so that the ghost area of the candidate image is converted into a non-ghost area corresponding to the ghost area of the candidate image, and the high-dynamic-range image without ghosts is obtained.
When a pixel in a ghost region of the candidate image is replaced with a pixel in a region of the target image corresponding to the ghost region of the candidate image, the pixel is replaced with a pixel in the target image corresponding to the pixel for each pixel in the ghost region of the candidate image, and the position of the pixel in the region of the target image corresponding to the ghost region in the target image is the same as the position of the pixel in the ghost region of the candidate image.
For example, the plurality of images for generating the high dynamic range image are two images for generating the high dynamic range image among all images continuously taken. Two images for generating a high dynamic range image include: an overexposed image, an underexposed image, an overexposed image as a reference image, and an underexposed image as another image. The target image of the two images used to generate the high dynamic range image is an overexposed image. Firstly, the two images used for generating the high dynamic range image are fused by using a preset high dynamic range fusion algorithm to obtain a candidate image, and then, a ghost area of the candidate image is determined based on a final ghost mask image corresponding to the underexposed image. And finally, replacing the pixels in the ghost area of the candidate image by the pixels in the area corresponding to the ghost area of the candidate image in the overexposed image to obtain the high dynamic range image.
For example, the plurality of images for generating the high dynamic range image are three images for generating the high dynamic range image out of all the images continuously taken. Three images for generating a high dynamic range image include: the image processing method comprises the following steps of overexposed images, normally exposed images and underexposed images, wherein the normally exposed images are reference images, and the underexposed images and the overexposed images are other images. The target image among the three images used to generate the high dynamic range image is an overexposed image. Firstly, the three images used for generating the high dynamic range image are fused by utilizing a preset high dynamic range fusion algorithm to obtain a candidate image, and then, a ghost area of the candidate image is determined based on a final ghost mask image corresponding to the underexposed image and a final ghost mask image corresponding to the overexposed image. And finally, replacing the pixels in the ghost area of the candidate image by the pixels in the area, corresponding to the ghost area of the candidate image, in the overexposed image to obtain the high dynamic range image.
In some embodiments, further comprising: reducing the size of the candidate image by a preset multiple to obtain a small-size candidate image, and reducing the high dynamic range image by the preset multiple to obtain a small-size high dynamic range image; performing Gaussian blur processing on the small-size candidate image to obtain a small-size candidate image after Gaussian blur, and performing Gaussian blur processing on the small-size high dynamic range image to obtain a small-size high dynamic range image after Gaussian blur; generating a brightness difference weight image, wherein the pixel value of a pixel in the brightness difference weight image is a brightness ratio obtained by dividing the brightness of the pixel in the small-size candidate image after Gaussian blur by the brightness of the pixel in the small-size high dynamic range image after Gaussian blur; increasing the size of the brightness difference weight image to the size of the high dynamic range image to obtain a large-size brightness difference weight image; and adjusting the brightness of the pixels in the high dynamic range image based on the large-size brightness difference weight image to obtain the brightness-adjusted high dynamic range image.
Since the non-ghost region in the high dynamic range image corresponding to the ghost region of the candidate image is obtained by replacing the pixels in the ghost region of the candidate image with the corresponding pixels, the luminance of the corresponding pixels in the non-ghost region in the high dynamic range image corresponding to the ghost region of the candidate image is different from the luminance of the pixels in the ghost region of the candidate image, and therefore, a luminance abnormality may occur in the high dynamic range image with respect to the luminance of the non-ghost region in the high dynamic range image corresponding to the ghost region of the candidate image with respect to the luminance of the surrounding region of the non-ghost region corresponding to the ghost region of the candidate image.
After the high dynamic range image is obtained, the brightness abnormality of the non-ghost region corresponding to the ghost region of the candidate image in the high dynamic range image can be eliminated, so that the high dynamic range image has no ghost and no abnormal brightness region, and the brightness of the high dynamic range image is natural.
In order to eliminate the luminance abnormality of the non-ghost region corresponding to the ghost region of the candidate image in the high dynamic range image, the size of the candidate image and the high dynamic range image are first reduced by the same preset multiple, for example, 8 times, resulting in a small-size candidate image and a small-size high dynamic range image.
The content of the candidate image of small size and the number of pixels included are the same as those of the candidate image, except for the size. The image content and the number of pixels included of the small-sized high dynamic range image are the same as the content and the number of pixels included of the high dynamic range image, except for the size.
And performing Gaussian blur processing on the small-size candidate image, and simultaneously performing Gaussian blur processing on the small-size high dynamic range image. The gaussian blur process may also be referred to as a gaussian smoothing process.
After the small-size candidate image is subjected to the gaussian blurring process, the small-size candidate image may be referred to as a gaussian-blurred small-size candidate image to indicate that the small-size candidate image is subjected to the gaussian blurring process. After the small-size high dynamic range image is subjected to the gaussian blur process, the small-size high dynamic range image may be referred to as a gaussian-blurred small-size high dynamic range image to indicate that the small-size high dynamic range image is subjected to the gaussian blur process.
Then, a luminance difference weight image is generated based on the gaussian-blurred small-size candidate image and the gaussian-blurred small-size high dynamic range image.
For each pixel in the brightness difference weight image, the pixel value of the pixel is a brightness ratio obtained by dividing the brightness of the pixel corresponding to the pixel in the small-size candidate image after gaussian blurring by the brightness of the pixel corresponding to the pixel in the small-size high dynamic range image after gaussian blurring.
For each pixel in the brightness difference weight image, the position of the pixel in the brightness difference weight image is the same as the position of the pixel corresponding to the pixel in the small-size candidate image after the gaussian blur in the small-size candidate image.
For each pixel in the brightness difference weight image, the position of the pixel in the brightness difference weight image is the same as the position of the pixel corresponding to the pixel in the small-size high dynamic range image after the gaussian blur.
And finally, increasing the size of the brightness difference weight image to the size of the high dynamic range image to obtain a large-size brightness difference weight image, and adjusting the brightness of the pixels in the high dynamic range image based on the large-size brightness difference weight image.
When the brightness of the pixel in the high dynamic range image is adjusted based on the large-size brightness difference weight image, for each pixel in the high dynamic range image, the original brightness of the pixel may be multiplied by the pixel value of the pixel corresponding to the pixel in the brightness difference weight image, so as to obtain the adjusted brightness of the pixel.
After adjusting the luminance of the pixels in the high dynamic range image based on the large-size luminance difference weight image, each pixel in the high dynamic range image has an adjusted luminance.
The brightness of the pixels in the high dynamic range image is adjusted based on the large-size brightness difference weight image, so that the brightness of the high dynamic range image and the brightness of the candidate image with natural brightness can reach the same brightness level, and therefore the brightness abnormity of the non-ghost area corresponding to the ghost area of the candidate image in the high dynamic range image is eliminated, the high dynamic range image has no ghost or abnormal brightness area, and the brightness of the high dynamic range image is natural.
Please refer to fig. 2, which shows a flowchart of the high dynamic range image generation method provided in the present application.
The plurality of images for generating the high dynamic range image are two images for generating the high dynamic range image among all images continuously taken. Two images for generating a high dynamic range image include: the image processing method comprises the steps of overexposure images and underexposure images, wherein the exposure amount of the overexposure images is larger than that of the underexposure images, the overexposure images are reference images, and the underexposure images are other images.
The reference image may be referred to as an ev + image. The other images may be referred to as ev-images.
In step 201, the brightness of the ev-image is adjusted through histogram matching so that the brightness of the ev-image and the brightness of the ev + image reach the same brightness level.
Step 202, calculating the brightness difference value of two pixels at the same position from the ev-image and the ev + image to obtain a brightness difference image, and generating a candidate ghost mask image, namely a candidate ghost mask, based on the brightness difference image.
And step 203, detecting the highlight region of the ev + image, and setting the pixel value of each pixel in the highlight relevant part corresponding to the highlight region of the ev + image in the candidate ghost mask to be 0 to obtain the final ghost mask.
And step 204, fusing the ev + image and the ev-image to obtain a candidate image, namely dst1, and determining a ghost area of dst1 based on the final ghost mask.
In step 205, pixels in the ghost area of dst1 are replaced by pixels at corresponding positions in the ev + image, and a high dynamic range image, namely dst2, is obtained.
In step 206, the dst1 and dst2 are reduced by a preset multiple, and the small-sized dst1 and dst2 are subjected to gaussian smoothing processing, so that a brightness difference weight image, namely mask2, is obtained based on the small-sized dst1 and dst 2.
And step 207, increasing the size of the mask2 to the size of the dst2 to obtain a large-size mask2, and eliminating the brightness abnormality of the brightness abnormality area based on the large-size mask2 and the dst 2. The luminance abnormal region is a non-ghost region in the dst2 corresponding to a ghost region of the candidate image. So that dst2 has no ghost image and no abnormal brightness region, and the brightness of dst2 is natural.
Referring to fig. 3, a block diagram of a high dynamic range image generating apparatus provided in the present application is shown. The image generation apparatus includes: an acquisition unit 301, a processing unit 302, and a generation unit 303.
The acquisition unit 301 is configured to acquire a plurality of images for generating a high dynamic range image, the plurality of images including: a reference image, at least one other image;
the processing unit 302 is configured to, for each other image, adjust the brightness of the other image by histogram matching; generating a candidate ghost mask image corresponding to the other image based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image; setting a highlight relevant part in a ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area to obtain a final ghost mask image corresponding to the other image, wherein the highlight relevant part corresponds to highlight areas of the reference image and the image with the maximum exposure in the other image;
the generating unit 303 is configured to generate a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images.
In some embodiments, the processing unit 302 includes:
a normalization module configured to calculate a normalized pixel value of each pixel associated with a ghost region in the luminance difference image corresponding to the other image, wherein the pixel associated with the ghost region is a pixel whose pixel value is greater than a luminance difference threshold value; setting the pixel value of each pixel associated with the ghost area as the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel in the brightness difference image corresponding to the other image, which is not associated with the ghost area, as 0 to obtain a candidate ghost mask image corresponding to the other image, wherein the pixel not associated with the ghost area is a pixel of which the pixel value is less than or equal to the brightness difference threshold value.
In some embodiments, the normalization module is further configured to determine, for each pixel associated with a ghost region in the luminance difference image corresponding to the other image, a pixel value of the pixel associated with the ghost region for normalization based on whether the pixel value of the pixel associated with the ghost region is greater than a preset luminance difference value upper limit value; and performing normalization processing on the pixel value used for normalization of each pixel associated with the ghost area to obtain a normalized pixel value of each pixel associated with the ghost area.
In some embodiments, the image generation apparatus further comprises:
the denoising unit is configured to perform noise pixel removal processing on the candidate ghost mask image corresponding to each other image to obtain a candidate ghost mask image corresponding to the other image after the noise processing; the processing unit 302 is further configured to set a highlight-related part in a ghost mask region of the candidate ghost mask image corresponding to the other image after the noise processing as a non-ghost mask region.
In some embodiments, the image generation apparatus further comprises:
the filtering unit is configured to perform mean filtering processing on the final ghost mask image corresponding to each other image to obtain a final ghost mask image corresponding to the other filtered image; the processing unit 302 is further configured to generate a high dynamic range image based on the final ghost mask image and the plurality of images for each of the filtered other images.
In some embodiments, the generating unit 303 comprises:
the high dynamic range image generation module is configured to fuse the plurality of images by using a preset high dynamic range fusion algorithm to obtain a candidate image; determining a ghost area of the candidate image based on the final ghost mask image corresponding to each other image; and replacing pixels in the ghost area of the candidate image by pixels in an area corresponding to the ghost area of the candidate image in the target image to obtain the high dynamic range image, wherein the target image is the image with the largest exposure amount in the plurality of images.
In some embodiments, the image generation apparatus further comprises:
a brightness abnormality eliminating unit configured to reduce the size of the candidate image by a preset multiple to obtain a small-size candidate image, and reduce the high dynamic range image by the preset multiple to obtain a small-size high dynamic range image; performing Gaussian blur processing on the small-size candidate image to obtain a small-size candidate image after Gaussian blur, and performing Gaussian blur processing on the small-size high dynamic range image to obtain a small-size high dynamic range image after Gaussian blur; generating a brightness difference weight image, wherein the pixel value of a pixel in the brightness difference weight image is a brightness ratio obtained by dividing the brightness of the pixel in the small-size candidate image after Gaussian blur by the brightness of the pixel in the small-size high dynamic range image after Gaussian blur; increasing the size of the brightness difference weight image to the size of the high dynamic range image to obtain a large-size brightness difference weight image; and adjusting the brightness of the pixels in the high dynamic range image based on the large-size brightness difference weight image to obtain the brightness-adjusted high dynamic range image.
Fig. 4 is a block diagram of an electronic device provided in this embodiment. Electronic device 400 includes a processing component 422 that further includes one or more processors, and memory resources, represented by memory 432, for storing instructions, such as application programs, that are executable by processing component 422. The application programs stored in memory 432 may include one or more modules that each correspond to a set of instructions. Further, the processing component 422 is configured to execute instructions to perform the above-described methods.
The electronic device 400 may further include a power supply component 426 configured to perform power management of the electronic device 400, a wired or wireless network interface 450 configured to connect the electronic device 400 to a network, and an input-output (I/O) interface 458 the electronic device 400 may be operable based on an operating system stored in the memory 432, such as windows server (tm), MacOS XTM, UnixTM, &lttttranslation = & &gtt translation & &l &/t &gtt &inux (tm), FreeBSDTM, or the like.
In an exemplary embodiment, a storage medium comprising instructions, such as a memory comprising instructions, executable by an electronic device to perform the above method is also provided. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (10)

1. A method of high dynamic range image generation, the method comprising:
acquiring a plurality of images for generating a high dynamic range image, the plurality of images comprising: a reference image, at least one other image;
for each other image, adjusting the brightness of the other image through histogram matching; generating a candidate ghost mask image corresponding to the other image based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image; setting a highlight relevant part in a ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area to obtain a final ghost mask image corresponding to the other image, wherein the highlight relevant part corresponds to highlight areas of the reference image and the image with the maximum exposure in the other image;
a high dynamic range image is generated based on the final ghost mask image and the plurality of images for each of the other images.
2. The method of claim 1, wherein generating the candidate ghost mask image corresponding to the other image based on the luminance-adjusted difference between the other image and the corresponding pixel of the reference image comprises:
calculating a normalized pixel value of each pixel associated with a ghost area in the brightness difference image corresponding to the other image, wherein the pixel associated with the ghost area is a pixel of which the pixel value is greater than a brightness difference threshold value;
setting the pixel value of each pixel associated with the ghost area as the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel in the brightness difference image corresponding to the other image, which is not associated with the ghost area, as 0 to obtain a candidate ghost mask image corresponding to the other image, wherein the pixel not associated with the ghost area is a pixel of which the pixel value is less than or equal to the brightness difference threshold value.
3. The method of claim 2, wherein the calculating a normalized pixel value for each pixel associated with a ghost region in the luminance difference image corresponding to the other image comprises:
for each pixel associated with a ghost area in the brightness difference image corresponding to the other image, determining a pixel value of the pixel associated with the ghost area for normalization based on whether the pixel value of the pixel associated with the ghost area is greater than a preset brightness difference upper limit value;
and performing normalization processing on the pixel value used for normalization of each pixel associated with the ghost area to obtain a normalized pixel value of each pixel associated with the ghost area.
4. The method of claim 1, further comprising:
for each other image, performing noise pixel removal processing on the candidate ghost mask image corresponding to the other image to obtain a candidate ghost mask image corresponding to the other image after the noise processing, wherein the noise pixel removal processing is used for removing noise pixels in the candidate ghost mask image corresponding to the other image;
the setting the highlight relevant part in the ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area includes: and setting a highlight relevant part in a ghost mask area of the candidate ghost mask image corresponding to the other image after the noise processing as a non-ghost mask area.
5. The method of claim 4, further comprising:
for each other image, performing mean filtering processing on the final ghost mask image corresponding to the other image to obtain a final ghost mask image corresponding to the other image after filtering;
generating a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images, comprising: and generating a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each other image after filtering.
6. The method of any one of claims 1-5, wherein generating the high dynamic range image based on the final ghost mask image and the plurality of images for each of the other images comprises:
fusing the plurality of images by using a preset high dynamic range fusion algorithm to obtain candidate images;
determining a ghost area of the candidate image based on the final ghost mask image corresponding to each other image;
and replacing pixels in the ghost area of the candidate image by pixels in an area corresponding to the ghost area of the candidate image in the target image to obtain the high dynamic range image, wherein the target image is the image with the largest exposure amount in the plurality of images.
7. The method of claim 6, further comprising:
reducing the size of the candidate image by a preset multiple to obtain a small-size candidate image, and reducing the high dynamic range image by the preset multiple to obtain a small-size high dynamic range image;
performing Gaussian blur processing on the small-size candidate image to obtain a small-size candidate image after Gaussian blur, and performing Gaussian blur processing on the small-size high dynamic range image to obtain a small-size high dynamic range image after Gaussian blur;
generating a brightness difference weight image, wherein the pixel value of a pixel in the brightness difference weight image is a brightness ratio obtained by dividing the brightness of the pixel in the small-size candidate image after Gaussian blur by the brightness of the pixel in the small-size high dynamic range image after Gaussian blur;
increasing the size of the brightness difference weight image to the size of the high dynamic range image to obtain a large-size brightness difference weight image;
and adjusting the brightness of the pixels in the high dynamic range image based on the large-size brightness difference weight image to obtain the brightness-adjusted high dynamic range image.
8. A high dynamic range image generation apparatus, characterized in that the apparatus comprises:
an acquisition unit configured to acquire a plurality of images for generating a high dynamic range image, the plurality of images including: a reference image, at least one other image;
a processing unit configured to adjust, for each of the other images, the brightness of the other image by histogram matching; generating a candidate ghost mask image corresponding to the other image based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image; setting a highlight relevant part in a ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area to obtain a final ghost mask image corresponding to the other image, wherein the highlight relevant part corresponds to highlight areas of the reference image and the image with the maximum exposure in the other image;
a generating unit configured to generate a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images.
9. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of any one of claims 1 to 7.
10. A storage medium having instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the method of any of claims 1-7.
CN202010183368.1A 2020-03-16 2020-03-16 High dynamic range image generation method, device, electronic equipment and storage medium Active CN111429369B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010183368.1A CN111429369B (en) 2020-03-16 2020-03-16 High dynamic range image generation method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010183368.1A CN111429369B (en) 2020-03-16 2020-03-16 High dynamic range image generation method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111429369A true CN111429369A (en) 2020-07-17
CN111429369B CN111429369B (en) 2023-07-28

Family

ID=71549512

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010183368.1A Active CN111429369B (en) 2020-03-16 2020-03-16 High dynamic range image generation method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111429369B (en)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3436943A1 (en) * 1983-11-29 1985-06-05 Veb Pentacon Dresden Kamera- Und Kinowerke, Ddr 8021 Dresden Exposure measuring circuit
US5983043A (en) * 1997-12-24 1999-11-09 Canon Kabushiki Kaisha Potential measuring apparatus and potential measuring method
CN1607818A (en) * 2003-10-15 2005-04-20 富士通天株式会社 Image processing device, operation supporting device, and operation supporting system
CN103002225A (en) * 2011-04-20 2013-03-27 Csr技术公司 Multiple exposure high dynamic range image capture
US20180352132A1 (en) * 2017-05-31 2018-12-06 Guangdong Oppo Mobile Telecommunications Corp., Lt D. Image processing method and related products
CN109587395A (en) * 2018-11-23 2019-04-05 福建工程学院 It is a kind of based on image procossing and virtually that is enhanced prevent the system and its implementation of dazzle
CN110869695A (en) * 2017-04-19 2020-03-06 Drs网络和成像系统公司 Active hyperspectral imaging equipment

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3436943A1 (en) * 1983-11-29 1985-06-05 Veb Pentacon Dresden Kamera- Und Kinowerke, Ddr 8021 Dresden Exposure measuring circuit
US5983043A (en) * 1997-12-24 1999-11-09 Canon Kabushiki Kaisha Potential measuring apparatus and potential measuring method
CN1607818A (en) * 2003-10-15 2005-04-20 富士通天株式会社 Image processing device, operation supporting device, and operation supporting system
CN103002225A (en) * 2011-04-20 2013-03-27 Csr技术公司 Multiple exposure high dynamic range image capture
CN110869695A (en) * 2017-04-19 2020-03-06 Drs网络和成像系统公司 Active hyperspectral imaging equipment
US20180352132A1 (en) * 2017-05-31 2018-12-06 Guangdong Oppo Mobile Telecommunications Corp., Lt D. Image processing method and related products
CN109587395A (en) * 2018-11-23 2019-04-05 福建工程学院 It is a kind of based on image procossing and virtually that is enhanced prevent the system and its implementation of dazzle

Also Published As

Publication number Publication date
CN111429369B (en) 2023-07-28

Similar Documents

Publication Publication Date Title
CN108335279B (en) Image fusion and HDR imaging
CN111028189B (en) Image processing method, device, storage medium and electronic equipment
CN110121882B (en) Image processing method and device
CN109936698B (en) Automatic exposure control method and device, electronic equipment and storage medium
CN110599433B (en) Double-exposure image fusion method based on dynamic scene
US9479705B2 (en) Exposure value adjustment apparatus, method, and non-transitory tangible machine-readable medium thereof
CN110248105B (en) Image processing method, camera and computer storage medium
CN111915505B (en) Image processing method, device, electronic equipment and storage medium
KR20110138114A (en) Method and apparatus for processing frames obtained by multi-exposure
WO2017047494A1 (en) Image-processing device
CN109413335B (en) Method and device for synthesizing HDR image by double exposure
CN112738411B (en) Exposure adjusting method, exposure adjusting device, electronic equipment and storage medium
CN110443766B (en) Image processing method and device, electronic equipment and readable storage medium
CN113344821B (en) Image noise reduction method, device, terminal and storage medium
Guthier et al. Flicker reduction in tone mapped high dynamic range video
Pourreza-Shahri et al. Exposure bracketing via automatic exposure selection
US9338354B2 (en) Motion blur estimation and restoration using light trails
Kinoshita et al. Pseudo multi-exposure fusion using a single image
KR101336240B1 (en) Method and apparatus for image processing using saved image
WO2020206659A1 (en) Method and apparatus for combining low-dynamic range images to a single image
US20200007732A1 (en) Image processing method and electronic device
CN111429369B (en) High dynamic range image generation method, device, electronic equipment and storage medium
Chung et al. Under-exposed image enhancement using exposure compensation
CN115496673A (en) Image processing method, image processing apparatus, image processing system, and storage medium
CN110689502B (en) Image processing method and related device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant