CN111429369B - High dynamic range image generation method, device, electronic equipment and storage medium - Google Patents

High dynamic range image generation method, device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111429369B
CN111429369B CN202010183368.1A CN202010183368A CN111429369B CN 111429369 B CN111429369 B CN 111429369B CN 202010183368 A CN202010183368 A CN 202010183368A CN 111429369 B CN111429369 B CN 111429369B
Authority
CN
China
Prior art keywords
image
ghost
pixel
images
candidate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010183368.1A
Other languages
Chinese (zh)
Other versions
CN111429369A (en
Inventor
王涛
李绪琴
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Megvii Technology Co Ltd
Original Assignee
Beijing Megvii Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Megvii Technology Co Ltd filed Critical Beijing Megvii Technology Co Ltd
Priority to CN202010183368.1A priority Critical patent/CN111429369B/en
Publication of CN111429369A publication Critical patent/CN111429369A/en
Application granted granted Critical
Publication of CN111429369B publication Critical patent/CN111429369B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G06T5/90—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/741—Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20208—High dynamic range [HDR] image processing

Abstract

The embodiment of the application provides a high dynamic range image generation method and device, wherein the method comprises the following steps: acquiring a reference image and at least one other image, and for each other image, adjusting the brightness of the other image through histogram matching; generating candidate ghost mask images corresponding to other images based on the difference between the other images after brightness adjustment and the corresponding pixels of the reference image; setting a highlight association part in a ghost mask area of the candidate ghost mask images corresponding to other images as a non-ghost mask area to obtain a final ghost mask image corresponding to the other images; a high dynamic range image is generated based on the final ghost mask image and the plurality of images corresponding to each of the other images. The highlight region is prevented from being determined as a ghost region, and the occurrence of abnormal brightness of the highlight region in the highlight image, for example, the highlight compression condition in which the brightness of the highlight region is darkened, improves the quality of the generated highlight image.

Description

High dynamic range image generation method, device, electronic equipment and storage medium
Technical Field
The present invention relates to the field of image processing, and in particular, to a method and apparatus for generating a high dynamic range image, an electronic device, and a storage medium.
Background
The high dynamic range (high dynamic range, HDR) fusion algorithm is used for fusing a plurality of images of different exposure of the same scene shot by the camera to obtain a high dynamic range image, thereby greatly improving the quality of the image shot in night scenes. When moving objects appear in a photographed scene, a high dynamic range image obtained by fusion may generate a ghost, which may also be referred to as an artifact, and it is necessary to remove the ghost in the high dynamic range image.
At present, the general method for removing the ghosts is as follows: the brightness of the images with different exposure amounts is adjusted to the same brightness, the brightness difference between pixels at the same position of the images with different exposure amounts is calculated, the brightness difference between pixels at the same position of the images with different exposure amounts is compared with a threshold value, and if the brightness difference between pixels at the same position of the images with different exposure amounts is larger than the threshold value, the same position is a position belonging to a ghost area. After determining the position of each belonging to the ghost area, the ghost area of the high dynamic range image is determined, and then, the ghosts in the high dynamic range image are removed.
However, the luminance difference between the plurality of pixels at the same position in the highlight region from the images of different exposure amounts is generally large, resulting in that the highlight region is erroneously determined as a ghost region according to the comparison result of the luminance difference between the plurality of pixels at the same position in the highlight region from the images of different exposure amounts and the threshold value. And the highlight region is not ghosted, the highlight region is erroneously determined to belong to the ghost region, so that the brightness of the pixels in the highlight region is adjusted in a manner of removing the ghost, and the ghost is removed, and the abnormal brightness condition of the highlight region in the high dynamic range image, such as brightness darkening and high light compression condition, is caused.
Disclosure of Invention
In order to overcome the problems in the related art, the application provides a high dynamic range image generation method, a device, an electronic device and a storage medium.
According to a first aspect of embodiments of the present application, there is provided a high dynamic range image generating method, including:
acquiring a plurality of images for generating a high dynamic range image, the plurality of images comprising: a reference image, at least one other image;
for each other image, adjusting the brightness of the other image through histogram matching; generating candidate ghost mask images corresponding to the other images based on the difference between the corresponding pixels of the other images and the reference image after brightness adjustment; setting a highlight association part in a ghost mask region of the candidate ghost mask image corresponding to the other image as a non-ghost mask region to obtain a final ghost mask image corresponding to the other image, wherein the highlight association part corresponds to a highlight region of an image with the largest exposure in the reference image and the other image;
A high dynamic range image is generated based on the final ghost mask image and the plurality of images corresponding to each of the other images.
According to a second aspect of embodiments of the present application, there is provided a high dynamic range image generating apparatus including:
an acquisition unit configured to acquire a plurality of images for generating a high dynamic range image, the plurality of images including: a reference image, at least one other image;
a processing unit configured to adjust, for each other image, the brightness of the other image by histogram matching; generating candidate ghost mask images corresponding to the other images based on the difference between the corresponding pixels of the other images and the reference image after brightness adjustment; setting a highlight association part in a ghost mask region of the candidate ghost mask image corresponding to the other image as a non-ghost mask region to obtain a final ghost mask image corresponding to the other image, wherein the highlight association part corresponds to a highlight region of an image with the largest exposure in the reference image and the other image;
and a generation unit configured to generate a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images.
The high dynamic range image generation method and device provided by the application are characterized in that a plurality of images for generating the high dynamic range image are acquired, and the plurality of images comprise: a reference image, at least one other image; for each other image, adjusting the brightness of the reference image or the brightness of the other image through histogram matching; generating candidate ghost mask images corresponding to the other images based on the difference between the corresponding pixels of the other images and the reference image after brightness adjustment; setting a highlight association part in a ghost mask region of the candidate ghost mask image corresponding to the other image as a non-ghost mask region to obtain a final ghost mask image corresponding to the other image, wherein the highlight association part corresponds to a highlight region of an image with the largest exposure in the reference image and the other image; and fusing the multiple images based on the final ghost mask image corresponding to each other image to obtain a high dynamic range image. It is achieved that the highlight-related portion in the ghost mask region of the candidate ghost mask image is set as a non-ghost mask region, and a final ghost mask image is obtained, so that the actual ghost region in the candidate image is accurately determined using the final ghost mask image, so that the highlight region in the candidate image is not determined as a ghost region, the brightness of the pixels in the highlight region in the candidate image is not adjusted in such a manner that the ghost is removed, and the situation of abnormal brightness of the highlight region in the high dynamic range image, such as a highlight compression situation in which the brightness of the highlight region becomes dark, is avoided.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
FIG. 1 illustrates a flow chart of one embodiment of a high dynamic range image generation method provided herein;
FIG. 2 is a schematic flow chart of the high dynamic range image generation method provided by the present application;
fig. 3 shows a block diagram of a high dynamic range image generating apparatus provided by the present application;
fig. 4 shows a block diagram of the electronic device provided by the present application.
Detailed Description
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the associated invention and are not limiting of the invention. It should be further noted that, for convenience of description, only a portion related to the present invention is shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
FIG. 1 illustrates a flow chart of one embodiment of a high dynamic range method provided herein, the method comprising:
Step 101, a plurality of images for generating a high dynamic range image are acquired.
In the present application, a plurality of images for generating a high dynamic range image may be selected from all images of the same scene taken in succession. The plurality of images for generating the high dynamic range image includes: reference image, at least one other image.
In this application, for each of a plurality of images for generating a high dynamic range image, the exposure degree of the image refers to the exposure amount employed at the time of photographing the image. The exposure amount of each of the plurality of images for generating the high dynamic range image is different.
For example, the plurality of images for generating the high dynamic range image are images for generating the high dynamic range image of two of all images continuously photographed. The two images for generating the high dynamic range image include: an overexposed image, an underexposed image, the overexposed image having a greater exposure than the underexposed image. The overexposed image is a reference image, and the underexposed image is another image.
For another example, the plurality of images are images for generating a high dynamic range image of three of all images continuously photographed. The three images for generating the high dynamic range image include: an underexposed image, a normally exposed image, an overexposed image, the overexposed image having an exposure greater than the normally exposed image, the normally exposed image having an exposure greater than the underexposed image. The normally exposed image is a reference image, and the underexposed image and the overexposed image are the exposure amounts of other images.
Step 102, a final ghost mask image corresponding to each other image is generated.
In this application, in order to generate a final ghost mask image corresponding to each other image, the final ghost mask image corresponding to each other image may be obtained by: adjusting the brightness of the other images through histogram matching; generating candidate ghost mask images corresponding to the other images based on the difference between the other images after brightness adjustment and the corresponding pixels of the reference image; and setting a highlight association part in a ghost mask region of the candidate ghost mask image corresponding to the other image as a non-ghost mask region to obtain a final ghost mask image corresponding to the other image, wherein the highlight association part in the ghost mask region of the candidate ghost mask image corresponding to the other image corresponds to a highlight region of the image with the largest exposure in the reference image and the other image.
For each other image, the brightness of the other image is adjusted through histogram matching, so that the brightness of the reference image and the brightness of the other image reach the same brightness level.
In other words, the brightness of the reference image is kept unchanged, and the brightness of each other image is targeted at the brightness of the reference image when the brightness of each other image is adjusted, so that the brightness of the plurality of images is adjusted to the same standard after the brightness of each other image is adjusted.
In this application, for each other image, after the brightness of the other image is adjusted, the brightness of the other image, that is, the brightness of the pixels in the other image, may be referred to as the other image after the brightness adjustment, so as to indicate that the other image is subjected to the brightness adjustment.
For each other image, when generating a candidate ghost mask image corresponding to the other image based on the difference between the other image after brightness adjustment and the corresponding pixel of the reference image, a brightness difference image corresponding to the other image may be generated, where the brightness difference image corresponding to the other image describes the difference between the other image and the corresponding pixel of the reference image; and generating candidate ghost mask images corresponding to the other images based on the brightness difference images corresponding to the other images.
For each other image, the pixel value of the pixel in the luminance difference image corresponding to the other image is the luminance difference between the luminance of the first pixel in the reference image and the luminance of the second pixel corresponding to the first pixel in the other image.
Each pixel in the reference image may be referred to as a first pixel. Each pixel in any one of the other images may be referred to as a second pixel.
For each first pixel in the reference image, the first pixel corresponds to one second pixel in the other image.
For each first pixel, the position of the first pixel in the reference image is the same as the position of the second pixel corresponding to the first pixel in the other image.
In the present application, for each other image, a candidate ghost mask image corresponding to the other image may be generated based on the luminance difference image corresponding to the other image.
For each other image, when the exposure of the reference image is greater than the exposure of the other image, the brightness of a first pixel in the reference image may be subtracted from the brightness of a second pixel in the other image corresponding to the first pixel to obtain a brightness difference between the brightness of the first pixel and the brightness of the second pixel in the other image corresponding to the first pixel. When the exposure of the other image is larger than the exposure of the reference image, the brightness of the second pixel corresponding to the first pixel in the reference image in the other image is subtracted from the brightness of the first pixel in the reference image to obtain a brightness difference value between the brightness of the first pixel and the brightness of the second pixel corresponding to the first pixel in the other image. Thus, the brightness difference values are all larger than zero. The subtraction may be directly performed on two luminances for calculating the luminance difference value without considering the exposure amount, and in the case where the obtained calculation result is negative, the absolute value of the calculation result, that is, the magnitude of the difference between the luminance of the first pixel in the reference image and the luminance of the second pixel corresponding to the first pixel in the other image may be used as the luminance difference value.
For each other image, after computing the difference between the reference image and the corresponding pixel of the other image, a luminance difference image corresponding to the other image may be generated, where the pixel value of each pixel in the luminance difference image corresponding to the other image is the luminance difference between the first pixel and the second pixel of the corresponding location.
It should be understood that the luminance of the second pixel corresponding to the first pixel in the other image described above refers to the luminance of the second pixel corresponding to the first pixel in the other image after the luminance adjustment is performed on the other image.
In the present application, for each other image, a candidate ghost mask image corresponding to the other image may be generated based on the luminance difference image corresponding to the other image.
When the candidate ghost mask image corresponding to the other image is generated based on the luminance difference image corresponding to the other image, it may be determined, for each pixel in the luminance difference image corresponding to the other image, whether a pixel value of the pixel is greater than a luminance difference threshold, where the pixel value of the pixel is an absolute value of a luminance difference between the luminance of the first pixel corresponding to the pixel and the luminance of the second pixel corresponding to the first pixel after the luminance adjustment. The position of the first pixel corresponding to the pixel in the reference image is the same as the position of the pixel in the brightness difference image corresponding to the other images. When the pixel value of a pixel in the luminance difference image corresponding to the other image is greater than the luminance difference threshold, the pixel value of the pixel may be maintained. When the pixel value of a pixel in the luminance difference image corresponding to the other image is less than or equal to the luminance difference threshold value, the pixel value of the pixel may be set to 0.
For each other image, after determining, for each pixel in the luminance difference image corresponding to the other image, the pixel value of the pixel to be held or set to 0, a candidate ghost mask image corresponding to the other image may be obtained.
For each other image, the candidate ghost mask image corresponding to the other image includes: a ghost mask region and a non-ghost mask region.
For each other image, the pixel value of each pixel in the ghost mask area of the candidate ghost mask image corresponding to the other image is a non-0 value.
For each other image, the pixel value of each pixel in the non-ghost mask area of the candidate ghost mask image corresponding to that other image is 0.
In some embodiments, for each other image, generating a candidate ghost mask image corresponding to the other image based on differences in corresponding pixels of the other image and the reference image after brightness adjustment includes: calculating the normalized pixel value of each pixel associated with the ghost area in the brightness difference images corresponding to the other images, wherein the pixel associated with the ghost area is a pixel with a pixel value greater than a brightness difference threshold; setting the pixel value of each pixel associated with the ghost area as the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel not associated with the ghost area in the brightness difference image corresponding to the other image as 0, thereby obtaining a candidate ghost mask image corresponding to the other image, wherein the pixels not associated with the ghost area are pixels with pixel values less than or equal to the brightness difference threshold.
For each of the other images, pixels in the luminance difference image corresponding to the other image having a pixel value greater than the luminance difference threshold may be referred to as pixels associated with the ghost area, and pixels in the luminance difference image corresponding to the other image having a pixel value less than or equal to the luminance difference threshold may be referred to as pixels not associated with the ghost area.
For each other image, when calculating the normalized pixel value of each pixel associated with the ghost area in the luminance difference image corresponding to the other image, the pixel value of the pixel associated with the ghost area may be divided by the maximum value of the pixel values of the pixels in the luminance difference image corresponding to the other image for each pixel associated with the ghost area in the luminance difference image corresponding to the other image, and the obtained result may be taken as the normalized pixel value of the pixel associated with the ghost area.
For each other image, after setting the pixel value of each pixel associated with the ghost area in the luminance difference image corresponding to the other image to the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel not associated with the ghost area to 0, a candidate ghost mask image corresponding to the other image is obtained.
For each other image, the candidate ghost mask image corresponding to the other image includes a ghost mask region, a non-ghost mask region. Pixels in the ghost mask areas of the candidate ghost mask images corresponding to the other images are pixels associated with the ghost areas, and pixels in the non-ghost mask areas of the candidate ghost mask images corresponding to the other images are pixels not associated with the ghost areas.
The pixel value of each pixel associated with the ghost area in the ghost mask areas of the candidate ghost mask images corresponding to the other images is the normalized pixel value of the pixel associated with the ghost area. The pixel value of each of the pixels of the candidate ghost mask image corresponding to the other image, which are unassociated with the ghost region, is 0.
In some embodiments, for each other image, calculating a normalized pixel value for each pixel associated with the ghost region in the luminance difference image corresponding to the other image comprises: for each pixel associated with the ghost area in the luminance difference images corresponding to the other images, determining a pixel value for normalization of the pixel associated with the ghost area based on whether the pixel value of the pixel associated with the ghost area is greater than a preset luminance difference upper value; and carrying out normalization processing on the pixel values for normalization of each pixel associated with the ghost area to obtain normalized pixel values of each pixel associated with the ghost area.
In order to calculate the normalized pixel value of the pixel associated with the ghost area in each of the luminance difference images corresponding to the other images, a preset luminance difference upper limit value may be preset.
For each other image, regarding each pixel associated with the ghost area in the luminance difference image corresponding to the other image, when the pixel value of the pixel associated with the ghost area is less than or equal to the preset luminance difference upper limit value, the pixel value of the pixel is used as the pixel value for normalization of the pixel, and when the pixel value of the pixel associated with the ghost area is greater than the preset luminance difference upper limit value, the preset luminance difference upper limit value is used as the pixel value for normalization of the pixel.
For each other image, when normalization processing is performed on the pixel value for normalization of each pixel associated with the ghost area, the pixel value for normalization of the pixel is divided by a preset luminance difference upper limit value for each pixel associated with the ghost area, and a normalized pixel value of the pixel is obtained.
For example, the luminance difference threshold is 20, and the preset luminance difference upper limit is 100.
For each other image, for each pixel value in the luminance difference image corresponding to the other image, less than or equal to 20 pixels, which are pixels not associated with the ghost area.
For each other image, for each pixel value in the luminance difference image corresponding to the other image, greater than 20 pixels associated with the ghost area, when the pixel value of the pixel associated with the ghost area is less than or equal to 100, the pixel value of the pixel is taken as the pixel value of the pixel for normalization, and when the pixel value of the pixel associated with the ghost area is greater than 100, 100 is taken as the pixel value of the pixel for normalization.
For each other image, when normalization processing is performed on the pixel value for normalization of each pixel associated with the ghost area, the pixel value for normalization of the pixel is divided by 100 for each pixel associated with the ghost area, resulting in a normalized pixel value for the pixel.
In the present application, for each other image, the highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the other image may be set as a non-ghost mask region, so as to obtain a final ghost mask image corresponding to the other image, where the highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the other image corresponds to the highlight region of the image with the largest exposure in the reference image and the other image.
For each of a plurality of images for generating a high dynamic range image, a high light area of the image may be detected in advance. For each of a plurality of images used to generate a high dynamic range image, the brightness of each pixel in the high light area of the image is greater than a high brightness threshold, such as 200.
For each other image, when the reference image and the image with the largest exposure in the other images are the reference image, that is, when the exposure of the reference image is larger than that of the other images, the highlight-related part in the ghost mask region of the candidate ghost mask image corresponding to the other image corresponds to the highlight region of the reference image, the candidate ghost mask image corresponding to the other image is the same as the size of the reference image, and the position of the highlight-related part in the ghost mask region of the candidate ghost mask image corresponding to the other image in the candidate ghost mask image corresponding to the other image is the same as the position of the highlight region in the reference image.
For example, the plurality of images for generating the high dynamic range image are images for generating the high dynamic range image of two of all images continuously photographed. The two images for generating the high dynamic range image include: the method comprises the steps of exposing an overexposed image and an underexposed image, wherein the exposure of the overexposed image is larger than that of the underexposed image, the overexposed image is a reference image, and the underexposed image is other images.
The highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the underexposed image corresponds to the highlight region of the overexposed image, and the position of the highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the underexposed image in the candidate ghost mask image corresponding to the underexposed image is the same as the position of the highlight region in the overexposed image.
For each other image, when the reference image and the image with the largest exposure in the other images are the other image, that is, the exposure of the other image is larger than the exposure of the reference image, the highlight association part in the ghost mask region of the candidate ghost mask image corresponding to the other image corresponds to the highlight region of the other image, and the position of the highlight association part in the ghost mask region of the candidate ghost mask image corresponding to the other image in the candidate ghost mask image corresponding to the other image is the same as the position of the highlight region of the other image in the other image.
For each other image, when the highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the other image is set as the non-ghost mask region, the pixel values of all the pixels of the highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the other image may be set to 0 directly, so that all the pixels of the highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the other image become pixels unassociated with the ghost region, and the highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the other image becomes the non-ghost mask region, resulting in the final ghost mask image corresponding to the other image.
The ghost mask region of the final ghost mask image corresponding to the other image is the remaining portion of the ghost mask regions of the candidate ghost mask image corresponding to the other image except for the highlight-related portion.
The non-ghost mask areas of the final ghost mask images corresponding to the other images are composed of the non-ghost mask areas of the candidate ghost mask images corresponding to the other images and the highlight associated portions in the ghost mask areas of the candidate ghost mask images corresponding to the other images.
In some embodiments, for each other image, performing noise pixel removal processing on a candidate ghost mask image corresponding to the other image to obtain a candidate ghost mask image corresponding to the other image after noise processing; setting a highlight-related portion in a ghost mask region of a candidate ghost mask image corresponding to the other image as a non-ghost mask region, comprising: the highlight-related portion in the ghost mask region of the candidate ghost mask image corresponding to the other image after the noise processing is set as the non-ghost mask region.
In the present application, before the highlight related portion in the ghost mask region of the candidate ghost mask image corresponding to the other image is set as the non-ghost mask region, noise pixel removal processing may be performed on the candidate ghost mask image corresponding to the other image for each other image, to obtain a candidate ghost mask image corresponding to the other image after the noise processing.
For each other image, after the noise pixel removal process is performed on the candidate ghost mask image corresponding to the other image, the candidate ghost mask image corresponding to the other image may be referred to as a candidate ghost mask image corresponding to the other image after the noise process, so as to indicate that the candidate ghost mask image corresponding to the other image is subjected to the noise process.
Noise pixels may also be referred to as outliers. For each other image, performing noise pixel removal processing on the candidate ghost mask image corresponding to the other image may include: and performing etching operation such as 7*7 and dilation operation such as 11×11 on the candidate ghost mask image corresponding to the other image.
Step 103, generating a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each other image.
In the application, when the multiple images are fused based on the final ghost mask image corresponding to each other image to obtain the high dynamic range image, the multiple images can be fused by adopting a direct fusion method of the images with different exposure values of the existing high dynamic range fusion algorithm to obtain the candidate image.
After the candidate image is obtained, a ghost area of the candidate image may be determined based on the final ghost mask image corresponding to each of the other images.
For each other image, the size of the final ghost mask image corresponding to the other image is the same as the size of the candidate image, the ghost mask region of the final ghost mask image corresponding to the other image corresponds to a region of the candidate image, the position of the ghost mask region of the final ghost mask image corresponding to the other image in the final ghost mask image corresponding to the other image is the same as the position of the region corresponding to the ghost mask region in the candidate image, and the region of the candidate image corresponding to the ghost mask region of the final ghost mask image corresponding to the other image is the ghost region of the candidate image or a part of the ghost region of the candidate image.
For the ghost mask region of the final ghost mask image corresponding to each other image, a region of the candidate image corresponding to the ghost mask region may be determined. After determining the region of each of the candidate images corresponding to the ghost mask region of the final ghost mask image corresponding to the other images, the ghost region of the candidate image may be determined.
After the ghost area of the candidate image is accurately determined, the pixels in the ghost area of the candidate image may be replaced with the pixels at the corresponding positions in one of the plurality of images, for example, the pixels in the ghost area of the candidate image are replaced with the corresponding pixels in the reference image in the plurality of images, thereby eliminating the ghost in the candidate image so that the ghost area of the candidate image is converted into a non-ghost area corresponding to the ghost area of the candidate image, resulting in a high dynamic range image free of ghost, the luminance of the pixels in the non-ghost area in the high dynamic range image being the luminance of the pixels in the image from which it comes.
Pixels at corresponding locations in one of the plurality of images are from a region of the image corresponding to the ghost region of the candidate image. The position of the region of the image corresponding to the ghost region of the candidate image in the image is the same as the position of the ghost region of the candidate image in the candidate image.
When a pixel in a ghost area of a candidate image is replaced with a pixel at a corresponding position in one image, the pixel is replaced with a pixel corresponding to the pixel in the one image for each pixel in the ghost area of the candidate image, and a position of the pixel corresponding to the pixel in the one image in an area corresponding to the ghost area is the same as a position of the pixel in the ghost area.
When the existing high dynamic range fusion algorithm used for fusing the plurality of images is a high dynamic range fusion algorithm used for fusing the plurality of images based on the weight of each pixel in each image, the existing high dynamic range fusion algorithm calculates the weight of each pixel in each image used for fusing.
In the present application, for each other image, for each pixel associated with a ghost region in the final ghost mask image corresponding to the other image, the pixel associated with the ghost region corresponds to one pixel in the other image, the position of the pixel associated with the ghost region in the ghost mask region is the same as the position of one pixel in the other image corresponding thereto, the pixel value of the pixel associated with the ghost region is divided by the maximum value of the pixel values of the pixels in the ghost mask region, and the weight for fusion of the pixel corresponding to the pixel associated with the ghost region is reduced according to the obtained result, for example, the obtained result is multiplied with the weight for fusion of the pixel corresponding to the pixel associated with the ghost region calculated by the existing high dynamic range fusion algorithm, to obtain the final weight for fusion of the pixel corresponding to the ghost region.
When the pixel value of each pixel associated with the ghost area in the final ghost mask image corresponding to each other image is the normalized pixel value of each pixel associated with the ghost area, for each other image, the weight for fusion of the pixel corresponding to the ghost area is reduced according to the normalized pixel value of the pixel associated with the ghost area, for example, the normalized pixel value of the pixel associated with the ghost area is multiplied by the weight for fusion of the pixel corresponding to the ghost area calculated by the existing high dynamic range fusion algorithm, to obtain the final weight for fusion of the pixel corresponding to the ghost area.
In some embodiments, for each other image, performing mean filtering processing on final ghost mask images corresponding to the other images to obtain final ghost mask images corresponding to the filtered other images; generating a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images, comprising: a high dynamic range image is generated based on the final ghost mask image and the plurality of images corresponding to each of the filtered other images.
In the present application, before generating the high dynamic range image based on the final ghost mask image and the multiple images corresponding to each other image, the average filtering process may be performed on the final ghost mask image corresponding to each other image, to obtain the final ghost mask image corresponding to the filtered other image.
For each other image, after the average filtering process is performed on the final ghost mask image corresponding to the other image, the candidate ghost mask image corresponding to the other image may be referred to as a final ghost mask image corresponding to the other image after filtering, so as to indicate that the final ghost mask image corresponding to the other image is subjected to the average filtering process.
In the application, for each other image, the final ghost mask image corresponding to the other image is subjected to mean filtering, and the pixel value of each pixel associated with the ghost region in the ghost mask region of the final ghost mask image corresponding to the other image can be adjusted.
Since the pixel value of the pixel associated with the ghost area in the ghost mask area of the final ghost mask image corresponding to the other image is associated with the weight for fusion of the pixel associated with the ghost area in the other image, adjusting the pixel value of the pixel associated with the ghost area in the ghost mask area of the final ghost mask image corresponding to the other image by the mean value filtering process is equivalent to adjusting the weight for fusion of the pixel associated with the ghost area in the other image by the mean value filtering process. Thus, the details of the candidate image obtained after fusing the plurality of images at the edge of the ghost area can be made natural.
In some embodiments, generating the high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images includes: fusing a plurality of images by using a preset high dynamic range fusion algorithm to obtain candidate images; determining a ghost area of the candidate image based on the final ghost mask image corresponding to each of the other images; and replacing pixels in the ghost areas of the candidate images by pixels in the areas, corresponding to the ghost areas of the candidate images, of the target images, so as to obtain the high dynamic range image, wherein the target image is the image with the largest exposure amount in the plurality of images.
The preset high dynamic range fusion algorithm is an existing high dynamic range fusion algorithm. And fusing the multiple images by using a preset high dynamic range fusion algorithm based on the final ghost mask image corresponding to each other image to obtain a candidate image. Then, a ghost area of the candidate image is determined based on the final ghost mask image corresponding to each of the other images. Finally, pixels in the ghost area of the candidate image are replaced with pixels in an area corresponding to the ghost area in the target image, so that the ghost area of the candidate image is converted into a non-ghost area corresponding to the ghost area of the candidate image, thereby obtaining a high dynamic range image without ghosting.
When a pixel in a ghost area of a candidate image is replaced with a pixel in an area of the target image corresponding to the ghost area of the candidate image, the pixel is replaced with a pixel corresponding to the pixel in the target image for each pixel in the ghost area of the candidate image, and a position of the pixel corresponding to the pixel in the target image in the area corresponding to the ghost area of the target image is the same as a position of the pixel in the ghost area of the candidate image.
For example, the plurality of images for generating the high dynamic range image are images for generating the high dynamic range image of two of all images continuously photographed. The two images for generating the high dynamic range image include: an overexposed image, an underexposed image, the overexposed image being a reference image, the underexposed image being another image. The target image of the two images for generating the high dynamic range image is an overexposed image. Firstly, fusing the two images for generating the high dynamic range image by using a preset high dynamic range fusion algorithm to obtain a candidate image, and then determining a ghost area of the candidate image based on a final ghost mask image corresponding to the underexposed image. Finally, pixels in the ghost area of the candidate image are replaced with pixels in an area corresponding to the ghost area of the candidate image in the overexposed image, resulting in a high dynamic range image.
For example, the plurality of images for generating the high dynamic range image are three images for generating the high dynamic range image among all the images continuously photographed. The three images for generating the high dynamic range image include: the underexposure image and the underexposure image are other images. The target image of the three images for generating the high dynamic range image is an overexposed image. Firstly, fusing the three images for generating the high dynamic range image by using a preset high dynamic range fusion algorithm to obtain a candidate image, and then determining a ghost area of the candidate image based on a final ghost mask image corresponding to the underexposed image and a final ghost mask image corresponding to the overexposed image. Finally, pixels in the ghost area of the candidate image are replaced with pixels in an area corresponding to the ghost area of the candidate image in the overexposed image, resulting in a high dynamic range image.
In some embodiments, further comprising: reducing the size of the candidate image by a preset multiple to obtain a small-size candidate image, and reducing the high-dynamic-range image by the preset multiple to obtain a small-size high-dynamic-range image; carrying out Gaussian blur processing on the small-size candidate image to obtain a small-size candidate image after Gaussian blur, and carrying out Gaussian blur processing on the small-size high-dynamic-range image to obtain a small-size high-dynamic-range image after Gaussian blur; generating a brightness difference weight image, wherein the pixel value of a pixel in the brightness difference weight image is a brightness ratio obtained by dividing the brightness of the pixel in the small-size candidate image after Gaussian blur by the brightness of the pixel in the high-dynamic-range image after Gaussian blur; increasing the size of the brightness difference weight image to the size of the high dynamic range image to obtain a large-size brightness difference weight image; and adjusting the brightness of pixels in the high dynamic range image based on the large-size brightness difference weight image to obtain the brightness-adjusted high dynamic range image.
Since the non-ghost area in the high dynamic range image corresponding to the ghost area of the candidate image is obtained by replacing the pixels in the ghost area of the candidate image with the corresponding pixels, the luminance of the corresponding pixels in the non-ghost area in the high dynamic range image corresponding to the ghost area of the candidate image is different from the luminance of the pixels in the ghost area of the candidate image, and thus, a luminance abnormality occurs with respect to the luminance of the surrounding area of the non-ghost area in the ghost area corresponding to the candidate image in the high dynamic range image.
After the high dynamic range image is obtained, the brightness abnormality of the non-ghost area corresponding to the ghost area of the candidate image in the high dynamic range image can be eliminated, so that the high dynamic range image has no ghost and no area with brightness abnormality, and the brightness of the high dynamic range image is natural.
In order to eliminate the luminance abnormality of the non-ghost region corresponding to the ghost region of the candidate image in the high dynamic range image, the size of the candidate image and the high dynamic range image are first reduced by the same preset factor, for example, 8 times, to obtain a small-size candidate image and a small-size high dynamic range image.
The content and the number of pixels included in the small-size candidate image are the same as the content and the number of pixels included in the candidate image, but are different in size. The image content and the number of pixels included in the small-sized high dynamic range image are the same as the content and the number of pixels included in the high dynamic range image, but are different in size.
And carrying out Gaussian blur processing on the candidate images with small sizes, and simultaneously carrying out Gaussian blur processing on the high dynamic range images with small sizes. The gaussian blur process may also be referred to as gaussian smoothing.
After the gaussian blur processing is performed on the small-size candidate image, the small-size candidate image may be referred to as a small-size candidate image after the gaussian blur to indicate that the small-size candidate image has undergone the gaussian blur processing. After the gaussian blur processing is performed on the small-sized high dynamic range image, the small-sized high dynamic range image may be referred to as a small-sized high dynamic range image after the gaussian blur processing to indicate that the small-sized high dynamic range image has undergone the gaussian blur processing.
Then, a luminance difference weight image is generated based on the small-sized candidate image after gaussian blur and the small-sized high dynamic range image after gaussian blur.
For each pixel in the luminance difference weight image, the pixel value of the pixel is a luminance ratio obtained by dividing the luminance of the pixel corresponding to the pixel in the small-sized candidate image after gaussian blur by the luminance of the pixel corresponding to the pixel in the small-sized high dynamic range image after gaussian blur.
For each pixel in the luminance difference weight image, the position of the pixel in the luminance difference weight image is the same as the position of the pixel corresponding to the pixel in the small-sized candidate image after gaussian blur.
For each pixel in the luminance difference weight image, the position of the pixel in the luminance difference weight image is the same as the position of the pixel corresponding to the pixel in the small-sized high dynamic range image after gaussian blur.
And finally, increasing the size of the brightness difference weight image to the size of the high dynamic range image to obtain a large-size brightness difference weight image, and adjusting the brightness of pixels in the high dynamic range image based on the large-size brightness difference weight image.
When adjusting the brightness of a pixel in the high dynamic range image based on the large-size brightness difference weight image, the original brightness of the pixel may be multiplied by the pixel value of the pixel corresponding to the pixel in the brightness difference weight image for each pixel in the high dynamic range image, to obtain the adjusted brightness of the pixel.
After adjusting the luminance of the pixels in the high dynamic range image based on the large-size luminance difference weight image, each pixel in the high dynamic range image has the adjusted luminance.
The brightness of pixels in the high dynamic range image is adjusted based on the large-size brightness difference weight image, so that the brightness of the high dynamic range image and the brightness of a candidate image with natural brightness can reach the same brightness level, and therefore brightness anomalies of a non-ghost area corresponding to the ghost area of the candidate image in the high dynamic range image are eliminated, the high dynamic range image has no ghost and no brightness anomaly area, and the brightness of the high dynamic range image is natural.
Referring to fig. 2, a flow chart of a high dynamic range image generating method provided in the present application is shown.
The plurality of images for generating the high dynamic range image are images for generating the high dynamic range image of two of all the images continuously photographed. The two images for generating the high dynamic range image include: the method comprises the steps of exposing an overexposed image and an underexposed image, wherein the exposure of the overexposed image is larger than that of the underexposed image, the overexposed image is a reference image, and the underexposed image is other images.
The reference picture may be referred to as an ev + picture. The other images may be referred to as ev-images.
In step 201, the brightness of the ev-image is adjusted by histogram matching so that the brightness of the ev-image and the brightness of the ev+ image reach the same brightness level.
Step 202, calculating the brightness difference value of two pixels from the same position of the ev-image and the ev+ image, obtaining a brightness difference image, and generating a candidate ghost mask image, namely a candidate ghost mask, based on the brightness difference image.
Step 203, detecting the highlight region of the ev+ image, and setting the pixel value of each pixel in the highlight association portion corresponding to the highlight region of the ev+ image in the candidate ghost mask to be 0, thereby obtaining the final ghost mask.
In step 204, the ev+ image and the ev-image are fused to obtain a candidate image, namely dst1, and the ghost area of dst1 is determined based on the final ghost mask.
In step 205, pixels in the ghost area of dst1 are replaced with pixels at the corresponding positions in the ev+ image, resulting in a high dynamic range image, dst2.
In step 206, dst1 and dst2 are reduced by a preset multiple, and gaussian smoothing is performed on small-sized dst1 and dst2, so as to obtain a luminance difference weight image, i.e. mask2, based on small-sized dst1 and dst2.
Step 207, increasing the size of the mask2 to the size of dst2 to obtain a large-sized mask2, and eliminating the brightness abnormality of the brightness abnormality area based on the large-sized mask2 and dst 2. The luminance abnormality region is a non-ghost region in dst2 that corresponds to a ghost region of the candidate image. So that dst2 has neither ghost nor abnormal brightness, and the brightness of dst2 is natural.
Referring to fig. 3, a block diagram of a high dynamic range image generating apparatus provided in the present application is shown. The image generation device includes: acquisition unit 301, processing unit 302, generation unit 303.
The acquisition unit 301 is configured to acquire a plurality of images for generating a high dynamic range image, the plurality of images including: a reference image, at least one other image;
the processing unit 302 is configured to adjust, for each other image, the brightness of the other image by histogram matching; generating candidate ghost mask images corresponding to the other images based on the difference between the corresponding pixels of the other images and the reference image after brightness adjustment; setting a highlight association part in a ghost mask region of the candidate ghost mask image corresponding to the other image as a non-ghost mask region to obtain a final ghost mask image corresponding to the other image, wherein the highlight association part corresponds to a highlight region of an image with the largest exposure in the reference image and the other image;
The generating unit 303 is configured to generate a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images.
In some embodiments, the processing unit 302 includes:
a normalization module configured to calculate a normalized pixel value for each pixel associated with the ghost area in the luminance difference image corresponding to the other image, wherein the pixel associated with the ghost area is a pixel having a pixel value greater than a luminance difference threshold; setting the pixel value of each pixel associated with the ghost area as the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel not associated with the ghost area in the brightness difference image corresponding to the other image as 0, thereby obtaining a candidate ghost mask image corresponding to the other image, wherein the pixel not associated with the ghost area is a pixel with the pixel value smaller than or equal to the brightness difference threshold.
In some embodiments, the normalization module is further configured to determine, for each pixel associated with a ghost region in the luminance difference images corresponding to the other images, a pixel value for normalization of the pixel associated with the ghost region based on whether the pixel value of the pixel associated with the ghost region is greater than a preset luminance difference upper limit; and normalizing the pixel value for normalization of each pixel associated with the ghost area to obtain a normalized pixel value of each pixel associated with the ghost area.
In some embodiments, the image generating apparatus further comprises:
the denoising unit is configured to perform noise pixel removal processing on candidate ghost mask images corresponding to other images for each other image to obtain candidate ghost mask images corresponding to the other images after noise processing; the processing unit 302 is further configured to set a highlight-related portion in a ghost mask area of a candidate ghost mask image corresponding to the other image after the noise processing as a non-ghost mask area.
In some embodiments, the image generating apparatus further comprises:
the filtering unit is configured to perform mean value filtering processing on final ghost mask images corresponding to other images for each other image to obtain final ghost mask images corresponding to the other images after filtering; the processing unit 302 is further configured to generate a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the filtered other images.
In some embodiments, the generating unit 303 comprises:
the high dynamic range image generation module is configured to fuse the plurality of images by using a preset high dynamic range fusion algorithm to obtain candidate images; determining a ghost area of the candidate image based on the final ghost mask image corresponding to each other image; and replacing pixels in the ghost area of the candidate image by pixels in an area corresponding to the ghost area of the candidate image in the target image, so as to obtain a high dynamic range image, wherein the target image is the image with the largest exposure in the plurality of images.
In some embodiments, the image generating apparatus further comprises:
a brightness anomaly elimination unit configured to reduce the size of the candidate image by a preset multiple to obtain a small-size candidate image, and reduce the high-dynamic-range image by the preset multiple to obtain a small-size high-dynamic-range image; carrying out Gaussian blur processing on the small-size candidate image to obtain a small-size candidate image after Gaussian blur, and carrying out Gaussian blur processing on the small-size high-dynamic-range image to obtain a small-size high-dynamic-range image after Gaussian blur; generating a brightness difference weight image, wherein the pixel value of a pixel in the brightness difference weight image is a brightness ratio obtained by dividing the brightness of the pixel in the small-size candidate image after Gaussian blur by the brightness of the pixel in the high-dynamic-range image after Gaussian blur; increasing the size of the brightness difference weight image to the size of the high dynamic range image to obtain a large-size brightness difference weight image; and adjusting the brightness of pixels in the high dynamic range image based on the large-size brightness difference weight image to obtain the brightness-adjusted high dynamic range image.
Fig. 4 is a block diagram of an electronic device according to the present embodiment. Electronic device 400 includes a processing component 422 that further includes one or more processors and memory resources represented by memory 432 for storing instructions, such as application programs, executable by processing component 422. The application program stored in memory 432 may include one or more modules each corresponding to a set of instructions. Further, the processing component 422 is configured to execute instructions to perform the above-described methods.
The electronic device 400 may also include a power component 426 configured to perform power management of the electronic device 400, a wired or wireless network interface 450 configured to connect the electronic device 400 to a network, and an input output (I/O) interface 458. The electronic device 400 may operate based on an operating system stored in the memory 432, such as Windows Server, macOS XTM, unixTM, linuxTM, freeBSDTM or the like.
In an exemplary embodiment, a storage medium is also provided, e.g., a memory, comprising instructions executable by an electronic device to perform the above-described method. Alternatively, the storage medium may be a non-transitory computer readable storage medium, which may be, for example, ROM, random Access Memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.

Claims (9)

1. A method of generating a high dynamic range image, the method comprising:
acquiring a plurality of images for generating a high dynamic range image, the plurality of images comprising: a reference image, at least one other image;
for each other image, adjusting the brightness of the other image through histogram matching; generating candidate ghost mask images corresponding to the other images based on the difference between the corresponding pixels of the other images and the reference image after brightness adjustment; setting a highlight association part in a ghost mask region of the candidate ghost mask image corresponding to the other image as a non-ghost mask region to obtain a final ghost mask image corresponding to the other image, wherein the highlight association part corresponds to a highlight region of an image with the largest exposure in the reference image and the other image;
Generating a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images;
based on the difference between the corresponding pixels of the other image and the reference image after brightness adjustment, generating the candidate ghost mask image corresponding to the other image comprises:
calculating the normalized pixel value of each pixel associated with the ghost area in the brightness difference images corresponding to the other images, wherein the pixel associated with the ghost area is a pixel with a pixel value greater than a brightness difference threshold;
setting the pixel value of each pixel associated with the ghost area as the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel not associated with the ghost area in the brightness difference image corresponding to the other image as 0, thereby obtaining a candidate ghost mask image corresponding to the other image, wherein the pixel not associated with the ghost area is a pixel with the pixel value smaller than or equal to the brightness difference threshold.
2. The method of claim 1, wherein the calculating normalized pixel values for pixels associated with the ghost region for each of the luminance difference images corresponding to the other images comprises:
For each pixel associated with a ghost area in the luminance difference images corresponding to the other images, determining a pixel value for normalization of the pixel associated with the ghost area based on whether the pixel value of the pixel associated with the ghost area is greater than a preset luminance difference upper limit;
and normalizing the pixel value for normalization of each pixel associated with the ghost area to obtain a normalized pixel value of each pixel associated with the ghost area.
3. The method according to claim 1, wherein the method further comprises:
for each other image, performing noise pixel removal processing on the candidate ghost mask image corresponding to the other image to obtain a candidate ghost mask image corresponding to the other image after noise processing, wherein the noise pixel removal processing is used for removing noise pixels in the candidate ghost mask image corresponding to the other image;
the setting the highlight association part in the ghost mask area of the candidate ghost mask image corresponding to the other image as a non-ghost mask area comprises the following steps: and setting a highlight related part in a ghost mask area of the candidate ghost mask image corresponding to the other image after noise processing as a non-ghost mask area.
4. A method according to claim 3, characterized in that the method further comprises:
for each other image, carrying out mean value filtering processing on final ghost mask images corresponding to the other images to obtain final ghost mask images corresponding to the other images after filtering;
the generating a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each other image includes: a high dynamic range image is generated based on the final ghost mask image and the plurality of images corresponding to each of the other images after filtering.
5. The method of any of claims 1-4, wherein generating a high dynamic range image based on the final ghost mask image and the plurality of images for each other image comprises:
fusing the plurality of images by using a preset high dynamic range fusion algorithm to obtain candidate images;
determining a ghost area of the candidate image based on the final ghost mask image corresponding to each other image;
and replacing pixels in the ghost area of the candidate image by pixels in an area corresponding to the ghost area of the candidate image in the target image, so as to obtain a high dynamic range image, wherein the target image is the image with the largest exposure in the plurality of images.
6. The method of claim 5, wherein the method further comprises:
reducing the size of the candidate image by a preset multiple to obtain a small-size candidate image, and reducing the high-dynamic-range image by the preset multiple to obtain a small-size high-dynamic-range image;
carrying out Gaussian blur processing on the small-size candidate image to obtain a small-size candidate image after Gaussian blur, and carrying out Gaussian blur processing on the small-size high-dynamic-range image to obtain a small-size high-dynamic-range image after Gaussian blur;
generating a brightness difference weight image, wherein the pixel value of a pixel in the brightness difference weight image is a brightness ratio obtained by dividing the brightness of the pixel in the small-size candidate image after Gaussian blur by the brightness of the pixel in the high-dynamic-range image after Gaussian blur;
increasing the size of the brightness difference weight image to the size of the high dynamic range image to obtain a large-size brightness difference weight image;
and adjusting the brightness of pixels in the high dynamic range image based on the large-size brightness difference weight image to obtain the brightness-adjusted high dynamic range image.
7. A high dynamic range image generating apparatus, the apparatus comprising:
an acquisition unit configured to acquire a plurality of images for generating a high dynamic range image, the plurality of images including: a reference image, at least one other image;
a processing unit configured to adjust, for each other image, the brightness of the other image by histogram matching; generating candidate ghost mask images corresponding to the other images based on the difference between the corresponding pixels of the other images and the reference image after brightness adjustment; setting a highlight association part in a ghost mask region of the candidate ghost mask image corresponding to the other image as a non-ghost mask region to obtain a final ghost mask image corresponding to the other image, wherein the highlight association part corresponds to a highlight region of an image with the largest exposure in the reference image and the other image;
a generation unit configured to generate a high dynamic range image based on the final ghost mask image and the plurality of images corresponding to each of the other images;
the processing unit includes:
a normalization module configured to calculate a normalized pixel value for each pixel associated with the ghost area in the luminance difference image corresponding to the other image, wherein the pixel associated with the ghost area is a pixel having a pixel value greater than a luminance difference threshold; setting the pixel value of each pixel associated with the ghost area as the normalized pixel value of each pixel associated with the ghost area, and setting the pixel value of each pixel not associated with the ghost area in the brightness difference image corresponding to the other image as 0, thereby obtaining a candidate ghost mask image corresponding to the other image, wherein the pixel not associated with the ghost area is a pixel with the pixel value smaller than or equal to the brightness difference threshold.
8. An electronic device, comprising:
a processor;
a memory for storing the processor-executable instructions;
wherein the processor is configured to execute the instructions to implement the method of any one of claims 1 to 6.
9. A storage medium, which when executed by a processor of an electronic device, enables the electronic device to perform the method of any one of claims 1 to 6.
CN202010183368.1A 2020-03-16 2020-03-16 High dynamic range image generation method, device, electronic equipment and storage medium Active CN111429369B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010183368.1A CN111429369B (en) 2020-03-16 2020-03-16 High dynamic range image generation method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010183368.1A CN111429369B (en) 2020-03-16 2020-03-16 High dynamic range image generation method, device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111429369A CN111429369A (en) 2020-07-17
CN111429369B true CN111429369B (en) 2023-07-28

Family

ID=71549512

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010183368.1A Active CN111429369B (en) 2020-03-16 2020-03-16 High dynamic range image generation method, device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN111429369B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3436943A1 (en) * 1983-11-29 1985-06-05 Veb Pentacon Dresden Kamera- Und Kinowerke, Ddr 8021 Dresden Exposure measuring circuit
US5983043A (en) * 1997-12-24 1999-11-09 Canon Kabushiki Kaisha Potential measuring apparatus and potential measuring method
CN1607818A (en) * 2003-10-15 2005-04-20 富士通天株式会社 Image processing device, operation supporting device, and operation supporting system
CN103002225A (en) * 2011-04-20 2013-03-27 Csr技术公司 Multiple exposure high dynamic range image capture

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3612790B1 (en) * 2017-04-19 2022-07-06 DRS Network & Imaging Systems, LLC Active hyperspectral imager
CN107197146B (en) * 2017-05-31 2020-06-30 Oppo广东移动通信有限公司 Image processing method and device, mobile terminal and computer readable storage medium
CN109587395B (en) * 2018-11-23 2020-11-06 福建工程学院 Glare prevention system based on image processing and virtual enhancement and implementation method thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3436943A1 (en) * 1983-11-29 1985-06-05 Veb Pentacon Dresden Kamera- Und Kinowerke, Ddr 8021 Dresden Exposure measuring circuit
US5983043A (en) * 1997-12-24 1999-11-09 Canon Kabushiki Kaisha Potential measuring apparatus and potential measuring method
CN1607818A (en) * 2003-10-15 2005-04-20 富士通天株式会社 Image processing device, operation supporting device, and operation supporting system
CN103002225A (en) * 2011-04-20 2013-03-27 Csr技术公司 Multiple exposure high dynamic range image capture

Also Published As

Publication number Publication date
CN111429369A (en) 2020-07-17

Similar Documents

Publication Publication Date Title
US9479705B2 (en) Exposure value adjustment apparatus, method, and non-transitory tangible machine-readable medium thereof
CN111986129B (en) HDR image generation method, equipment and storage medium based on multi-shot image fusion
US20170318240A1 (en) Methods and apparatus for automated noise and texture optimization of digital image sensors
JP4847633B2 (en) Imaging apparatus and image restoration method
JP2015532070A (en) Scene recognition method and apparatus
WO2017047494A1 (en) Image-processing device
KR20110138114A (en) Method and apparatus for processing frames obtained by multi-exposure
CN110443766B (en) Image processing method and device, electronic equipment and readable storage medium
Ko et al. Artifact-free low-light video enhancement using temporal similarity and guide map
US20160117809A1 (en) Image processing apparatus, control method thereof and computer-readable storage medium
WO2011132416A1 (en) Imaging device and image restoration method
CN113344821B (en) Image noise reduction method, device, terminal and storage medium
US20220101503A1 (en) Method and apparatus for combining low-dynamic range images to a single image
CN109035167B (en) Method, device, equipment and medium for processing multiple faces in image
US9338354B2 (en) Motion blur estimation and restoration using light trails
KR101336240B1 (en) Method and apparatus for image processing using saved image
Pourreza-Shahri et al. Exposure bracketing via automatic exposure selection
CN111429369B (en) High dynamic range image generation method, device, electronic equipment and storage medium
CN115278103B (en) Security monitoring image compensation processing method and system based on environment perception
US20200007732A1 (en) Image processing method and electronic device
Chung et al. Under-exposed image enhancement using exposure compensation
CN110689502B (en) Image processing method and related device
JP6254813B2 (en) X-ray image processing apparatus and method, X-ray imaging apparatus, and program
CN113436106B (en) Underwater image enhancement method and device and computer storage medium
US20200342575A1 (en) Image processing apparatus, control method of same, and non-transitory computer-readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant