WO2022160895A1

WO2022160895A1 - Image processing method, image processing apparatus, electronic system and readable storage medium

Info

Publication number: WO2022160895A1
Application number: PCT/CN2021/132503
Authority: WO
Inventors: 蒋霆; 饶青; 韩明燕; 刘帅成
Original assignee: 北京迈格威科技有限公司
Priority date: 2021-01-28
Filing date: 2021-11-23
Publication date: 2022-08-04
Also published as: CN113012081A

Abstract

Provided are an image processing method, an image processing apparatus, an electronic system and a readable storage medium. The image processing method comprises: acquiring a plurality of initial images collected under different exposures, and a mask image corresponding to the plurality of initial images, wherein the mask image is used for indicating a foreground region in the plurality of initial images; fusing brightness information of the plurality of initial images to obtain initial brightness information; on the basis of the mask image, fusing the initial brightness information and brightness information of a reference frame image in the plurality of initial images to obtain final brightness information; and obtaining a target image on the basis of the final brightness information and color information of the reference frame image. According to the method, a mask image indicating a foreground region of a plurality of images with different exposures is obtained on the basis of the images, and then brightness information of a foreground region in a reference frame image and a non-foreground region in initial brightness information are fused by means of the mask image to ensure matching of a foreground region in a fused image and the foreground region in the reference frame image, thereby improving the visual effect of the foreground region in the fused image and also improving the overall effect of the fused image.

Description

Image processing method, image processing device, electronic system and readable storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese Patent Application No. 202110122061.5 and entitled "Image Processing Method, Device and Electronic System" filed with the State Intellectual Property Office of China on January 28, 2021, the entire contents of which are incorporated herein by reference Applying.

technical field

The present application relates to the technical field of image processing, and in particular, to an image processing method, an image processing device, an electronic system, and a readable storage medium.

Background technique

HDR (High-Dynamic Range, high dynamic range) images can provide more dynamic range and image details than ordinary images. HDR images use multiple images with different exposures, and then use software to convert these images with different exposures. The combined image can better reflect the visual effect in the real environment.

In the related art, the HDR image synthesis method usually adopts the method of linear fusion, but the contrast of the synthesized image obtained by this method is low, and the phenomenon of graying is easy to appear visually, so that the portrait area of the synthesized image also looks gray visually. The overall effect of the image is not natural enough.

SUMMARY OF THE INVENTION

The present application provides an image processing method, an image processing device, an electronic system, and a readable storage medium, so as to at least improve the effect of a portrait area in a composite image, and at the same time improve the overall effect of the composite image.

Some embodiments of the present application provide an image processing method, the method may include: acquiring multiple initial images collected under different exposure degrees, and mask images corresponding to the multiple initial images, wherein the mask images are based on At least a part of the images in the multiple initial images are generated, and the mask image is used to indicate the foreground area in the multiple initial images; the brightness information of the multiple initial images is fused to obtain the initial brightness information; based on the mask image, the The initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images are fused to obtain the final brightness information; the target image is obtained based on the final brightness information and the color information of the reference frame images.

In an optional embodiment, the above-mentioned step of fusing the initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images based on the mask image to obtain the final brightness information may include: combining the initial brightness information with the reference frame image. The pixel brightness information of the frame image is alpha-fused through the mask image to obtain the final brightness information.

In an optional implementation manner, the pixel value of each pixel in the mask image may be 0 or 255; the initial brightness information and the pixel brightness information of the reference frame image are alpha-fused through the mask image to obtain the final brightness The information step may include: calculating the final brightness information by the following formula: new_res=ev0*(M/255)+(1-M/255)*res;

Among them, new_res represents the final brightness information; ev0 represents the pixel brightness information of the reference frame image; M represents the pixel value of each pixel in the mask image; res represents the initial brightness information.

In an optional embodiment, the above-mentioned step of acquiring multiple initial images collected under different exposure degrees and mask images corresponding to the multiple initial images may include: acquiring multiple initial images collected under different exposure degrees, Among them, each initial image corresponds to one exposure; select a specified number of initial images from multiple initial images; perform fusion processing on the brightness information of the specified number of initial images to obtain a fusion image; binarize the fusion image Process to get the mask image.

In an optional implementation manner, the above-mentioned designated number may be two; the above-mentioned step of selecting a designated number of initial images from a plurality of initial images may include: selecting an initial image with a maximum exposure and an exposure Minimum initial image.

In an optional embodiment, the above-mentioned step of obtaining a fused image by fusing the brightness information of a specified number of initial images may include: for each pair of two adjacent initial images in the specified number of initial images, for The two initial images adjacent to the current exposure are fused to obtain the first fusion image of the two adjacent initial images at the current exposure; each pair of exposures in the multiple first fusion images obtained for the specified number of initial images For two adjacent first fusion images, perform fusion processing on the two first fusion images adjacent to the current exposure degree to obtain a second fusion image of the two adjacent first fusion images at the current exposure degree; The image is used as a new first fusion image, continue to perform fusion processing for each pair of adjacent first fusion images with exposure, and perform fusion processing on the two adjacent first fusion images at the current exposure to obtain the adjacent current exposure. The steps of the second fused image of the two first fused images until a final fused image is obtained.

In an optional embodiment, the above-mentioned step of fusing two initial images adjacent to the current exposure to obtain a first fusion image of the two initial images adjacent to the current exposure may include: For each pixel in the two adjacent initial images, perform the following operations: according to the brightness value of the current pixel in the two adjacent initial images at the current exposure, find the weighted weight from the preset weight matrix, where, The weight matrix includes weighted weights corresponding to any two brightness values; according to the found weighted weights and the brightness values of the current pixel points in the two initial images adjacent to the current exposure, it is determined that the current pixel point in the first fusion image corresponds to brightness value.

In an optional implementation manner, the above-mentioned step of searching for a weighted weight from a preset weight matrix according to the brightness value of the current pixel in the two initial images adjacent to the current exposure may include: In the two initial images, the first brightness value of the current pixel in the first initial image and the second brightness value of the current pixel in the second initial image are determined; the sum of the first brightness value and one is determined as The number of target columns; the sum of the second brightness value and one is determined as the number of target rows; according to the number of target columns and the number of target rows, the position of the weighted weight in the weight matrix is determined, and the value corresponding to the position in the weight matrix is determined as Find the weighted weight.

In an optional embodiment, the above step of determining the brightness value corresponding to the current pixel in the first fused image according to the found weighted weight and the brightness value of the current pixel in the two initial images adjacent to the current exposure may be: Including: determining the brightness value corresponding to the current pixel in the first fusion image by the following formula: B _i =w1 _i *A1 _i +w2 _i *A2 _i ; wherein, B _i represents the i-th pixel in the first fusion image , where the i-th pixel is the current pixel; w1 _i represents the weighted weight found from the weight matrix, which is the i-th pixel in the first initial image A1 of the two adjacent initial images at the current exposure. The weighted weight corresponding to each pixel; w2 _i represents the weighted weight corresponding to the i-th pixel in the second initial image A2 in the two initial images adjacent to the current exposure, where w2 _i =1-w1 _i ; A1 _i represents the brightness value of the ith pixel in the first initial image A1; A2 _i represents the brightness value of the ith pixel in the second initial image A2.

In an optional embodiment, the above-mentioned step of performing binarization processing on the fusion image to obtain a mask image may include: performing guided filtering processing on the fusion image after the binarization processing to obtain a filtered mask image.

In an optional embodiment, the above-mentioned step of fusing the brightness information of the multiple initial images to obtain the initial brightness information may include: for each pair of adjacent initial images in the multiple initial images, for the current The two initial images with adjacent exposure degrees are fused to obtain the first brightness map of the two initial images adjacent to the current exposure degree; each pair of adjacent exposure degrees in the multiple first brightness maps obtained for the multiple initial images The two first luminance maps of For the new first brightness map, continue to execute the two adjacent first brightness maps for each pair of exposure degrees, and perform fusion processing on the two adjacent first brightness maps of the current exposure degree to obtain two adjacent current exposure degrees. The steps of the second brightness map of the first brightness map are until a brightness map is obtained; the brightness information of the obtained brightness map is determined as the initial brightness information.

In an optional implementation manner, the color space of the above-mentioned initial image may be in the form of YUV, the luminance information may be the Y channel, and the color information may be the UV channel.

Other embodiments of the present application provide an image processing apparatus, the apparatus may include: an image acquisition module, the image acquisition module is configured to acquire a plurality of initial images collected under different exposure degrees, and a plurality of initial images a mask image corresponding to the image, wherein the mask image is generated based on at least a part of the multiple initial images, and the mask image is used to indicate the foreground area in the multiple initial images; the first fusion module, the first The fusion module is configured to perform fusion processing on brightness information of multiple initial images to obtain initial brightness information; a second fusion module, the second fusion module is configured to combine the initial brightness information with the multiple images based on the mask image. The pixel brightness information of the reference frame image in the initial image is fused to obtain final brightness information; the information integration module is configured to obtain the target image based on the final brightness information and the color information of the reference frame image.

In an optional embodiment, the second fusion module may be configured to perform alpha fusion of the initial brightness information and the pixel brightness information of the reference frame image through the mask image to obtain the final brightness information.

In an optional implementation manner, the pixel value of each pixel in the mask image may be 0 or 255; the second fusion module may be configured to obtain the final brightness information by calculating the following formula :

new_res=ev0*(M/255)+(1-M/255)*res;

Wherein, new_res represents the final brightness information; ev0 represents the pixel brightness information of the reference frame image; M represents the pixel value of each pixel in the mask image; res represents the initial brightness information.

In an optional implementation manner, the image acquisition module may include: an initial image acquisition module, the initial image acquisition module is configured to acquire a plurality of initial images collected under different exposure degrees, wherein each of the initial images The initial image corresponds to a kind of exposure; an image selection module, the image selection module is configured to select a specified number of the initial images from the plurality of initial images; a fusion processing module, the fusion processing module is configured to for performing fusion processing on the brightness information of the specified number of the initial images to obtain a fusion image; and a binarization processing module, the binarization processing module is configured to binarize the fusion image processing to obtain the mask image.

Still other embodiments of the present application provide an electronic system, the electronic system may include an image capture device, a processing device and a storage device, the image capture device may be used to acquire preview video frames or image data; the storage device may store There is a computer program that can execute the image processing method described in any one of the foregoing embodiments when the computer program is run by the processing device.

Still other embodiments of the present application provide a computer-readable storage medium, where a computer program may be stored thereon, and when the computer program is run by a processing device, the image described in any one of the foregoing embodiments may be executed. Approach.

The embodiments of the present application bring at least the following beneficial effects:

In an image processing method, an image processing device, an electronic system and a readable storage medium provided by the present application, a plurality of initial images collected under different exposure degrees and a mask image corresponding to the plurality of initial images are first obtained, wherein the The mask image is generated based on at least a part of the multiple initial images; the mask image is used to indicate the foreground area in the multiple initial images; further, the brightness information of the multiple initial images is fused to obtain the initial brightness information; Based on the mask image, the initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images are fused to obtain the final brightness information; then based on the final brightness information and the color information of the reference frame images, the target image is obtained, and also That is, the fused image. This method obtains a mask image indicating the foreground area of the image based on multiple images with different exposures, and then fuses the brightness information of the foreground area in the reference frame image and the non-foreground area in the initial brightness information through the mask image. The effect of the foreground area in the image is better, thus ensuring that the foreground area in the fused image matches the foreground area in the reference frame image, thereby improving the visual effect of the foreground area in the fused image, and also improving the fusion image. overall effect.

Other features and advantages of the present application will be set forth in the description that follows, or some of the features and advantages may be inferred or unambiguously determined from the description, or may be learned by practicing the above-described techniques of the present application.

In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the preferred embodiments are exemplified below, and are described in detail as follows in conjunction with the accompanying drawings.

Description of drawings

In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the accompanying drawings that need to be used in the description of the specific embodiments or the prior art will be briefly introduced below. The drawings are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic structural diagram of an electronic system according to an embodiment of the present application;

FIG. 2 is a flowchart of an image processing method provided by an embodiment of the present application;

FIG. 3 is a flowchart of another image processing method provided by an embodiment of the present application;

4 is a schematic diagram of a mask image provided by an embodiment of the present application;

5 is a flowchart of another image processing method provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

Thus, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

With the continuous improvement of the functions of mobile phones, users have higher and higher demands for the functions of mobile phones such as cameras. As a functional application of mobile phones, HDR has been widely used. HDR, also known as wide dynamic range technology, is a technology used to let the camera see the characteristics of the image under very strong contrast. When high-brightness areas under strong light sources (sunlight, lamps or reflections, etc.) and relatively low-brightness areas such as shadows and backlights coexist in the image, the image output by the camera will appear bright areas that become white due to overexposure. The dark areas become black due to underexposure, which seriously affects the image quality. There is a limit to how a camera can perform in the brightest and darkest areas of the same scene, and this limitation is commonly referred to as "dynamic range". An HDR image uses multiple images with different exposures and then combines them into a single image using software. Its advantage is that you end up with a picture with details in both shadows and highlights.

The HDR image synthesis method in the related art usually adopts the method of linear fusion, but the contrast of the synthesized image obtained by this method is low, and the phenomenon of graying is easy to appear visually, so that the portrait area of the synthesized image also looks gray visually. , so that the overall effect of the image is not natural enough.

First, an example electronic system 100 , and a readable storage medium for implementing an image processing method, an image processing apparatus, and an electronic system for implementing an embodiment of the present application will be described with reference to FIG. 1 .

As shown in FIG. 1 is a schematic structural diagram of an electronic system, the electronic system 100 may include one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and one or more image acquisition devices 110, these components may be interconnected by a bus system 112 and/or other form of connection mechanism (not shown). It should be noted that the components and structures of the electronic system 100 shown in FIG. 1 are only exemplary and not limiting, and the electronic system may also have other components and structures as required.

The processing device 102 may be a gateway or an intelligent terminal, or may be a device including a central processing unit (CPU) or other forms of processing units with data processing capability and/or instruction execution capability, and may The data of other components in the system 100 are processed and other components in the electronic system 100 may also be controlled to perform desired functions.

The storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory, or the like. The non-volatile memory may include, for example, read only memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer-readable storage medium, and the processing device 102 may execute the program instructions to implement the client functions (implemented by the processing device) in the embodiments of the present application described below. and/or other desired functionality. Various application programs and various data, such as various data used and/or generated by the application program, etc. may also be stored in the computer-readable storage medium.

The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, mouse, microphone, touch screen, and the like.

The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.

The image capturing device 110 may capture preview video frames or image data, and may store the captured preview video frames or image data in the storage device 104 for use by other components.

Exemplarily, each device in the example electronic system for implementing the image processing method, text recognition method, apparatus, and electronic system according to the embodiments of the present application may be integrated or distributed, such as the processing device 102, the storage device 104. The input device 106 and the output device 108 are integrated and arranged into one body, and the image capturing device 110 is arranged at a designated position where the target image can be captured. When the various devices in the above electronic system are integrated, the electronic system can be implemented as a smart terminal such as a camera, a smart phone, a tablet computer, a computer, a vehicle-mounted terminal, and the like.

Another embodiment of the present application provides an image processing method, which can be executed by a processing device in the above electronic system; the processing device can be any device or chip with data processing capability. The processing device can independently process the received information, or can be connected with a server to jointly analyze and process the information. As shown in Figure 2, the image processing method may include the following specific steps:

Step S202, obtaining a plurality of initial images collected under different exposure degrees, and a mask image corresponding to the plurality of initial images; wherein, the mask image is generated based on at least a part of the images in the plurality of initial images; to indicate foreground regions in multiple initial images.

The above-mentioned initial image may be a picture or photo taken by a camera or a camera. During specific implementation, the camera, camera and other devices connected through communication can be used to transfer the image after shooting, or the initial image can be obtained from a storage device that stores images that have been captured and to be processed. The scene captured by each initial image in the obtained multiple initial images may be the same, but the exposure corresponding to each initial image may be different, and the exposure refers to the intensity of the light and the length of time. Generally, the higher the exposure, the whiter the captured image; the lower the exposure, the darker the captured image.

The above-mentioned mask image may be generated based on at least a part of the plurality of initial images, and the at least part of the images may be at least two images of the plurality of initial images. During specific implementation, at least a part of images can be selected from multiple initial images, and then the grayscale images corresponding to the selected initial images are fused to obtain a mask image. The mask image can be used to indicate the foreground area in multiple images, and the pixel value of the image area corresponding to the foreground area in the mask image can be different from the pixel value of the image area other than the foreground area, so that the foreground area can be distinguished and non-foreground areas. The foreground area usually refers to the area of the image that contains a human figure.

Step S204, performing fusion processing on the brightness information of the multiple initial images to obtain initial brightness information.

The brightness information of the above-mentioned initial image can usually refer to the brightness corresponding to each pixel in the initial image, and the brightness can also be called a grayscale value, a grayscale value or a brightness value, which is used to indicate the brightness and darkness of the image; The value of the brightness can range from 0 to 255. In some embodiments, the color space of the initial image may be in the form of YUV (Y represents brightness, and U and V represent chromaticity), and the brightness information of the initial image is the data corresponding to the Y channel of the initial image.

In the specific implementation, the brightness information of multiple initial images with different exposure degrees is fused together, and the brightness information corresponding to a fused image can be obtained, and the brightness information is also the initial brightness information; the initial brightness information can include The brightness corresponding to each pixel in the fused image.

Step S206, based on the above-mentioned mask image, perform fusion processing on the initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images to obtain final brightness information.

The above-mentioned reference frame image may be an initial image specified by a user from a plurality of initial images according to preset rules, for example, the reference frame image may be a specified value of exposure (the specified value can be set according to user requirements, for example, it can be set to 0. Or the initial image of 1, etc.), or it may be the image with the best foreground area effect among multiple initial images, etc. The pixel brightness information of the reference frame image may include the brightness corresponding to each pixel in the reference frame image.

Since the foreground area in the reference frame image has the best effect, the foreground area in the reference frame image is obtained through the foreground area indicated by the mask image, so that the pixel brightness information of the foreground area in the reference frame image is used as the final brightness information. The brightness of each pixel corresponding to the foreground area; through the foreground area indicated by the mask image, the brightness information corresponding to the image area except the foreground area is obtained from the initial brightness information, so that the initial brightness information except the foreground area is obtained. The brightness information corresponding to the other image areas is used as the brightness corresponding to each pixel in the image areas other than the foreground area in the final brightness information.

In step S208, a target image is obtained based on the final brightness information and the color information of the reference frame image.

The color information of the reference frame image may refer to the chromaticity corresponding to each pixel in the initial image, and the function of the chromaticity may be to describe the color and saturation of the reference frame image and to specify the color of the pixel. In some embodiments, the color space of the reference frame image may be in the form of YUV, and the color information of the reference frame image is the data corresponding to the reference frame image on the UV channel.

In the specific implementation, the final brightness information and the color information of the reference frame image are fused to obtain the target image, that is, the pixel value corresponding to each pixel in the target image can be determined by each pixel in the final brightness information. It is determined by the brightness of each pixel and the chromaticity of each pixel in the reference frame image.

In an image processing method provided by an embodiment of the present application, a plurality of initial images collected under different exposure degrees and a mask image corresponding to the plurality of initial images are first obtained; wherein, the mask image is based on the At least a part of the image is generated; the mask image is used to indicate the foreground area in the multiple initial images; then the brightness information of the multiple initial images is fused to obtain the initial brightness information; based on the mask image, the initial brightness information and the The pixel brightness information of the reference frame images in the multiple initial images is fused to obtain the final brightness information; then based on the final brightness information and the color information of the reference frame images, the target image, that is, the fused image, is obtained. This method obtains a mask image indicating the foreground area of the image based on multiple images with different exposures, and then fuses the brightness information of the foreground area in the reference frame image and the non-foreground area in the initial brightness information through the mask image. The effect of the foreground area in the image is better, thus ensuring that the foreground area in the fused image matches the foreground area in the reference frame image, thereby improving the visual effect of the foreground area in the fused image, and also improving the fusion image. overall effect.

Yet another embodiment of the present application also provides another image processing method, which can be implemented on the basis of the methods described in the above embodiments; the method focuses on the acquisition of multiple initial images collected under different exposures, and the Describe the specific process of the mask images corresponding to the multiple initial images (implemented through the following steps S302-S308); as shown in Figure 3, the method may include the following specific steps:

Step S302, acquiring a plurality of initial images collected under different exposure degrees; wherein each initial image corresponds to one exposure degree.

In step S304, a specified number of initial images are selected from the plurality of initial images.

The above specified number may be at least two, and the specific value of the specified number may be set according to research and development requirements, for example, may be set to two or three, etc. In some embodiments, in order to improve computational efficiency, the specified number may be set to two, that is, two initial images may be randomly selected from multiple initial images, or the one with the highest exposure may be selected from multiple initial images The initial image and the initial image with minimal exposure.

Step S306, performing fusion processing on the brightness information of a specified number of initial images to obtain a fusion image.

The brightness information of a specified number of initial images can be fused into a fused image, and the fused image can only contain the brightness value corresponding to each pixel. During specific implementation, if the specified number is greater than two, the above step S306 can be implemented through the following steps 10-12:

Step 10: For each pair of two adjacent initial images in the specified number of exposures, perform fusion processing on the two adjacent initial images at the current exposure to obtain the first image of the two adjacent initial images at the current exposure. a fused image.

During specific implementation, the above-mentioned specified number of initial images may be sorted in order of exposure from large to small or from small to large. If the specified number is three, then there are two pairs of two initial images with adjacent exposures in the specified number of initial images, that is, two first fusion images can be obtained; if the specified number is two, then the specified number In the initial image of , there are only one pair of two initial images with adjacent exposure degrees, that is, a first fused image can be obtained, and the first fused image is also the fused image obtained in step S306 .

Specifically, the specific process of performing fusion processing on two initial images adjacent to the current exposure to obtain the first fusion image of the two adjacent initial images at the current exposure may be: for the two initial images adjacent to the current exposure For each pixel in the image, perform the following steps 20-21 to obtain the first fused image:

Step 20: Search for weighting weights from a preset weighting matrix according to the brightness values of the current pixels in the two initial images adjacent to the current exposure; wherein the weighting matrix includes weighting weights corresponding to any two brightness values.

The above-mentioned brightness value may be equivalent to the brightness on the Y channel, and may also be referred to as a grayscale value, and the value of the brightness value may be an integer between 0 and 255. The above weight matrix can be set in advance by the research and development personnel according to the research and development requirements. The weight matrix can contain weighting weights corresponding to any two gray values, and each value contained in the weight matrix can be a positive value less than 1. number, that is, a positive number with a weighted weight less than 1. The weight matrix can be a 256*256 matrix, that is, the weight matrix has 256 rows and 256 columns, wherein, 256 rows can correspond to the rows corresponding to integers between 0-255 respectively; 256 columns can respectively correspond to 0- The integers between 255 correspond to the columns in order.

In some embodiments, the above step 20 can be implemented by the following steps 30-32:

Step 30: Determine the first brightness value of the current pixel in the first initial image and the second brightness value of the current pixel in the second initial image from the two initial images adjacent to the current exposure.

The above-mentioned first initial image may be an image with a larger exposure among the two initial images, or an image with a smaller exposure.

Step 31, the sum of the first brightness value and one is determined as the target number of columns; the sum of the second brightness value and one is determined as the target number of rows.

Since the brightness value starts from 0, and the number of rows or columns of the matrix starts from 1, the sum of the first brightness value and one can be determined as the target number of columns to be queried, and the second brightness can be The sum of the value and one determines the number of target rows to find.

Step 32: Determine the position of the weighted weight in the weight matrix according to the target number of columns and the target row, and determine the value corresponding to the position in the weight matrix as the found weighted weight.

The position of the weighted weight to be searched can be located in the weight matrix according to the number of target rows and the number of target columns, and the value corresponding to the position is determined as the weighted weight to be searched for.

Step 21: Determine the brightness value corresponding to the current pixel point in the first fused image according to the found weighted weight and the brightness value of the current pixel point in the two initial images adjacent to the current exposure.

During specific implementation, the luminance value corresponding to the current pixel in the first fused image can be determined by the following formula:

B _i =w1 _i *A1 _i +w2 _i *A2 _i ;

Among them, B _i represents the brightness value of the ith pixel in the first fusion image, where the ith pixel is the current pixel; w1 _i represents the weighted weight found from the weight matrix, which is the adjacent current exposure. The weighted weight corresponding to the _i -th pixel in the first initial image A1 in the two initial images of , where w2 _i =1-w1 _i ; A1 _i represents the brightness value of the ith pixel in the first initial image A1; A2 _i represents the brightness of the ith pixel in the second initial image A2 value.

Step 11: Perform fusion processing on the two adjacent first fusion images of the current exposure for each pair of two adjacent first fusion images in the multiple first fusion images obtained from the specified number of initial images, to obtain: The second fused image of the two first fused images adjacent to the current exposure.

Step 12, take the second fused image as a new first fused image, continue to execute the two adjacent first fused images for each pair of exposure degrees, and perform fusion processing on the two adjacent first fused images of the current exposure degree to obtain: The steps of the second fusion image of the two first fusion images adjacent to the current exposure, until a final fusion image is obtained.

If the specified number is three or more, then there are at least two pairs of two adjacent initial images in the specified number of initial images, that is, at least two first fusion images can be obtained; In the manner of step 20-21 (replace the initial image in step 20-21 with the first fusion image), continue to perform fusion processing on two adjacent first fusion images of each pair of exposure degrees in the plurality of first fusion images, A second fusion image corresponding to the two adjacent first fusion images of each pair of exposure degrees is obtained. If the specified number is three, then the obtained second fusion image is also the fusion image obtained in step S306; if the specified number exceeds three, then the second fusion image obtained in step 11 is at least two, not one , so it is still necessary to continue to fuse multiple second fused images corresponding to the specified number of initial images until a final fused image is obtained. The specific process of performing fusion processing on the plurality of second fusion images corresponding to the specified number of initial images may be the same as the process of performing fusion processing on the first fusion images.

Step S308, perform binarization processing on the fusion image to obtain a mask image.

The fused image can only contain the brightness value of each pixel, so the fused image can also be called a grayscale image, and the brightness value (or grayscale value) of each pixel in the grayscale image can be at 0. -255, so you can set the brightness value less than the set value in the fused image as the first pixel value, and set the brightness value greater than or equal to the set value as the second pixel value to complete the second pixel value of the fused image. Value processing, resulting in a mask image containing only two values. In some embodiments, the above-mentioned first pixel value may be set to 0, and the second pixel value may be set to 255, so as to obtain a mask image with only black and white colors, as shown in FIG. 4 , which is a schematic diagram of the mask image. The white area (with a pixel value of 255) in FIG. 4 may refer to the foreground area, and the foreground area may include the image area where the portrait is located, that is, the mask image can deduct the area of the portrait, and the portrait area can be deducted. It is segmented from other regions. Therefore, in some embodiments, the masked image can also be called a pseudo-keyed image. Although a part of the background in the masked image is also judged to be the Qinjing region, as long as the portrait region of the foreground region is accurate, Can.

In some embodiments, the specific process of performing binarization processing on the fusion image to obtain the mask image may also be: performing guided filtering processing on the fusion image after the binarization processing to obtain the filtered mask image, so that the The foreground areas in the resulting mask image are oversmoothed with areas of the image outside the foreground areas in the mask image. That is to say, the over-smoothed mask image of the foreground area and the background area (equivalent to the image area other than the foreground area) can be obtained through the guided filtering process, which is beneficial to the subsequent fusion processing and improves the overall effect of the image. In specific implementation, the foregoing foreground area may refer to the white area in FIG. 4 , and the background area may refer to the black area in FIG. 4 .

Step S310, performing fusion processing on the brightness information of the multiple initial images to obtain initial brightness information.

Step S312 , based on the mask image, perform fusion processing on the initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images to obtain final brightness information.

In step S314, a target image is obtained based on the final luminance information and the color information of the reference frame image.

In the above image processing method, the light and dark characteristics between images with different exposure degrees can obtain a mask image, and the mask image can better segment the foreground area in the original image, that is, the portrait area is better Then, the foreground area in the mask image is fused with the reference frame image with better foreground area, which can protect the portrait well and make the portrait area of the fused image perform better. At the same time, this method does not need to segment the foreground area through a pre-trained model, which can effectively reduce the running time and improve the efficiency of image processing.

Another embodiment of the present application also provides another image processing method, which can be implemented on the basis of the methods described in the above embodiments; the method focuses on the fusion processing of the brightness information of the plurality of initial images to obtain The specific process of the initial brightness information (implemented by the following steps S504-S510), and based on the mask image, the initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images are fused to obtain the final brightness information. The specific process (implemented by the following step S512); as shown in Figure 5, the method may include the following specific steps:

Step S502, acquiring a plurality of initial images collected under different exposure degrees, and a mask image corresponding to the plurality of initial images; the mask image is used to indicate a foreground area in the plurality of initial images.

Step S504, for each pair of adjacent two initial images of the exposure degree in the multiple initial images, perform fusion processing on the two adjacent initial images of the current exposure degree to obtain the first two initial images adjacent to the current exposure degree. Brightness map.

The specific implementation of the above step S504 may be the same as the implementation of the above step 10, that is, the specified number of initial images in step 10 are replaced by multiple initial images, and the first brightness map is replaced by the first fusion image, that is, The implementation of the above step S504. Specifically, for the manner of obtaining the first luminance maps of the two initial images adjacent to the current exposure, reference may be made to the foregoing steps 20-21, which will not be repeated here.

Step S506, for each pair of two adjacent first brightness maps in the multiple first brightness maps obtained from the multiple initial images, perform fusion processing on the two adjacent first brightness maps at the current exposure to obtain the current exposure. The second brightness map of the two first brightness maps with adjacent exposures.

Step S508, taking the second brightness map as the new first brightness map, and continuing to perform fusion processing for the two adjacent first brightness maps of each pair of exposure degrees, and performing fusion processing on the two adjacent first brightness maps of the current exposure degree, The step of obtaining the second brightness map of the two first brightness maps adjacent to the current exposure, until one brightness map is obtained.

During specific implementation, for the specific process of the above steps S506-S508, reference may be made to steps 11-12 in the above-mentioned still another embodiment, and details are not repeated here.

Step S510, determining the brightness information of the obtained brightness map as the initial brightness information.

In step S512, the initial brightness information and the pixel brightness information of the reference frame image are alpha-fused through the mask image to obtain the final brightness information.

In specific implementation, the specific process of alpha fusion may be as follows: according to the foreground area in the mask image, determine the foreground area in the reference frame image, and then determine the pixel brightness information corresponding to the foreground area in the reference frame image as the final The first brightness information corresponding to the foreground area in the brightness information; according to the image area except the foreground area in the mask image, the brightness information corresponding to the image area except the foreground area in the initial brightness information is determined, and then the initial brightness information The brightness information corresponding to the image area except the foreground area is determined as the second brightness information corresponding to the image area except the foreground area in the final brightness information; the first brightness information and the second brightness information are integrated to obtain the final brightness information.

In some embodiments, the pixel value of each pixel in the above-mentioned mask image may be 0 or 255; wherein, when the pixel value is 0, the corresponding image area may be black, that is, the black area in FIG. 4; When the value is 255, the corresponding image area can be white, that is, the white area in Figure 4, and the white area can be the foreground area, so the final brightness information can be calculated by the following formula:

new_res=ev0*(M/255)+(1-M/255)*res;

Wherein, new_res represents the final brightness information; ev0 represents the pixel brightness information of the reference frame image, and the pixel brightness information includes the brightness value of each pixel in the reference frame image; M represents the pixel value of each pixel in the mask image (the The pixel value is 0 or 255); res represents the initial brightness information, and the initial brightness information includes the brightness value of each pixel in the image after the fusion of multiple initial images.

In step S514, a target image is obtained based on the final brightness information and the color information of the reference frame image.

In order to facilitate the understanding of the embodiments of the present application, the present application will be further introduced below by taking the acquisition of three initial images with different exposure degrees as an example. First, obtain three initial images with different exposures in the form of YUV in the color space, which are ev+1, ev0 and ev-2 respectively; among them, the exposure of ev+1 is +1, the exposure of ev0 is 0, and the exposure of ev- The exposure of 2 is -2; then the three initial images are split on the Y channel and the UV channel. The brightness information of the initial image is obtained on the Y channel, and the color information of the initial image is obtained on the UV channel. . Furthermore, on the Y channel, the ev+1 and ev-2 selected from the three initial images are fused (also called filter fusion) to obtain a fused image, and then the fused image is binarized , to obtain a mask image (also known as an image mask); then perform guided filtering on the obtained mask image to over-smooth the foreground and background areas in the mask image to obtain the final mask image. Then, the obtained three initial images with different exposure degrees are fused on the Y channel to obtain the initial brightness information. Furthermore, the final luminance information is obtained through the above formula for calculating the final luminance information. In this case, the above reference frame image refers to ev0, that is, an initial image with an exposure of 0. Finally, the final brightness information obtained in the Y channel is combined with the color information of the reference frame image on the UV channel to obtain the target image.

The above image processing method, which obtains a mask image for indicating the foreground area of the image based on multiple images with different exposures, and then fuses the brightness information of the foreground area in the reference frame image and the non-foreground in the initial brightness information through the mask image. area to ensure that the foreground area in the fused image is consistent with that in the reference frame image, thereby improving the visual effect of the foreground area in the fused image and improving the overall effect of the fused image.

Corresponding to the above image processing method embodiments, yet another embodiment of the present application provides an image processing apparatus, as shown in FIG. 6 , the apparatus may include:

The image acquisition module 60 can be configured to acquire multiple initial images collected under different exposure degrees, and mask images corresponding to the multiple initial images; wherein, the mask image is based on at least a part of the multiple initial images Generated; this mask image is used to indicate foreground areas in multiple initial images.

The first fusion module 61 may be configured to perform fusion processing on the brightness information of the above-mentioned multiple initial images to obtain initial brightness information.

The second fusion module 62 may be configured to perform fusion processing on the initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images based on the mask image to obtain final brightness information.

The information integration module 63 may be configured to obtain the target image based on the final luminance information and the color information of the reference frame image.

The color space of the above-mentioned initial image may be in the form of YUV, the luminance information may be the Y channel, and the color information may be the UV channel.

Optionally, the above-mentioned second fusion module 62 may be configured to: perform alpha fusion of the initial brightness information and the pixel brightness information of the reference frame image through the mask image to obtain the final brightness information.

During specific implementation, the pixel value of each pixel in the above-mentioned mask image may be 0 or 255; the above-mentioned second fusion module 62 may also be configured to: obtain the final brightness information by calculating the following formula:

new_res=ev0*(M/255)+(1-M/255)*res;

Specifically, the above-mentioned image acquisition module 60 may include: an initial image acquisition module, configured to acquire a plurality of initial images collected under different exposure degrees, wherein each initial image corresponds to one exposure degree; an image selection module, configured is used to select a specified number of initial images from a plurality of initial images; the fusion processing module is configured to perform fusion processing on the brightness information of the specified number of initial images to obtain a fusion image; the binarization processing module is configured to It is used to binarize the fused image to obtain a mask image.

Optionally, the above-mentioned fusion processing module can be configured to: for each pair of two adjacent initial images of exposure in a specified number of initial images, perform fusion processing on the two adjacent initial images of the current exposure to obtain the current exposure. The first fusion image of the two initial images with adjacent exposures; the two first fusion images of each pair of adjacent exposures in the multiple first fusion images obtained for the specified number of initial images, the adjacent first fusion images for the current exposure The two first fused images obtained are fused to obtain the second fused image of the two first fused images adjacent to the current exposure; the second fused image is taken as the new first fused image, and the execution of each pair of exposures is continued. The two adjacent first fusion images are fused to the two adjacent first fusion images of the current exposure, and the second fusion image of the two adjacent first fusion images of the current exposure is obtained. A final fused image.

Optionally, the above-mentioned designated number may be two; the above-mentioned image selection module may be configured to: select the initial image with the maximum exposure and the initial image with the minimum exposure from the plurality of initial images.

Specifically, the above-mentioned fusion processing module may also be configured to perform the following operation for each pixel in the two adjacent initial images with the current exposure: according to the current exposure, the current pixel in the two adjacent initial images Find the weighted weight from the preset weight matrix, where the weight matrix includes weighted weights corresponding to any two brightness values; according to the found weighted weights and the two initial images adjacent to the current exposure The brightness value of the current pixel point determines the brightness value corresponding to the current pixel point in the first fusion image.

During specific implementation, the above-mentioned fusion processing module may also be configured to: from two initial images adjacent to the current exposure, determine the first brightness value of the current pixel in the first initial image and the second initial image The second brightness value of the current pixel in the The number of rows, determine the position of the weighted weight in the weight matrix, and determine the value corresponding to the position in the weight matrix as the found weighted weight.

In some embodiments, the above-mentioned fusion processing module may also be configured to: determine the brightness value corresponding to the current pixel in the first fusion image by the following formula:

B _i =w1 _i *A1 _i +w2 _i *A2 _i ;

Optionally, the above binarization processing module may also be configured to: perform guided filtering processing on the binarized fusion image to obtain a filtered mask image.

Optionally, the above-mentioned first fusion module 61 can also be configured to: for each pair of two initial images adjacent to the exposure degree in the plurality of initial images, perform fusion processing on the two adjacent initial images of the current exposure degree, The first brightness maps of the two initial images adjacent to the current exposure are obtained; for each pair of two adjacent first brightness maps in the multiple first brightness maps obtained from the multiple initial images, the current exposure is determined relative to the current exposure. The two adjacent first brightness maps are fused to obtain the second brightness map of the two first brightness maps adjacent to the current exposure; the second brightness map is used as the new first brightness map, and the execution of each pair of exposures is continued. The steps of fusing the two adjacent first brightness maps of the current exposure to obtain the second brightness map of the two adjacent first brightness maps of the current exposure, until Until a brightness map is obtained; the brightness information of the obtained brightness map is determined as the initial brightness information.

The above-mentioned image processing apparatus may first acquire a plurality of initial images collected under different exposure degrees, and a mask image corresponding to the plurality of initial images, wherein the mask image is generated based on at least a part of the plurality of initial images, and the mask image is generated based on at least a part of the initial images. The mask image is used to indicate the foreground area in the multiple initial images; then the brightness information of the multiple initial images is fused to obtain the initial brightness information; then based on the mask image, the initial brightness information and the reference in the multiple initial images are combined. The pixel brightness information of the frame image is fused to obtain the final brightness information; then based on the final brightness information and the color information of the reference frame image, the target image, that is, the fused image, is obtained. This method obtains a mask image indicating the foreground area of the image based on multiple images with different exposures, and then fuses the brightness information of the foreground area in the reference frame image and the non-foreground area in the initial brightness information through the mask image. The effect of the foreground area in the image is better, thus ensuring that the foreground area in the fused image matches the foreground area in the reference frame image, thereby improving the visual effect of the foreground area in the fused image, and also improving the fusion image. overall effect.

Yet another embodiment of the present application provides an electronic system, the electronic system may include: an image acquisition device, a processing device, and a storage device, the image acquisition device may be used to acquire preview video frames or image data, and the storage device may store There is a computer program that, when run by a processing device, can execute the image processing method as described above, or the steps of the image processing method described above.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the electronic system described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here.

Embodiments of the present application further provide a computer-readable storage medium, where a computer program may be stored on the computer-readable storage medium, and when the computer program is run by a processing device, the above-mentioned image processing method may be executed.

The image processing method, the image processing apparatus, and the computer program product of the electronic system provided by the embodiments of the present application may include a computer-readable storage medium storing program codes, and the instructions included in the program codes may be used to execute the preceding For the specific implementation of the method described in the method embodiment, reference may be made to the method embodiment, which will not be repeated here.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, for the specific working process of the system and/or device described above, reference may be made to the corresponding process in the foregoing method embodiments, which will not be repeated here.

In addition, in the description of the embodiments of the present application, unless otherwise expressly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it may be a fixed connection or a detachable connection , or integrally connected; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or the internal communication between the two components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood in specific situations.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution, and the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

In the description of this application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, which is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the indicated device or element must have a specific orientation or a specific orientation. construction and operation, and therefore should not be construed as limiting the application. Furthermore, the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed to indicate or imply relative importance.

Finally, it should be noted that the above-mentioned embodiments are only specific implementations of the present application, and are used to illustrate the technical solutions of the present application, rather than limit them. The embodiments describe the application in detail, and those of ordinary skill in the art should understand that any person skilled in the art can still modify the technical solutions described in the foregoing embodiments within the technical scope disclosed in the application. Or can easily think of changes, or equivalently replace some of the technical features; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be covered in this application. within the scope of protection. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Industrial Applicability

The present application provides an image processing method, an image processing device, an electronic system and a readable storage medium for obtaining multiple initial images collected under different exposure degrees and mask images corresponding to the multiple initial images, and the mask images are used for Indicate the foreground area in the multiple initial images; fuse the brightness information of the multiple initial images to obtain the initial brightness information; based on the mask image, fuse the initial brightness information and the brightness information of the reference frame images in the multiple initial images to obtain the final brightness information; The target image is obtained based on the final luminance information and the color information of the reference frame image. This method obtains a mask image indicating the foreground area of the image based on multiple images with different exposures, and then uses the mask image to fuse the brightness information of the foreground area in the reference frame image and the non-foreground area in the initial brightness information to ensure the image after fusion. The foreground area in the image is matched with the reference frame image, thereby improving the visual effect of the foreground area in the fused image and the overall effect of the fused image.

In addition, it can be understood that the image processing method, image processing apparatus, electronic system and readable storage medium of the present application are reproducible and can be applied in various applications. For example, the image processing method, image processing apparatus, electronic system, and readable storage medium of the present application can be applied to the field of image processing technology and the like.

Claims

An image processing method, characterized in that the image processing method comprises:

Obtaining multiple initial images collected under different exposure degrees and mask images corresponding to the multiple initial images, wherein the mask images are generated based on at least a part of the multiple initial images, and the mask images are generated based on at least a part of the multiple initial images. The version image is used to indicate the foreground area in the plurality of initial images;

Performing fusion processing on the brightness information of the plurality of initial images to obtain initial brightness information;

Based on the mask image, the initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images are fused to obtain final brightness information;

A target image is obtained based on the final luminance information and the color information of the reference frame image.
The image processing method according to claim 1, wherein the initial luminance information is obtained by fusing the initial luminance information and pixel luminance information of reference frame images in the plurality of initial images based on the mask image. The steps for final luminance information include:

The initial brightness information and the pixel brightness information of the reference frame image are alpha-fused through the mask image to obtain final brightness information.
The image processing method according to claim 2, wherein the pixel value of each pixel in the mask image is 0 or 255; The steps of obtaining final brightness information by performing alpha fusion on the mask image include:

Through the following formula, the final brightness information is obtained by calculation:

new_res=ev0*(M/255)+(1-M/255)*res;

Wherein, new_res represents the final brightness information; ev0 represents the pixel brightness information of the reference frame image; M represents the pixel value of each pixel in the mask image; res represents the initial brightness information.
The image processing method according to any one of claims 1 to 3, wherein the acquiring a plurality of initial images collected under different exposure degrees and the mask images corresponding to the plurality of initial images Steps include:

acquiring a plurality of initial images collected under different exposure degrees, wherein each of the initial images corresponds to one exposure degree;

From the plurality of initial images, select a specified number of the initial images;

Perform fusion processing on the brightness information of the specified number of the initial images to obtain a fusion image;

Binarization is performed on the fusion image to obtain the mask image.
The image processing method according to claim 4, wherein the specified number is two; the step of selecting a specified number of the initial images from the plurality of initial images comprises:

From the plurality of initial images, the initial image with the maximum exposure and the initial image with the minimum exposure are selected.
The image processing method according to claim 4, wherein the step of performing fusion processing on the brightness information of the specified number of the initial images to obtain a fusion image comprises:

For each pair of two adjacent initial images in the specified number of initial images, the two initial images adjacent to the current exposure are fused to obtain two initial images adjacent to the current exposure. the first fused image of the image;

For each pair of two adjacent first fusion images in the multiple first fusion images obtained from the specified number of initial images, the two adjacent first fusion images at the current exposure degree are fused to obtain the obtained the second fusion image of the two adjacent first fusion images at the current exposure;

Using the second fusion image as a new first fusion image, continue to perform fusion processing for each pair of adjacent first fusion images with exposure, and perform fusion processing on the two adjacent first fusion images at the current exposure to obtain The step of the second fusion image of the two first fusion images adjacent to the current exposure degree is until a final fusion image is obtained.
The image processing method according to claim 6, wherein the first fusion image of the two initial images adjacent to the current exposure is obtained by performing fusion processing on two initial images adjacent to the current exposure. The steps include:

For each pixel in the two initial images adjacent to the current exposure, do the following:

According to the brightness value of the current pixel in the two initial images adjacent to the current exposure, the weighting weight is searched from a preset weight matrix, wherein the weight matrix includes weighting weights corresponding to any two brightness values;

According to the found weighted weight and the brightness value of the current pixel in the two initial images adjacent to the current exposure, the brightness value corresponding to the current pixel in the first fused image is determined.
The image processing method according to claim 7, wherein the step of searching the weighted weight from a preset weight matrix according to the brightness value of the current pixel in the two initial images adjacent to the current exposure degree include:

From two initial images adjacent to the current exposure, determine the first brightness value of the current pixel in the first initial image and the second brightness value of the current pixel in the second initial image;

The addition of the first brightness value and one is determined as the target number of columns; the addition of the second brightness value and one is determined as the target number of rows;

The position of the weighted weight in the weight matrix is determined according to the target number of columns and the target row, and the value corresponding to the position in the weight matrix is determined as the found weighted weight.
The image processing method according to claim 7, characterized in that, determining the first pixel according to the found weighted weight and the brightness value of the current pixel in the two initial images adjacent to the current exposure. The step of fusing the brightness value corresponding to the current pixel in the image includes:

Determine the brightness value corresponding to the current pixel in the first fused image by the following formula:

B i =w1 i *A1 i +w2 i *A2 i ;

Wherein, B i represents the brightness value of the ith pixel in the first fusion image, where the ith pixel is the current pixel; w1 i represents the weighted weight found from the weight matrix, which is the The weighted weight corresponding to the i-th pixel in the first two initial images adjacent to the current exposure degree; w2 i represents the second one in the two initial images adjacent to the current exposure degree The weighted weight corresponding to the i-th pixel in the initial image A2, wherein w2 i =1-w1 i ; A1 i represents the brightness value of the i-th pixel in the first initial image A1; A2 i represents The brightness value of the ith pixel in the second initial image A2.
The image processing method according to claim 4, wherein the step of performing binarization processing on the fusion image to obtain the mask image comprises:

A guided filtering process is performed on the binarized fused image to obtain the filtered mask image.
The image processing method according to any one of claims 1 to 10, wherein the step of obtaining the initial luminance information by fusing the luminance information of the plurality of initial images comprises:

For each pair of two adjacent initial images in the multiple initial images, the two initial images adjacent to the current exposure are fused to obtain the first image of the two adjacent initial images at the current exposure. a luminance map;

For each pair of two first brightness maps adjacent to the exposure degree in the multiple first brightness maps obtained from the multiple initial images, perform fusion processing on the two adjacent first brightness maps at the current exposure degree to obtain the The second brightness map of the two first brightness maps adjacent to the current exposure;

Taking the second brightness map as a new first brightness map, continue to perform fusion processing on the two adjacent first brightness maps for each pair of exposure degrees, and obtain The steps of the second brightness map of the two adjacent first brightness maps of the current exposure, until a brightness map is obtained;

The obtained luminance information of the one luminance map is determined as the initial luminance information.
The image processing method according to any one of claims 1 to 11, wherein the color space of the initial image is a YUV form, the luminance information is a Y channel, and the color information is a UV channel.
An image processing device, characterized in that the image processing device comprises:

An image acquisition module, the image acquisition module is configured to acquire multiple initial images collected under different exposure degrees, and mask images corresponding to the multiple initial images, wherein the mask images are based on the multiple initial images. at least a part of the initial images is generated; the mask image is used to indicate the foreground area in the plurality of initial images;

a first fusion module, where the first fusion module is configured to perform fusion processing on the brightness information of the plurality of initial images to obtain initial brightness information;

The second fusion module, the second fusion module is configured to perform fusion processing on the initial brightness information and the pixel brightness information of the reference frame images in the multiple initial images based on the mask image, to obtain a final brightness information;

an information integration module, the information integration module is configured to obtain a target image based on the final luminance information and the color information of the reference frame image.
The image processing apparatus according to claim 13, wherein the second fusion module is configured to perform alpha fusion of the initial brightness information and the pixel brightness information of the reference frame image through the mask image to obtain the final brightness information .
The image processing apparatus according to claim 14, wherein the pixel value of each pixel in the mask image is 0 or 255; the second fusion module is configured to obtain the following formula by calculating The final brightness information:

new_res=ev0*(M/255)+(1-M/255)*res;

Wherein, new_res represents the final brightness information; ev0 represents the pixel brightness information of the reference frame image; M represents the pixel value of each pixel in the mask image; res represents the initial brightness information.
The image processing device according to any one of claims 13 to 15, wherein the image acquisition module comprises:

an initial image acquisition module, the initial image acquisition module is configured to acquire a plurality of initial images collected under different exposure degrees, wherein each of the initial images corresponds to one exposure degree;

an image selection module, the image selection module is configured to select a specified number of the initial images from the plurality of initial images;

a fusion processing module, the fusion processing module is configured to perform fusion processing on the brightness information of the specified number of the initial images to obtain a fusion image; and

A binarization processing module, the binarization processing module is configured to perform binarization processing on the fusion image to obtain the mask image.
An electronic system, characterized in that the electronic system comprises: an image acquisition device, a processing device and a storage device;

The image acquisition device is used to acquire preview video frames or image data;

A computer program is stored on the storage device, and the computer program executes the image processing method of any one of claims 1 to 12 when executed by the processing device.
A computer-readable storage medium on which a computer program is stored, characterized in that the computer program executes the image processing described in any one of claims 1 to 12 when the computer program is run by a processing device method.