CN115719316A - Image processing method and device, electronic equipment and computer readable storage medium - Google Patents

Abstract

The disclosure provides an image processing method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a first image of a current scene, and determining a plurality of exposure areas corresponding to different depth information in the first image, wherein each exposure area corresponds to different exposure time and exposure step length; for each exposure area, generating a plurality of second images according to the exposure step length in the exposure time; respectively determining third images from the second images corresponding to the exposure areas, and fusing the third images corresponding to the exposure areas to obtain fourth images; and respectively carrying out contrast enhancement processing on each exposure area in the fourth image according to the corresponding exposure information to obtain a target image. The exposure processing effect of the image can be improved.

Description

Image processing method and device, electronic device, and computer-readable storage medium

technical field

The present disclosure relates to the technical field of image processing, and in particular to an image processing method, an image processing device, electronic equipment, and a computer-readable storage medium.

Background technique

With the rapid development of imaging technology, the functions of imaging equipment are becoming more and more powerful, and users' requirements for image quality are gradually increasing. However, due to various reasons, overexposure or underexposure often exists in the image, which affects the image quality.

At present, the image exposure processing method of the related art still has the disadvantage of unsatisfactory exposure effect, which affects the perception of the user to a certain extent.

Contents of the invention

The purpose of the present disclosure is to provide an image processing method, an image processing device, an electronic device, and a computer-readable storage medium, so as to improve the image exposure processing effect at least to a certain extent.

According to the first aspect of the present disclosure, there is provided an image processing method, including: acquiring a first image of the current scene, determining a plurality of exposure areas corresponding to different depth information in the first image, each of the exposure areas corresponding to Different exposure times and exposure steps; for each of the exposure areas, a plurality of second images are generated according to the exposure steps within the exposure time; respectively determined from the second images corresponding to each of the exposure areas A third image is obtained, and the third image corresponding to each exposure area is fused to obtain a fourth image; contrast enhancement processing is performed on each exposure area in the fourth image according to corresponding exposure information to obtain a target image.

According to a second aspect of the present disclosure, an image processing device is provided, including: an information determination module, configured to acquire a first image of a current scene, and determine multiple exposure areas corresponding to different depth information in the first image, each One of the exposure areas corresponds to different exposure times and exposure steps; the first processing module is configured to, for each of the exposure areas, generate a plurality of second images according to the exposure steps within the exposure time; the second A second processing module, configured to determine a third image from the second images corresponding to each of the exposure areas, and fuse the third images corresponding to each of the exposure areas to obtain a fourth image; a third processing module, using performing contrast enhancement processing on each exposure area in the fourth image according to corresponding exposure information to obtain a target image.

According to a third aspect of the present disclosure, there is provided an electronic device, including: a processor; and a memory for storing one or more programs, and when the one or more programs are executed by the one or more processors, one or more A plurality of processors implement the method described above.

According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented.

In the image processing solution provided by the embodiments of the present disclosure, on the one hand, the depth information of the first image is used to determine the exposure regions corresponding to different depth information, and the exposure parameters of the exposure regions, namely the exposure time and the exposure step length, are determined respectively. The exposure parameters of the exposure area are not evenly distributed, but are determined based on the depth information. While ensuring the exposure effect of images in different depth areas, the exposure time can be reasonably adjusted to improve the shooting speed; The third image is respectively determined in the two images, and the third image corresponding to each exposure area is fused to obtain a fourth image, so that each exposure area of the fourth image is high-quality exposure, and the exposure effect is improved; on the other hand, based on Exposure information performs local contrast enhancement processing on each exposure area, and restores the contrast information of each exposure area while retaining image details, so that the target image is more in line with the perception of objects by the human eye.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure. Apparently, the drawings in the following description are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings according to these drawings without creative efforts. In the attached picture:

FIG. 1 shows a schematic diagram of an exemplary application environment involved in an image processing method and device that can be applied to an embodiment of the present disclosure;

Fig. 2 schematically shows a comparison diagram of a scene seen by human eyes and a captured image in an exemplary embodiment of the present disclosure;

FIG. 3 schematically shows a flowchart of an image processing method in an exemplary embodiment of the present disclosure;

Fig. 4 schematically shows a schematic diagram of dividing an exposure area in an exemplary embodiment of the present disclosure;

Fig. 5 schematically shows a schematic diagram of a subdivided exposure area in an exemplary embodiment of the present disclosure;

Fig. 6 schematically shows a flowchart of an implementation manner of determining multiple exposure regions in an exemplary embodiment of the present disclosure;

FIG. 7 schematically shows a flowchart of an implementation manner of acquiring a fourth image of an exemplary image processing method of the present disclosure;

Fig. 8 schematically shows a flowchart of an implementation manner of image enhancement processing in an exemplary embodiment of the present disclosure;

Fig. 9 schematically shows a schematic diagram of stages involved in obtaining a target image in an exemplary embodiment of the present disclosure;

Fig. 10 schematically shows a schematic flowchart of another image processing in an exemplary embodiment of the present disclosure;

FIG. 11 schematically shows the composition of an image processing device in an exemplary embodiment of the present disclosure;

Fig. 12 shows a schematic diagram of an electronic device to which the embodiments of the present disclosure can be applied.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and/or processor means and/or microcontroller means.

Fig. 1 shows a schematic diagram of an exemplary application environment involved in an image processing method and apparatus to which embodiments of the present disclosure can be applied.

As shown in Figure 1, the terminal device can be a smart device with image processing functions, such as smart phones, computers, tablets, smart watches, vehicle-mounted devices, wearable devices, monitoring devices and other smart devices, and the terminal device can also be It is referred to as a mobile terminal, a terminal, a mobile device, etc., and the present disclosure does not limit the type of the terminal device.

In the embodiment of the present disclosure, the image processing apparatus of the terminal device may determine multiple exposure areas based on the depth information of the first image based on the image processing method of the embodiment of the present disclosure, and each exposure area corresponds to a different exposure time and exposure step, For each exposure area, a plurality of second images are generated according to the exposure step within the exposure time; third images are respectively determined from the second images corresponding to each exposure area, and the third images corresponding to each exposure area are fused The fourth image is obtained, and finally the contrast enhancement processing is performed on each exposure area in the fourth image according to the corresponding exposure information to obtain the target image.

It should be noted that the image processing method provided by the embodiment of the present disclosure may be executed by a terminal device. The image processing method provided by the embodiments of the present disclosure may also be executed by a server, and correspondingly, the image processing apparatus may also be set in the server. Wherein, the terminal device may send the first image to the server, so that the server determines multiple exposure areas based on the first image based on the image processing method of the embodiment of the present disclosure, and sends the corresponding exposure time and exposure step to the terminal device, so that the terminal device Generate a plurality of second images based on the exposure time and the exposure step length and send them to the server, so that the server determines the third image from the second images corresponding to each exposure area and fuses them to obtain the fourth image, and finally performs each exposure in the fourth image The area is subjected to contrast enhancement processing according to the exposure information, and the obtained target image is sent to the terminal device. The server may be a background system that provides image processing-related services in the embodiments of the present disclosure, and may include an electronic device or a cluster formed by multiple electronic devices with computing functions such as a portable computer, a desktop computer, and a smart phone. In the embodiments of the present disclosure, an image processing method is executed by a terminal device as an example for description.

In related technologies, HDR (High Dynamic Range Imaging, high dynamic range imaging) and other technologies are still not effective in extreme scenes such as backlighting, and it is difficult to take ideal images. There may be severe overexposure of background light sources, or close-range corresponding The subject is underexposed. For example, the photographed image corresponding to the white detail area in the screen may be pale to the human eye, and the black detail area seen by the human eye may be shot in pitch black, which greatly affects the image quality. Figure 2 shows the scenery (a) seen by the human eye and the captured image (b) respectively. In the captured image, the bright part is pale and the dark part is pitch black.

However, in the related art, multiple frames of original images are collected, multiple regions with proper exposure are selected in each original image, and finally the multiple regions corresponding to multiple frames of original images are fused to generate a final target image. However, the exposure processing method for generating multiple frames by this method has the problem that the exposure time is too long, and it still cannot solve the situation of slight overexposure or underexposure, and the quality of the obtained target image cannot meet the requirements. The image gap is flattened, but there is still a difference from the actual shooting scene perceived by the human eye.

Based on one or more of the above problems, an exemplary embodiment of the present disclosure provides an image processing method. Referring to Fig. 3, the image processing method may include the following steps S310 to S340:

In step S310, the first image of the current scene is acquired, and multiple exposure areas corresponding to different depth information in the first image are determined, and each exposure area corresponds to a different exposure time and exposure step.

In an exemplary embodiment of the present disclosure, the current scene is the scene to be photographed, and the depth information refers to the distance detection information between the terminal and the photographed object in the depth direction, indicating the distance between the sensor (such as an image sensor, a depth sensor) and the photographed object. Distance, different objects have corresponding depth information, and the depth information reflects the position of different objects in the image from the device. Wherein, the first image may be a frame of raw image randomly obtained from the preview image. In the embodiment of the present disclosure, a frame of raw image may be regularly selected during the image preview process, and the depth information may be determined by using the raw image, so as to perform operations based on the depth information. Subsequent image processing operations.

After obtaining the first image, determine multiple exposure areas in the first image according to the depth information. The exposure areas refer to areas with different depths of field that meet the preset exposure conditions. For example, according to the depth information, select the distant view area and the near view area. Expose the areas that meet the preset exposure conditions, and get multiple exposure areas.

Wherein, each exposure area corresponds to a different exposure time and exposure step. FIG. 4 shows a schematic diagram of dividing exposure areas according to an exemplary embodiment of the present disclosure. As shown in FIG. 4 , each exposure area corresponds to different depth information, and the exposure time of each exposure area may be the same or different, which is not specifically limited in this embodiment of the present disclosure. Further, referring to FIG. 5, a schematic diagram of a subdivided exposure area according to an exemplary embodiment of the present disclosure is shown. As shown in FIG. 5, the exposure level is subdivided according to the exposure step in each exposure area. It is worth explaining Yes, the exposure steps of each exposure area can be the same or different, and can be set according to actual exposure needs.

In the embodiment of the present disclosure, multiple exposure areas are first determined based on depth information, and each exposure area has its own exposure time and exposure step. Based on this, the exposure time and exposure time of different exposure areas can be limited without affecting the quality of the captured image Exposure steps to reduce the shooting time and increase the shooting speed.

In step S320, for each exposure region, a plurality of second images are generated according to the exposure step within the exposure time.

In an exemplary embodiment of the present disclosure, after determining the exposure area, the exposure time corresponding to the exposure area, and the exposure step length, for each exposure area, a plurality of second images may be obtained within the corresponding exposure time according to the exposure step length , that is to say, for each exposure area, multiple raw images are generated. Continuing to refer to FIG. 5 , in the exposure time corresponding to each exposure area, each exposure step generates a corresponding frame of raw image.

In step S330, third images are respectively determined from the second images corresponding to each exposure area, and the third images corresponding to each exposure area are fused to obtain a fourth image.

In an exemplary embodiment of the present disclosure, since each exposure area obtains a plurality of second images, in order to fuse the second images generated by the exposure areas corresponding to different depth information, the third image is firstly determined from each exposure area , so that the third images corresponding to the exposure regions are fused to obtain a fourth image.

Optionally, the third image may be randomly determined from the plurality of second images corresponding to the exposure area; optionally, the third image may also be determined from the second images according to image clarity, texture information, etc., the embodiment of the present disclosure The manner of determining the third image can be selected according to actual needs, and there is no special limitation on this.

The fourth image is obtained by fusing the third images corresponding to different exposure areas, so that the obtained fourth image meets the preset exposure conditions in the areas corresponding to different depth information, such as meeting the preset exposure requirements, thereby improving the fourth image. The exposure effect of the image.

In step S340, contrast enhancement processing is performed on each exposure area in the fourth image according to the corresponding exposure information to obtain a target image.

In an exemplary embodiment of the present disclosure, a fourth image is obtained by fusion of third images corresponding to different exposure parameters (exposure time, exposure step length). In order to further improve the contrast of each area in the fourth image, the fourth image is Contrast enhancement process. Wherein, the contrast enhancement algorithm adopted may be an ACE (Automatic Color Enhancement, automatic color equalization) algorithm, of course, the embodiments of the present disclosure may also adopt other contrast enhancement algorithms considering local information, which is not specifically limited.

Wherein, for each exposure area, contrast enhancement processing may be performed according to the exposure information of the exposure area. For example, for the area with smaller exposure, increase the degree of contrast enhancement, and for the area with greater exposure, reduce the degree of contrast enhancement, so that for the entire image, while retaining the details of the image, the overall picture has a more layered sense to restore the corresponding exposure of each exposure area. perception of objects by the human eye.

Through the image processing method of the exemplary embodiment of the present disclosure, the depth information of the first image is used to determine the exposure regions corresponding to different depth information, and the exposure parameters of the exposure regions, namely, the exposure time and the exposure step length, are determined respectively. Since each exposure region The exposure parameters are not evenly distributed, but are determined based on depth information. While ensuring the exposure of images in different depth regions, the exposure time can be reasonably adjusted to improve the shooting speed; Three images, and the third image corresponding to each exposure area is fused to obtain the fourth image, so that each exposure area of the fourth image is high-quality exposure, and the exposure effect is improved; in addition, based on the exposure information, each exposure area is partially processed. Contrast enhancement processing, while retaining image details, restores the contrast information of each exposure area, making the target image more in line with the human eye's perception of objects.

In an exemplary embodiment, an implementation manner of determining a plurality of exposure regions is provided. As shown in FIG. 6, determining a plurality of exposure regions corresponding to different depth information in the first image includes steps S610 to S630:

Step S610: Determine a plurality of regions with different depths of field that meet preset exposure conditions in the first image according to the depth information, as a plurality of exposure regions.

In an embodiment of the present disclosure, multiple regions with different depths of field that meet preset exposure conditions may be selected in the first image according to the depth information, wherein the multiple regions with different depths of field include at least multiple near-field regions with different depths of field in the first image, For the foreground area, the foreground area and the foreground area that meet the preset exposure conditions are respectively selected in the first image. The preset exposure conditions include relevant conditions indicating the exposure effect, such as the clarity of the image meeting the setting conditions, the richness of the image texture meeting the setting conditions, etc., which can be set according to the actual situation, and there are no special restrictions on this.

Step S620: Based on the preset correspondence between the depth information and the exposure time, use the depth information of each exposure area to determine the exposure time of each exposure area.

After determining the exposure area corresponding to different depth information, determine the exposure time corresponding to the exposure area according to the depth information of the exposure area, wherein the preset correspondence between the depth information and the exposure time is determined according to the historical depth information and the exposure time .

Step S630: Determine the exposure step corresponding to each exposure time.

After obtaining the exposure time of each exposure area, the embodiment of the present disclosure subdivides the exposure level for each exposure time range, that is, determines the exposure step corresponding to each exposure time, so that the exposure step is within the exposure time range A plurality of second images are generated.

In some possible embodiments, the exposure step corresponding to the exposure time may be determined according to a preset relationship between the exposure time and the exposure step. Wherein, if the exposure time is short, the exposure steps are dense; on the contrary, if the exposure time is long, the exposure steps are sparse, and based on this, the image shooting time can be limited as a whole.

In some possible embodiments, the number of pixels in the exposure area corresponding to the exposure time may be obtained, and the exposure step corresponding to the exposure time may be determined according to the ratio of the number of pixels to the total number of pixels in the first image. Wherein, when the ratio of the number of pixels to the total pixels of the first image is relatively high, the exposure step length is relatively dense; if the ratio of the number of pixels to the total pixels of the first image is relatively low, the exposure step is relatively sparse, That is, a denser exposure step is adopted for a larger exposure area to preserve more image details.

In some possible embodiments, the shake information corresponding to the first image may also be acquired, and based on the preset correspondence between the shake information and the exposure step, the exposure step corresponding to the shake information may be determined. In actual implementation, information related to device shake or image shake can be obtained, such as device shake detected by sensors, such as motion detection results corresponding to image content obtained through optical flow information, etc. The present disclosure includes but is not limited to The above methods for obtaining jitter information. For example, if there is a lot of jitter in the image content, the exposure step size can be selected to be relatively sparse, so as to ensure that the obtained second image is valid and usable.

In an exemplary embodiment, an implementation manner of acquiring a fourth image is provided. As shown in FIG. 7 , determining a third image from the second images corresponding to each exposure area, and fusing the third images corresponding to each exposure area to obtain a fourth image may include steps S710 and S720:

Step S710: For each exposure area, acquire image texture information of the second image corresponding to the exposure area, and determine a third image from the second image according to the image texture information.

Each exposure area includes a plurality of second images, and the third image with the most preserved texture details may be selected from the plurality of second images.

In some possible embodiments, noise reduction processing can be performed on each second image, and then the third image can be determined from the second image according to the contrast information of the second image after noise reduction processing, then the image with the highest degree of clarity can be obtained and the image can be preserved. The image with the most details is used as the third image to ensure that the final fusion result can retain more image details.

Step S720: Fusion the third images corresponding to the exposure areas to obtain a fourth image.

After obtaining the third image corresponding to each exposure area, the third images are fused based on each exposure area to obtain a fourth image.

In an exemplary embodiment, an implementation manner of image enhancement processing is provided. As shown in FIG. 8 , performing contrast enhancement processing on each exposure area in the fourth image according to the corresponding exposure information, and obtaining the target image may include steps S810 to S830:

Step S810: For each exposure area in the fourth image, perform contrast enhancement on the pixels in the exposure area to obtain enhanced pixels.

The fourth image is obtained by fusing images with different exposure parameters (exposure time, exposure step length). In order to further restore the contrast information of different regions of the image, so that the final image conforms to the real perception of the current scene by the human eye, the first image The pixels in each exposure area in the image are subjected to contrast enhancement processing to obtain enhanced pixels.

Wherein, an adaptive contrast enhancement ACE algorithm may be selected to perform contrast enhancement processing on the fourth image. Of course, other enhancement algorithms may also be selected according to actual needs in the embodiment of the present disclosure, which is not specifically limited.

The following uses the ACE algorithm to perform contrast enhancement processing on the fourth image as an example for illustration.

First, set the pixel point in the fourth image as x(i,j), take x(i,j) as the center, and the window size is (2n+1)×(2n+1), and obtain the local mean and local The variances are:

为局部方差，则x(i,j)对应的增强后的像素值f(i,j)为：Among them, m _x (i, j) is the local mean of x (i, j),

is the local variance, then the enhanced pixel value f(i,j) corresponding to x(i,j) is:

为增益系数，D为常数，通常设置为全局图像的平局像素值。in,

Is the gain coefficient, D is a constant, usually set to the average pixel value of the global image.

In an exemplary embodiment of the present disclosure, contrast enhancement is performed on each pixel in the fourth image through the above formula (3), to obtain a corresponding enhanced pixel. Certainly, in the embodiment of the present disclosure, the gain coefficient may also be set as a constant, and the contrast enhancement is performed on the pixels in each exposure area in the fourth image by using the formula (3) based on the constant value. Based on this, the ACE algorithm is used to restore the problem of narrowing the overall contrast gap of the image caused by using different exposure times for each exposure area of the image.

Step S820: Based on the exposure information of the exposure area, replace the pixels in the exposure area with the enhanced pixels.

FIG. 9 shows a schematic diagram of the stages involved in obtaining a target image according to an exemplary embodiment of the present disclosure. As shown in FIG. 9 , after contrast enhancement is performed on each pixel in the fourth image, according to the enhanced pixel Dots form enhanced images.

In this step, in order to perform different degrees of contrast enhancement on different exposure areas in the fourth image, the pixels in the exposure area can also be replaced with the enhanced pixels according to the exposure information of the exposure area to obtain the replacement Processed exposure area.

In some possible embodiments, the target replacement ratio corresponding to the exposure information of the exposure area can be determined according to the preset correspondence between the exposure information and the pixel replacement ratio, and the target replacement ratio is used to indicate that the exposure is replaced by the enhanced pixel The proportion of pixels in the area, and then replace the pixels in the exposure area according to the target replacement ratio. Wherein, the exposure information may be related information used to reflect the exposure situation of the exposure area, such as exposure time and exposure degree.

Exemplarily, continue to refer to Fig. 9, if the exposure time of the exposure area A is short, the exposure area A itself is relatively bright, and image details need to be enhanced, then increase the degree of contrast enhancement, that is, the target replacement ratio is higher; while the exposure area B The exposure time of is longer, the exposure area B itself is darker, and the degree of contrast enhancement is reduced, that is, the target replacement ratio is lower. Based on this, the contrast information of the image can be restored by setting the contrast enhancement degree (pixel replacement degree) of different exposure regions by using different exposure time for different exposure regions.

In some possible embodiments, the preset correspondence between exposure information and pixel replacement ratio may be a linear mapping relationship between exposure time and pixel replacement ratio, and the target replacement ratio of exposure time for each exposure area is obtained based on the linear mapping relationship. For example, the linear mapping between the exposure time (a-b) and the pixel replacement ratio (0-1), if the exposure time of the exposure area A is a, then the exposure area A does not perform pixel replacement, and the original exposure area A of the fourth image is retained , if the exposure time of the exposure area B is b, then the exposure area B completely replaces the pixels of the exposure area with the enhanced pixels, that is, the exposure area B in the enhanced image is completely replaced with the exposure area B in the fourth image . Of course, the specific operations of other replacement ratios are similar, and pixel replacement is performed according to the corresponding target replacement ratios, which will not be listed here.

Step S830: Determine the target image according to each exposure area after replacement processing.

After pixel replacement processing is performed on each exposure area, the target image is determined according to all the replacement-processed exposure areas. Since different contrast enhancement levels are set for each exposure area according to the exposure time, the target image can retain more image details in the bright area while clearing the bright and dark areas, while the image details in the dark area are slightly blurred, and the entire target image is richer. Contains a sense of layering, which is in line with the perception of the current scene by the human eye.

In an exemplary embodiment, although the fourth image obtained by fusing the third image has a sufficient degree of clarity, the changes in the light and dark levels of the image are covered to a certain extent, in order to further clarify the changes in the light and dark levels of the fourth image before performing contrast enhancement , it is also possible to adjust the brightness of each exposure area in the fourth image based on the exposure information corresponding to the first image, so as to change the degree of contrast between light and dark in the fourth image, and then perform contrast enhancement processing on the fourth image to retain the image Details, so that the target image conforms to human perception in terms of light and dark contrast and image layering.

Wherein, the brightness of each exposure area may be adjusted according to the corresponding relationship between the exposure time and the brightness adjustment value. For example, based on the exposure information corresponding to the first image, the brightness of different exposure areas in the fourth image is adjusted, so that the area corresponding to the low exposure time is brighter, and the area corresponding to the high exposure time is darker, so as to improve the contrast between light and dark in the fourth image.

In an exemplary embodiment, in order to ensure the real-time accuracy of exposure parameters (exposure time, exposure step, etc.), an implementation manner of updating depth information is also provided. Before acquiring the first image of the current scene and determining multiple exposure areas corresponding to different depth information in the first image, the first image and the depth information of the first image can also be determined from multiple preview images, and based on the shooting The optical flow change information of the current scene updates the depth information of the first image.

During actual implementation, during the image preview process, the first image is determined from multiple preview images and the depth information of the first image is generated. Wherein, the first image may be randomly selected, and of course, the time for selecting the first image may also be set according to actual needs, such as regularly selecting the first image during the image preview process, which is not specifically limited.

Wherein, since the optical flow information when the user shoots is changed compared with the previously determined optical flow information of the first image, and the change information is greater than the set threshold, the depth information of the first image may not have reference value, for further Improve the accuracy of depth information. If the optical flow change information when the user shoots is greater than the set threshold, the depth information will be updated again; on the contrary, if the optical flow change information when the user shoots is less than the set threshold, the most recent moment at the current moment will be used. The depth information of the preview image is used as the depth information of the first image. Wherein, re-updating the depth information may be reacquiring an image closer to the current moment and corresponding depth information from the preview image, and updating the first image and corresponding depth information.

Fig. 10 shows a schematic flow chart of image processing according to an exemplary embodiment of the present disclosure. The image processing of the embodiment of the present disclosure will be described below with reference to FIG. 10 .

Step S1010: Turn on the device and start image preview.

Step S1020: Determining depth information and exposure regions corresponding to different depth information.

Wherein, the first image and the depth information of the first image are determined from the plurality of preview images, and the depth information of the first image is updated based on the optical flow change information when shooting the current scene. After the depth information of the first image is determined, a plurality of exposure regions meeting preset exposure conditions at different depths are selected according to the depth information.

Step S1030: Determine the exposure time of the exposure area corresponding to different depth information.

For example, the exposure time corresponding to each exposure area is rang[p1 exposure start time, p1 exposure end time], [p2 exposure start time, p2 exposure end time],...,[pn exposure start time, pn exposure end time].

Step S1040: Setting exposure steps corresponding to different exposure times.

Wherein, the exposure step corresponding to the exposure time can be determined according to the preset relationship between the exposure time and the exposure step; or, the exposure time can be determined according to the ratio of the number of pixels in the exposure area to the total pixels of the first image The corresponding exposure step; or, based on the preset correspondence between the shake information and the exposure step, determine the exposure step corresponding to the shake information corresponding to the first image.

Step S1050: Obtain the third image corresponding to each exposure area and perform fusion.

Wherein, for each exposure area, a plurality of second images are generated according to the exposure step length within the exposure time, and then for each exposure area, a third image is obtained from the plurality of second images, and finally the first image of each exposure area is The three images are fused to obtain a fourth image.

Step S1060: Perform contrast enhancement processing on each exposure area in the fourth image according to the corresponding exposure information.

For exposure regions corresponding to different depth information, contrast enhancement processing is performed according to the corresponding exposure information, and the processed fourth image is output to obtain a target image.

It should be noted that, the specific implementation details of the above steps S1010 to S1060 have been recorded in the above method embodiments, and will not be repeated here.

As can be seen from the above, the image processing method of the embodiment of the present disclosure adjusts the sequence of generating depth information and setting exposure parameters, uses the depth information of the first image to determine the exposure regions corresponding to different depth information, and determines the exposure of the exposure regions respectively. Parameters, that is, exposure time and exposure step length, because the exposure parameters of each exposure area are not evenly distributed, but determined based on depth information, while ensuring the exposure effect of images in different depth areas, the exposure time can be reasonably adjusted to increase the shooting speed; from A third image is respectively determined from a plurality of second images corresponding to each exposure area, and the third images corresponding to each exposure area are fused to obtain a fourth image, so that each exposure area of the fourth image is exposed with high quality, improving Exposure effect: Based on the exposure information, local contrast enhancement processing is performed on each exposure area, and the contrast information of each exposure area is restored while retaining image details, so that the target image is more in line with the perception of objects by the human eye.

It should be noted that the above-mentioned figures are only schematic illustrations of processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not imply or limit the chronological order of these processes. In addition, it is also easy to understand that these processes may be executed synchronously or asynchronously in multiple modules, for example.

Further, referring to FIG. 11 , an exemplary embodiment of the present disclosure provides an image processing device 1100 , including an information determination module 1110 , a first processing module 1120 , a second processing module 1130 and a third processing module 1140 . in:

An information determination module 1110, configured to acquire a first image of the current scene, and determine multiple exposure areas corresponding to different depth information in the first image, each exposure area corresponding to a different exposure time and exposure step;

The first processing module 1120 is configured to generate a plurality of second images according to the exposure step within the exposure time for each exposure area;

The second processing module 1130 is configured to respectively determine a third image from the second images corresponding to each exposure area, and fuse the third images corresponding to each exposure area to obtain a fourth image;

The third processing module 1140 is configured to perform contrast enhancement processing on each exposure area in the fourth image according to corresponding exposure information to obtain a target image.

In an exemplary embodiment, the information determination module 1110 is configured to:

According to the depth information, in the first image, a plurality of regions with different depths of field that meet the preset exposure conditions are determined as a plurality of exposure regions; based on the corresponding relationship between the preset depth information and exposure time, the depth information of each exposure region is used to determine The exposure time of each exposure area; determine the exposure step corresponding to each exposure time.

In an exemplary embodiment, the information determination module 1110 is configured to:

An exposure step corresponding to the exposure time is determined according to a preset relationship between the exposure time and the exposure step.

In an exemplary embodiment, the information determination module 1110 is configured to:

The number of pixels in the exposure area corresponding to the exposure time is obtained; and the exposure step corresponding to the exposure time is determined according to the ratio of the number of pixels to the total pixels of the first image.

In an exemplary embodiment, the information determination module 1110 is configured to:

The shake information corresponding to the first image is acquired, and based on the preset correspondence between the shake information and the exposure step, the exposure step corresponding to the shake information is determined.

In an exemplary embodiment, the second processing module 1120 is configured to:

For each exposure area, the image texture information of the second image corresponding to the exposure area is obtained, and the third image is determined from the second image according to the image texture information; the third image corresponding to each exposure area is fused to obtain a fourth image.

In an exemplary embodiment, the second processing module 1120 is configured to:

performing noise reduction processing on each second image; determining a third image from the second image according to contrast information of the second image after noise reduction processing.

In an exemplary embodiment, the third processing module 1130 is configured to:

For each exposure area in the fourth image, contrast enhancement is performed on the pixels in the exposure area to obtain enhanced pixels; based on the exposure information of the exposure area, the enhanced pixels are used to perform contrast enhancement on the pixels in the exposure area. Replacement processing: determine the target image according to each exposure area after the replacement processing.

In an exemplary embodiment, the third processing module 1130 is configured to:

According to the preset corresponding relationship between exposure information and pixel replacement ratio, determine the target replacement ratio corresponding to the exposure information of the exposure area, and the target replacement ratio is used to indicate the ratio of pixels in the exposure area to be replaced by enhanced pixels; according to the target replacement Scale replaces pixels in the exposed area.

In an exemplary embodiment, the image processing device 1100 further includes:

The fourth processing module is configured to adjust the brightness of each exposure area in the fourth image based on the exposure information corresponding to the first image, so as to change the degree of light and dark contrast of the fourth image.

In an exemplary embodiment, the image processing device 1100 further includes:

The information update module is configured to determine the first image and the depth information of the first image from multiple preview images; and update the depth information of the first image based on the optical flow change information when shooting the current scene.

The specific details of each module in the above device have been described in detail in the implementation of the method, and details not disclosed can be found in the implementation of the method, so details are not repeated here.

Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be embodied in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which can be collectively referred to herein as "circuit", "module" or "system".

An exemplary embodiment of the present disclosure also provides an electronic device used in the above method, and the electronic device may be the above-mentioned imaging device or server. Generally, the electronic device includes at least a processor and a memory, the memory is used to store executable instructions of the processor, and the processor is configured to execute the above method by executing the executable instructions.

The following uses the mobile terminal 1200 in FIG. 12 as an example to illustrate the structure of the electronic device in the embodiment of the present disclosure. Those skilled in the art will appreciate that, in addition to components specifically intended for mobile purposes, the configuration in Fig. 12 can also be applied to equipment of a stationary type. In some other implementations, the mobile terminal 1200 may include more or fewer components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be realized in hardware, software, or a combination of software and hardware. The interface connection relationship among the various components is only schematically shown, and does not constitute a structural limitation on the mobile terminal 1200 . In some other implementation manners, the mobile terminal may also adopt an interface connection manner different from that in FIG. 12 , or a combination of multiple interface connection manners.

As shown in Figure 12, the mobile terminal 1200 may specifically include: a processor 1201, a memory 1202, a bus 1203, a mobile communication module 1204, an antenna 1, a wireless communication module 1205, an antenna 2, a display screen 1206, a camera module 1207, and an audio module 1208 , a power module 1209 , and a sensor module 1210 .

The processor 1201 may include one or more processing units, for example: the processor 1201 may include an AP (Application Processor, an application processor), a modem processor, a GPU (Graphics Processing Unit, a graphics processor), an ISP (Image Signal Processor , image signal processor), controller, encoder, decoder, DSP (Digital Signal Processor, digital signal processor), baseband processor and/or NPU (Neural-Network Processing Unit, neural network processor), etc.

An encoder encodes (i.e. compresses) an image or video to reduce the data size for storage or transmission. The decoder can decode (ie decompress) the coded data of the image or video to restore the image or video data. Mobile terminal 1200 can support one or more encoders and decoders, for example: images such as JPEG (Joint Photographic Experts Group, Joint Photographic Experts Group), PNG (Portable Network Graphics, portable network graphics), BMP (Bitmap, bitmap) Format, video formats such as MPEG (Moving Picture Experts Group) 1, MPEG10, H.1063, H.1064, HEVC (High Efficiency Video Coding, high-efficiency video coding).

The processor 1201 can form a connection with the memory 1202 or other components through the bus 1203 .

The memory 1202 may be used to store computer-executable program code, which includes instructions. The processor 1201 executes various functional applications and data processing of the mobile terminal 1200 by executing instructions stored in the memory 1202 . The storage 1202 can also store application data, such as storing images, videos and other files.

The communication function of the mobile terminal 1200 can be realized by the mobile communication module 1204, the antenna 1, the wireless communication module 1205, the antenna 2, the modem processor and the baseband processor. Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. The mobile communication module 1204 can provide 3G, 4G, 5G and other mobile communication solutions applied on the mobile terminal 1200 . The wireless communication module 1205 can provide wireless communication solutions such as wireless local area network, bluetooth, and near field communication applied on the mobile terminal 1200 .

The display screen 1206 is used to implement display functions, such as displaying user interfaces, images, videos, etc., and displaying abnormal prompt information. The camera module 1207 is used to realize the shooting function, such as shooting images, videos, etc., to collect scene images. The audio module 1208 is used to implement audio functions, such as playing audio, collecting voice, and so on. The power module 1209 is used to implement power management functions, such as charging the battery, supplying power to the device, and monitoring the state of the battery. The sensor module 1210 may include one or more sensors for realizing corresponding sensing and detection functions.

In addition, the exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method in this specification is stored. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product runs on the terminal device, the program code is used to make the terminal device execute the above-mentioned Steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Methods" section.

It should be noted that the computer-readable medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. . Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Additionally, program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural Programming language - such as "C" or a similar programming language. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider). business to connect via the Internet).

Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, and these modifications, uses or adaptations follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not disclosed in the present disclosure . The specification and examples are to be considered exemplary only, with the true scope and spirit of the disclosure indicated by the appended claims.

Claims (14)

1. An image processing method, characterized in that, comprising: Acquiring a first image of the current scene, determining a plurality of exposure areas corresponding to different depth information in the first image, each of the exposure areas corresponding to a different exposure time and exposure step; For each of the exposure regions, generating a plurality of second images according to the exposure step within the exposure time; Determining a third image from the second images corresponding to each of the exposure areas, and fusing the third images corresponding to each of the exposure areas to obtain a fourth image; Perform contrast enhancement processing on each exposure area in the fourth image according to corresponding exposure information to obtain a target image. 2. The method according to claim 1, wherein the determining a plurality of exposure regions corresponding to different depth information in the first image comprises: Determining a plurality of regions with different depths of field meeting preset exposure conditions in the first image according to the depth information as the plurality of exposure regions; Determining the exposure time of each exposure area by using the depth information of each exposure area based on the preset correspondence between depth information and exposure time; An exposure step corresponding to each exposure time is determined. 3. The method according to claim 2, wherein said determining the exposure step length corresponding to each said exposure time comprises: The exposure step corresponding to the exposure time is determined according to the preset relationship between the exposure time and the exposure step. 4. The method according to claim 2, wherein said determining the exposure step length corresponding to each said exposure time comprises: Obtain the number of pixels in the exposure area corresponding to the exposure time; An exposure step corresponding to the exposure time is determined according to the ratio of the number of pixels to the total pixels of the first image. 5. The method according to claim 2, wherein said determining the exposure step length corresponding to each said exposure time comprises: Acquiring shake information corresponding to the first image, and determining an exposure step corresponding to the shake information based on a preset correspondence between the shake information and the exposure step. 6. The method according to claim 1, wherein the third images are respectively determined from the second images corresponding to each of the exposure areas, and the third images corresponding to each of the exposure areas are fused Get the fourth image, including: For each exposure area, acquire image texture information of a second image corresponding to the exposure area, and determine the third image from the second image according to the image texture information; The third images corresponding to each of the exposure regions are fused to obtain the fourth image. 7. The method according to claim 6, characterized in that, for each of the exposure areas, the image texture information of the second image corresponding to the exposure area is acquired, and the image texture information is obtained from the second image according to the image texture information. determining said third image among the two images, comprising: performing noise reduction processing on each of the second images; The third image is determined from the second image according to the contrast information of the noise-reduced second image. 8. The method according to claim 1, wherein the step of performing contrast enhancement processing on each exposure area in the fourth image according to the corresponding exposure information to obtain the target image comprises: For each exposure area in the fourth image, perform contrast enhancement on pixels in the exposure area to obtain enhanced pixels; Based on the exposure information of the exposure area, using the enhanced pixels to replace the pixels in the exposure area; The target image is determined according to each exposure area after replacement processing. 9. The method according to claim 7, characterized in that, using the enhanced pixel points to replace the pixels in the exposure area based on the exposure information of the exposure area comprises: According to the preset corresponding relationship between exposure information and pixel replacement ratio, determine a target replacement ratio corresponding to the exposure information of the exposure area, and the target replacement ratio is used to indicate that the enhanced pixel points are used to replace the pixels in the exposure area The ratio of pixels; Perform replacement processing on the pixels in the exposure area according to the target replacement ratio. 10. The method according to claim 8, characterized in that, before performing contrast enhancement processing on each exposure area in the fourth image according to the corresponding exposure information to obtain the target image, further comprising: Based on the exposure information corresponding to the first image, the brightness of each exposure area in the fourth image is adjusted, so as to change the degree of light-dark contrast of the fourth image. 11. The method according to any one of claims 1 to 10, characterized in that, in the acquisition of the first image of the current scene, and determining a plurality of exposure regions corresponding to different depth information in the first image Previously, the method further included: determining the first image and depth information of the first image from a plurality of preview images; The depth information of the first image is updated based on the optical flow change information when the current scene is captured. 12. An image processing device, comprising: An information determination module, configured to acquire a first image of the current scene, determine a plurality of exposure areas corresponding to different depth information in the first image, each of the exposure areas corresponds to a different exposure time and exposure step; A first processing module, configured to, for each of the exposure regions, generate a plurality of second images according to the exposure step within the exposure time; The second processing module is configured to respectively determine a third image from the second images corresponding to each of the exposure areas, and fuse the third images corresponding to each of the exposure areas to obtain a fourth image; The third processing module is configured to perform contrast enhancement processing on each exposure area in the fourth image according to corresponding exposure information to obtain a target image. 13. An electronic device, characterized in that it comprises: processor; and a memory for storing executable instructions of the processor; Wherein, the processor is configured to execute the method according to any one of claims 1 to 11 by executing the executable instructions. 14. A computer-readable storage medium, on which a computer program is stored, wherein, when the computer program is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images