WO2023236209A1

WO2023236209A1 - Image processing method and apparatus, electronic device, and storage medium

Info

Publication number: WO2023236209A1
Application number: PCT/CN2022/098240
Authority: WO
Inventors: 尹双双; 董家旭; 饶强; 陈妹雅; 刘阳晨旭; 江浩
Original assignee: 北京小米移动软件有限公司
Priority date: 2022-06-10
Filing date: 2022-06-10
Publication date: 2023-12-14
Also published as: CN117616777A

Abstract

The present disclosure relates to an image processing method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a first image and a second image which are collected by an image collection device for the same scene, wherein the first image is a normally exposed image, and the second image is an underexposed image; determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image; and performing blurring and rendering processing on the first image according to the effective light spot area.

Description

Image processing methods, devices, electronic equipment and storage media

Technical field

The present disclosure relates to the technical field of image processing, and specifically to an image processing method, device, electronic device and storage medium.

Background technique

In recent years, the functions of terminal devices have become more and more abundant, and the performance of various functions has gradually improved. For example, the camera program of the terminal device can provide a variety of photo modes, so that it has various functions of a professional camera to satisfy users in various situations. Photography needs in various scenarios. However, there is still a certain gap between the camera program of the terminal device and the professional camera. Take the physical blur of professional cameras as an example. When shooting, professional cameras keep objects at the depth of the focus object clear, while blurring and blurring objects at other depths to highlight the subject. In related technologies, photography with a physical blur function is mainly realized through professional lenses of professional cameras.

Contents of the invention

In order to overcome the problems existing in the related technology, embodiments of the present disclosure provide an image processing method, device, electronic device and storage medium to solve the defects in the related technology.

According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, the method including:

Obtaining a first image and a second image collected by the image acquisition device for the same scene, wherein the first image is a normal exposure image and the second image is an underexposure image;

determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;

The first image is blurred and rendered according to the effective light spot area.

In one embodiment, determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image includes:

The first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.

In one embodiment, after determining the effective light spot area in at least one first light spot area in the first image based on the at least one second light spot area in the second image, the method further includes:

Determine the target pixel point of each effective light spot area, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;

In each effective light spot area, the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.

In one embodiment, determining the color parameters of each pixel point in each effective light spot area according to the color parameters of the target pixel point includes:

When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;

In the case where the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, the jth effective light spot area is determined The color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.

Determine the target brightness parameter of each effective light spot area, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective light spot area;

In each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.

In one embodiment, before determining the effective light spot area in the at least one second light spot area in the first image based on the at least one second light spot area in the second image, the method further includes:

Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image.

In one embodiment, light spot detection is performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot in the second image. areas, including:

In the YUV domain, perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. .

In one embodiment, light spot detection is performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one first light spot area in the second image. The two-spot area includes:

Determine the pixels in the first image whose brightness is higher than the first brightness threshold as the first light spot pixels, and determine at least one connected domain composed of the first light spot pixels as the first light spot area;

Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.

In one embodiment, determining at least one connected domain composed of the first light spot pixels as the first light spot area includes:

Determine at least one connected domain composed of the first light spot pixels and with a number of pixels within a preset number range as the first light spot area;

Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:

At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.

According to a second aspect of an embodiment of the present disclosure, an image processing device is provided, and the device includes:

An acquisition module, configured to acquire the first image and the second image collected by the image acquisition device for the same scene, where the first image is a normal exposure image and the second image is an underexposure image;

a determining module configured to determine an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;

A rendering module, configured to perform blur rendering processing on the first image according to the effective light spot area.

In one embodiment, the determining module is specifically used to:

In one embodiment, a color module is also included for:

After determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image, determining a target pixel for each of the effective light spot areas. point, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;

In one embodiment, the color module is used to determine the color parameters of each pixel according to the color parameters of the target pixel in each of the effective light spot areas, specifically for:

In one embodiment, a brightness module is also included for:

After determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image, determining a target brightness of each effective light spot area. Parameter, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area;

In one embodiment, it also includes a detection module for:

Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image;

In one embodiment, the detection module is specifically used to:

In one embodiment, when the detection module is used to determine at least one connected domain composed of the first light spot pixels as the first light spot area, it is specifically used to:

According to a third aspect of an embodiment of the present disclosure, an electronic device is provided. The electronic device includes a memory and a processor. The memory is used to store computer instructions executable on the processor. The processor is used to execute the The computer instructions are based on the image processing method described in the first aspect.

According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

In the image processing method provided by the present disclosure, since the first image is a normal exposure image and the second image is an underexposure image, light spot areas that are mistakenly recognized such as light-colored substances in the normal exposure image will not be detected in the underexposure image. Identified as a light spot area, therefore the effective light spot area screened out in the first light spot area by using the second light spot area is more accurate, and the first light spot area that was mistakenly identified is removed, so that the third light spot area can be determined on the basis of determining the effective light spot area. An image is rendered with blur, imitating the physical blur function of a professional camera. If this method is applied to the camera program of the terminal device, the functions of the camera program can be enriched and the camera effect can be closer to that of a professional camera.

Description of the drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.

Figure 1 is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure;

FIG. 2 is a flowchart of an image processing method according to another exemplary embodiment of the present disclosure;

Figure 3 is a schematic structural diagram of an image processing device according to an exemplary embodiment of the present disclosure;

FIG. 4 is a structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with aspects of the disclosure as detailed in the appended claims.

The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the present disclosure, the first information may also be called second information, and similarly, the second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining."

When using a professional camera to take pictures, if you use a telephoto or large aperture lens, you will get a picture with a smaller depth of field. The focus object and other objects at the depth will remain clear, while the foreground and background will be blurred to varying degrees. and blur, which can play a role in highlighting the subject of the photograph. Among them, in the blurred background or foreground, point light sources are often blurred into light spots on the imaging plane due to their higher power density. Generally speaking, the greater the brightness of the point light source and the farther it is from the focus plane, the larger the radius of the spot formed will be.

Currently, due to the portability and cost requirements of terminal devices such as smartphones, smaller cameras are often used, which makes it difficult to take pictures with a blurred effect in mobile phone photography. Therefore, software algorithms are introduced into the camera programs of smartphones. Simulate physical blur, that is, perform blur rendering and other processing on the original image collected. However, when there are light spots in the original image collected, the identification of the spot area is not accurate enough, and the brightness level of the blur rendering result is poor. , less color information.

In a first aspect, at least one embodiment of the present disclosure provides an image processing method. Please refer to FIG. 1 , which shows the flow of the method, including step S101 and step S103.

The method can be applied to a terminal device, for example, to an algorithm that simulates physical blur in a camera program of the terminal device. The terminal device may have an image acquisition device such as a camera. These image acquisition devices can acquire images, and the camera program of the terminal device can control various parameters in the image acquisition process of the image acquisition device. This method can be applied to the scene where the camera program of the terminal device captures images. That is, this method is used to blur and render the graphics collected by the image acquisition device, thereby obtaining the image output by the camera program, which is what the user does when taking pictures with the camera program. the resulting image.

In step S101, a first image and a second image collected by the image capture device for the same scene are obtained, where the first image is a normal exposure image and the second image is an underexposure image.

Wherein, when the camera program of the terminal device is started and the user triggers the shooting function of the camera program through operation, the image acquisition device can continuously collect the first image and the second image for the same scene. The same scene is the scene that the user takes pictures of, that is, the real scene in the field of view of the image acquisition device. It can be understood that this step does not limit the order in which the first image and the second image are collected, that is, the first image can be collected first and then the second image, or the second image can be collected first and then the first image, or the first image can be collected first and then the first image. Different sub-cameras in the image collection device collect the first image and the second image simultaneously.

The first image is a normal exposure image, that is, the image collected under the normal exposure (ie, the default exposure) when the camera program takes the photo. The second image is an underexposed image, that is, the exposure is smaller than the exposure when the camera program takes the photo. Images captured under normal exposure conditions. It can be understood that the proportional relationship between the exposure amount when collecting under-exposed images and the exposure amount when collecting normal-exposed images can be set in advance, such as 80%, 75%, 60%, etc.; the exposure amount can be controlled by controlling the exposure time.

In step S102, an effective light spot area is determined in at least one first light spot area in the first image according to at least one second light spot area in the second image.

The first light spot area in the first image and the second light spot area in the second image can be collected in advance. That is, before step S102, light spot detection can be performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. Spot area.

The purpose of spot detection is to detect the spot area in the image, and the difference between the spot area and other areas is mainly reflected in the brightness. Therefore, the first image and the second image can be separately detected in the YUV domain. . If the first image and the second image are RGB images, the first image and the second image can be converted from the RGB domain to the YUV domain respectively before performing spot detection on the first image and the second image, then Spot detection can be completed using the Y channel (ie, brightness channel) of the first image and the second image.

Optionally, methods such as brightness threshold, energy function, and deep learning can be used for spot detection. Taking the method of brightness threshold as an example, light spot detection can be performed on the first image and the second image respectively in the following manner: pixels with brightness higher than the first brightness threshold in the first image are determined as first light spot pixels. , and determine at least one connected domain composed of the first light spot pixels as the first light spot area; determine the pixels with a brightness higher than the second brightness threshold in the second image as the second light spot pixels, and determine At least one connected area composed of the second light spot pixels is determined as the second light spot area.

Among them, the brightness value of the pixel is the value of the pixel in the Y channel. By traversing the brightness value of each pixel of the first image, the pixel with a brightness value higher than the first brightness threshold can be determined as the first light spot pixel, and the other pixels can be determined as non-first light spot pixels; by traversing For the brightness value of each pixel of the second image, the pixels with a brightness value higher than the second brightness threshold can be determined as the second light spot pixels, and the other pixels can be determined as non-second light spot pixels.

A connected domain division standard of a four-connected connected domain or an eight-connected connected domain can be used to determine the connected domain composed of the first light spot pixels and the connected domain composed of the second light spot pixels. Then each independent connected domain in the first image can be assigned a unique label value, such as a number, etc. This label value is also the label value of the first light spot area determined by the connected domain; each independent connected domain in the second image can be assigned Each independent connected domain is assigned a unique label value, such as a number, etc., and the label value is also the label value of the second light spot area determined by the connected domain. Label values can be assigned to connected domains using two-pass or seed-fill methods.

Because the light spot is formed by the out-of-focus point light source, the large-area light source out of focus does not form a light source. Therefore, when determining the first light spot area and the second light spot area, the area of the connected domain can be further measured, the connected domain with an area that is too large is excluded, and the connected domain with an area within a reasonable range is determined as the light spot area, so that large areas can be excluded The light source is mistakenly identified as a spot area, which improves the accuracy of spot area detection. The area of the connected domain can be characterized by the number of pixels in the connected domain, then at least one connected domain composed of the first light spot pixels and the number of pixels within the preset number range can be determined as the first light spot area; At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as the second light spot area. The preset quantity range can be set in advance, and can be less than a preset quantity threshold (for example, 300), etc.

The image acquisition device may have a position change when acquiring the first image and acquiring the second image. For example, the user may shake when holding the terminal device to take pictures. Changes in the position of the image acquisition device will cause the first image and the second image not to completely overlap, but to have some deviations. Therefore, before determining the effective light spot area, the first image and the second image can be aligned. One channel of the Y channel in the YUV domain can be used to complete the alignment process of the first image and the second image, that is, Align the Y channel of the first image with the Y channel of the second image, thereby completing the alignment process of the first image and the second image. For example, first increase the brightness of the second image through the Y channel histogram of the second image, then use optical flow alignment to align the first image and the second image after increasing the brightness, and finally complete the first image according to the alignment result. Alignment processing of an image and a second image. For example, after increasing the brightness, the second image is aligned with the first image by shifting 15 pixels upward and 20 pixels to the right. Then the second image can be shifted upward. 15 pixels and offset 20 pixels to the right to align with the first image. Through the alignment process, it can be ensured that pixels at the same position on the first image and the second image correspond to the same real-life scene, thereby improving the screening accuracy of the effective spot area.

In a possible embodiment, each first light spot area belonging to a first intersection in the at least one first light spot area can be determined as an effective light spot area, wherein the first intersection is the at least one The intersection of the first light spot area and the at least one second light spot area. For example, the first intersection can be determined based on the position coordinates of the first light spot area in the first image and the position coordinates of the second light spot area in the second image, that is, the first light spot area and the second light spot with the same position coordinates The area joins the first intersection. As another example, the first image marked with the first light spot area and the second image marked with the second light spot area are superimposed, and the first light spot area with an overlapping second light spot area on the second image is determined as an effective light spot. area.

In step S103, a blur rendering process is performed on the first image according to the effective light spot area.

When performing blur rendering processing on the first image, the depth information of the current imaging scene can be calculated through the multi-camera system or deep learning algorithm of the terminal device, and then the blur corresponding to the pixels outside the focus plane is calculated based on different depth information. radius, and finally generate a picture with a blur effect based on the blur radius corresponding to each pixel.

Specifically, the present disclosure can perform light spot detection on the first image and the second image respectively by acquiring the first image and the second image collected by the image acquisition device for the same scene, thereby obtaining at least one first light spot in the first image. area and at least one second spot area in the second image, and then these second spot areas in the second image can be used to determine the effective spot area among the first spot areas in the first image, that is, in the first image Some or all of these first light spot areas are determined as effective light spot areas, and finally the first image can be blurred and rendered based on these effective light spot areas. Since the first image is a normal exposure image and the second image is an underexposure image, the light spot areas mistakenly recognized such as light-colored substances in the normal exposure image will not be recognized as light spot areas in the underexposure image. Therefore, the second image is used The effective light spot area screened out in the first light spot area is more accurate, and the mistakenly identified first light spot area is removed, thereby achieving a better blur rendering effect and avoiding blurring of non-light spot areas. Rendered with a spot effect. If this method is applied to the camera program of the terminal device, the camera program can accurately identify the spot area in the image to be blurred, so that the camera program can achieve a better blurred rendering effect.

In some embodiments of the present disclosure, the color information in the spot area is easily lost, especially the color information in the overexposed spot area. The U and V channels in the overexposed spot area will have patchy distribution, that is, N, The V channel values are all close to 128, so the color information in the effective spot area in the first image may be lost. Therefore, after determining the effective light spot area in the first light spot area in the at least one first image based on at least one second light spot area in the second image, the effective light spot area can be determined in the following manner. The color information is restored: first, determine the target pixel point of each effective light spot area, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area, and the pixel point with the highest color saturation It mostly exists near the boundary of the effective light spot area, that is, within the halo. For example, the color saturation of each pixel can be judged based on the value of each pixel in the U and V channels in each effective light spot area. degree; next, in each effective light spot area, determine the color parameter of each pixel according to the color parameter of the target pixel, wherein the color parameter is used to blur the first image Rendering processing.

For example, when the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, in the The color parameters of each pixel in the i effective light spot areas are adjusted according to the color parameters of the target pixels, where i is an integer greater than 0 and not greater than N, and the N is an integer in the first image. The total number of effective spot areas. This is because in this case, the original color in the effective light spot area is one color, that is, the color of the target pixel. Therefore, the color parameters of each pixel can be adjusted using the color parameters of the target pixel. When making specific adjustments, you can directly use the value of the target pixel on the U channel to replace the value of each pixel on the U channel, and use the value of the target pixel on the V channel to replace the value of each pixel on the V channel; Alternatively, you can also do this for each pixel: randomly use a value between the value of the target pixel on the U channel and a value between the values of a certain pixel on the U channel to replace the value of the pixel on the U channel. , randomly replace the value of the pixel on the V channel with a value between the value of the target pixel on the V channel and the value of a certain pixel on the V channel.

Exemplarily, when the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, determine the jth The color parameter of each pixel in the effective light spot area remains unchanged, where j is an integer greater than 0 and not greater than N. This is because in this case, it means that the original colors in the effective spot area are at least two colors. If the color parameters of each pixel are adjusted using the color parameters of the target pixel, it will make part of the effective spot area Pixels are adjusted to a different color than their original color.

In this embodiment, by determining the target pixels in the effective light spot area, the lost color information of the pixels in the effective light spot area can be restored, so that in the image rendered according to the blurring of the effective light spot area, the color information in the light spot area can be restored. Colors stay true and color loss is avoided.

In some embodiments of the present disclosure, due to the relatively low latitude of the imaging system of the terminal device, it is difficult to generate brightness levels based on actual brightness in the effective spot area. Therefore, after the effective light spot area is determined in the at least one first light spot area in the first image based on the at least one second light spot area in the second image, the effective light spot area can be enhanced in the following manner Brightness level: First, determine the target brightness parameter of each effective spot area, where the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area. For example, a certain pixel The brightness parameter of a point is the value of the pixel on the Y channel; next, in each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness The parameters are used to perform blur rendering processing on the first image. For example, since the human eye's perception of brightness is non-linear, the brightness parameters of the pixels can be adjusted through the following formula that characterizes the gamma curve:

Among them, Y' is the brightness parameter adjustment result of the pixel, Y is the brightness parameter of the pixel, and Y _min is the target brightness parameter.

In this embodiment, according to the brightness parameters of the pixels in the effective spot area, its energy response value in the Y channel is remapped to enrich the brightness level, so that in the image rendered according to the blurring of the effective spot area, the light spot area The brightness is realistic and layered.

Please refer to FIG. 2 , which exemplarily shows the complete flow of the image processing method provided by the present disclosure. As can be seen from the figure, first obtain the normal exposure frame EV0 and underexposure frame EV- collected by the image acquisition device, and then perform YUV domain conversion on EV0 and EV- respectively, that is, convert to the YUV domain, and then convert EV0 and EV- Perform image alignment, then perform intensity threshold detection on EV0 to obtain the first light spot pixel point, then perform connected domain detection to obtain the first light spot area, and then perform the same detection as EV0 on EV- to obtain the second light spot area, and use the third light spot area The second light spot area filters the first light spot area, that is, the intersection of the two is retained, and then it is judged whether the remaining first light spot area enhances the color. In the case of a light-exposure spot, the overexposed spot is color enhanced, and finally the remaining third light spot area is enhanced. The light spot energy value is remapped in the first light spot area, thereby improving the brightness level of the first light spot area, and obtaining the image to be rendered. Finally, the image to be rendered can be blurred and rendered.

According to a second aspect of the embodiment of the present disclosure, an image processing device is provided. Please refer to FIG. 3. The device includes:

The acquisition module 301 is used to acquire the first image and the second image collected by the image acquisition device for the same scene, where the first image is a normal exposure image and the second image is an underexposure image;

Determining module 302, configured to determine an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;

The rendering module 303 is configured to perform blur rendering processing on the first image according to the effective light spot area.

In some embodiments of the present disclosure, the determining module is specifically used to:

In some embodiments of the present disclosure, a color module is also included for:

In some embodiments of the present disclosure, the color module is used to determine the color parameters of each pixel according to the color parameters of the target pixel in each of the effective light spot areas, specifically for:

In some embodiments of the present disclosure, a brightness module is also included for:

In some embodiments of the present disclosure, a detection module is also included for:

In some embodiments of the present disclosure, the detection module is specifically used to:

In some embodiments of the present disclosure, when the detection module is used to determine at least one connected domain composed of the first light spot pixels as the first light spot area, it is specifically used to:

Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the first aspect of the embodiment of the method, and will not be described in detail here.

According to a third aspect of an embodiment of the present disclosure, please refer to FIG. 4 , which exemplarily shows a block diagram of an electronic device. For example, the device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

Referring to Figure 4, the device 400 may include one or more of the following components: a processing component 402, a memory 404, a power supply component 406, a multimedia component 408, an audio component 410, an input/output (I/O) interface 412, a sensor component 414, and communications component 416.

Processing component 402 generally controls the overall operations of device 400, such as operations associated with display, phone calls, data communications, camera program operations, and recording operations. The processing element 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 402 may include one or more modules that facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

Memory 404 is configured to store various types of data to support operations at device 400 . Examples of such data include instructions for any application or method operating on device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

Power component 406 provides power to various components of device 400 . Power components 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 400 .

Multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor may not only sense the boundary of the touch or sliding operation, but also detect the duration and pressure associated with the touch or sliding operation. In some embodiments, multimedia component 408 includes a front-facing camera and/or a rear-facing camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

Audio component 410 is configured to output and/or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when device 400 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or sent via communication component 416 . In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

The I/O interface 412 provides an interface between the processing component 402 and a peripheral interface module, which may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.

Sensor component 414 includes one or more sensors for providing various aspects of status assessment for device 400 . For example, the sensor component 414 can detect the open/closed state of the device 400, the relative positioning of components, such as the display and keypad of the device 400, and the sensor component 414 can also detect a change in position of the device 400 or a component of the device 400. , the presence or absence of user contact with the device 400 , device 400 orientation or acceleration/deceleration and temperature changes of the device 400 . Sensor assembly 414 may also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 416 is configured to facilitate wired or wireless communication between apparatus 400 and other devices. The device 400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G or 5G or a combination thereof. In one exemplary embodiment, the communication component 416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communications component 416 also includes a near field communications (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, apparatus 400 may be configured by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are implemented for executing the power supply method of the above electronic device.

In a fourth aspect, in an exemplary embodiment, the present disclosure also provides a non-transitory computer-readable storage medium including instructions, such as a memory 404 including instructions. The instructions can be executed by the processor 420 of the device 400 to complete the above electronic tasks. The method of powering the device. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the disclosure that follow the general principles of the disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the disclosure is limited only by the appended claims.

Claims

An image processing method, characterized in that the method includes:

Obtaining a first image and a second image collected by the image acquisition device for the same scene, wherein the first image is a normal exposure image and the second image is an underexposure image;

determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;

The first image is blurred and rendered according to the effective light spot area.
The image processing method according to claim 1, wherein the effective light spot is determined in at least one first light spot area in the first image according to at least one second light spot area in the second image. areas, including:

The first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
The image processing method according to claim 1, characterized in that, according to at least one second light spot area in the second image, it is determined to be effective in at least one first light spot area in the first image. After the light spot area, it also includes:

Determine the target pixel point of each effective light spot area, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;

In each effective light spot area, the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
The image processing method according to claim 3, characterized in that, in each effective spot area, determining the color parameters of each pixel according to the color parameters of the target pixels includes:

When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;

In the case where the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, the jth effective light spot area is determined The color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
The image processing method according to claim 1, characterized in that, based on at least one second light spot area in the second image, determining the effective value in at least one second light spot area in the first image After the light spot area, it also includes:

Determine the target brightness parameter of each effective light spot area, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective light spot area;

In each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
The image processing method according to claim 1, characterized in that, based on at least one second light spot area in the second image, determining the effective value in at least one second light spot area in the first image Before the light spot area, it also includes:

Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image.
The image processing method according to claim 6, characterized in that, performing light spot detection on the first image and the second image respectively, to obtain at least one first light spot area and the third light spot area in the first image. At least one second light spot area in the two images includes:

In the YUV domain, perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. .
The image processing method according to claim 6, characterized in that: performing light spot detection on the first image and the second image respectively to obtain at least one first light spot area and the first light spot area in the first image. At least one second light spot area in the second image includes:

Determine the pixels in the first image whose brightness is higher than the first brightness threshold as the first light spot pixels, and determine at least one connected domain composed of the first light spot pixels as the first light spot area;

Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
The image processing method according to claim 8, wherein determining at least one connected domain composed of the first light spot pixels as the first light spot area includes:

Determine at least one connected domain composed of the first light spot pixels and with a number of pixels within a preset number range as the first light spot area;

Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:

At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
An image processing device, characterized in that the device includes:

An acquisition module, configured to acquire the first image and the second image collected by the image acquisition device for the same scene, where the first image is a normal exposure image and the second image is an underexposure image;

a determining module configured to determine an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;

A rendering module, configured to perform blur rendering processing on the first image according to the effective light spot area.
The image processing device according to claim 10, characterized in that the determining module is specifically used to:

The first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
The image processing device according to claim 10, further comprising a color module for:

After determining the effective light spot area in the at least one second light spot area in the first image based on the at least one second light spot area in the second image, determining the target pixel of each effective light spot area. point, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;

In each effective light spot area, the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
The image processing device according to claim 12, wherein the color module is used to determine the color parameter of each pixel according to the color parameter of the target pixel in each of the effective light spot areas, Specifically used for:

When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;

In the case where the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, the jth effective light spot area is determined The color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
The image processing device according to claim 10, further comprising a brightness module for:

After determining the effective light spot area in the at least one second light spot area in the first image based on the at least one second light spot area in the second image, determining the target brightness of each of the effective light spot areas. Parameter, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area;

In each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
The image processing device according to claim 10, further comprising a detection module for:

Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image.
The image processing device according to claim 15, characterized in that the detection module is specifically used for:

In the YUV domain, perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. .
The image processing device according to claim 15, characterized in that the detection module is specifically used for:

Determine the pixels in the first image whose brightness is higher than the first brightness threshold as the first light spot pixels, and determine at least one connected domain composed of the first light spot pixels as the first light spot area;

Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
The image processing device according to claim 17, wherein the detection module is used to determine at least one connected domain composed of the first light spot pixel points as the first light spot area, specifically for:

Determine at least one connected domain composed of the first light spot pixels and with a number of pixels within a preset number range as the first light spot area;

Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:

At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
An electronic device, characterized in that the electronic device includes a memory and a processor, the memory is used to store computer instructions that can be run on the processor, and the processor is used to execute the computer instructions based on the claims. The image processing method according to any one of 1 to 9.
A computer-readable storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the method of any one of claims 1 to 9 is implemented.