WO2023236209A1 - Image processing method and apparatus, electronic device, and storage medium - Google Patents

Image processing method and apparatus, electronic device, and storage medium Download PDF

Info

Publication number
WO2023236209A1
WO2023236209A1 PCT/CN2022/098240 CN2022098240W WO2023236209A1 WO 2023236209 A1 WO2023236209 A1 WO 2023236209A1 CN 2022098240 W CN2022098240 W CN 2022098240W WO 2023236209 A1 WO2023236209 A1 WO 2023236209A1
Authority
WO
WIPO (PCT)
Prior art keywords
light spot
image
spot area
area
effective
Prior art date
Application number
PCT/CN2022/098240
Other languages
French (fr)
Chinese (zh)
Inventor
尹双双
董家旭
饶强
陈妹雅
刘阳晨旭
江浩
Original Assignee
北京小米移动软件有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京小米移动软件有限公司 filed Critical 北京小米移动软件有限公司
Priority to PCT/CN2022/098240 priority Critical patent/WO2023236209A1/en
Priority to CN202280004273.6A priority patent/CN117616777A/en
Publication of WO2023236209A1 publication Critical patent/WO2023236209A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/187Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60Noise processing, e.g. detecting, correcting, reducing or removing noise

Definitions

  • the present disclosure relates to the technical field of image processing, and specifically to an image processing method, device, electronic device and storage medium.
  • the camera program of the terminal device can provide a variety of photo modes, so that it has various functions of a professional camera to satisfy users in various situations.
  • Photography needs in various scenarios.
  • photography with a physical blur function is mainly realized through professional lenses of professional cameras.
  • embodiments of the present disclosure provide an image processing method, device, electronic device and storage medium to solve the defects in the related technology.
  • an image processing method including:
  • the first image is blurred and rendered according to the effective light spot area.
  • determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image includes:
  • the first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
  • the method further includes:
  • the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
  • determining the color parameters of each pixel point in each effective light spot area according to the color parameters of the target pixel point includes:
  • the i-th effective light spot When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;
  • the jth effective light spot area is determined
  • the color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
  • the method further includes:
  • the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective light spot area
  • the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
  • the method before determining the effective light spot area in the at least one second light spot area in the first image based on the at least one second light spot area in the second image, the method further includes:
  • light spot detection is performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot in the second image.
  • areas including:
  • light spot detection is performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one first light spot area in the second image.
  • the two-spot area includes:
  • Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
  • determining at least one connected domain composed of the first light spot pixels as the first light spot area includes:
  • Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:
  • At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
  • an image processing device includes:
  • An acquisition module configured to acquire the first image and the second image collected by the image acquisition device for the same scene, where the first image is a normal exposure image and the second image is an underexposure image;
  • a determining module configured to determine an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;
  • a rendering module configured to perform blur rendering processing on the first image according to the effective light spot area.
  • the determining module is specifically used to:
  • the first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
  • a color module is also included for:
  • determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image determining a target pixel for each of the effective light spot areas. point, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;
  • the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
  • the color module is used to determine the color parameters of each pixel according to the color parameters of the target pixel in each of the effective light spot areas, specifically for:
  • the i-th effective light spot When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;
  • the jth effective light spot area is determined
  • the color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
  • a brightness module is also included for:
  • determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image determining a target brightness of each effective light spot area.
  • the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area
  • the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
  • it also includes a detection module for:
  • the detection module is specifically used to:
  • the detection module is specifically used to:
  • Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
  • the detection module when used to determine at least one connected domain composed of the first light spot pixels as the first light spot area, it is specifically used to:
  • Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:
  • At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
  • an electronic device includes a memory and a processor.
  • the memory is used to store computer instructions executable on the processor.
  • the processor is used to execute the The computer instructions are based on the image processing method described in the first aspect.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.
  • the image processing method since the first image is a normal exposure image and the second image is an underexposure image, light spot areas that are mistakenly recognized such as light-colored substances in the normal exposure image will not be detected in the underexposure image. Identified as a light spot area, therefore the effective light spot area screened out in the first light spot area by using the second light spot area is more accurate, and the first light spot area that was mistakenly identified is removed, so that the third light spot area can be determined on the basis of determining the effective light spot area.
  • An image is rendered with blur, imitating the physical blur function of a professional camera. If this method is applied to the camera program of the terminal device, the functions of the camera program can be enriched and the camera effect can be closer to that of a professional camera.
  • Figure 1 is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure
  • FIG. 2 is a flowchart of an image processing method according to another exemplary embodiment of the present disclosure.
  • Figure 3 is a schematic structural diagram of an image processing device according to an exemplary embodiment of the present disclosure.
  • FIG. 4 is a structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
  • first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other.
  • first information may also be called second information, and similarly, the second information may also be called first information.
  • word “if” as used herein may be interpreted as "when” or “when” or “in response to determining.”
  • At least one embodiment of the present disclosure provides an image processing method. Please refer to FIG. 1 , which shows the flow of the method, including step S101 and step S103.
  • the method can be applied to a terminal device, for example, to an algorithm that simulates physical blur in a camera program of the terminal device.
  • the terminal device may have an image acquisition device such as a camera. These image acquisition devices can acquire images, and the camera program of the terminal device can control various parameters in the image acquisition process of the image acquisition device.
  • This method can be applied to the scene where the camera program of the terminal device captures images. That is, this method is used to blur and render the graphics collected by the image acquisition device, thereby obtaining the image output by the camera program, which is what the user does when taking pictures with the camera program. the resulting image.
  • step S101 a first image and a second image collected by the image capture device for the same scene are obtained, where the first image is a normal exposure image and the second image is an underexposure image.
  • the image acquisition device can continuously collect the first image and the second image for the same scene.
  • the same scene is the scene that the user takes pictures of, that is, the real scene in the field of view of the image acquisition device. It can be understood that this step does not limit the order in which the first image and the second image are collected, that is, the first image can be collected first and then the second image, or the second image can be collected first and then the first image, or the first image can be collected first and then the first image.
  • Different sub-cameras in the image collection device collect the first image and the second image simultaneously.
  • the first image is a normal exposure image, that is, the image collected under the normal exposure (ie, the default exposure) when the camera program takes the photo.
  • the second image is an underexposed image, that is, the exposure is smaller than the exposure when the camera program takes the photo. Images captured under normal exposure conditions. It can be understood that the proportional relationship between the exposure amount when collecting under-exposed images and the exposure amount when collecting normal-exposed images can be set in advance, such as 80%, 75%, 60%, etc.; the exposure amount can be controlled by controlling the exposure time.
  • step S102 an effective light spot area is determined in at least one first light spot area in the first image according to at least one second light spot area in the second image.
  • the first light spot area in the first image and the second light spot area in the second image can be collected in advance. That is, before step S102, light spot detection can be performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. Spot area.
  • the purpose of spot detection is to detect the spot area in the image, and the difference between the spot area and other areas is mainly reflected in the brightness. Therefore, the first image and the second image can be separately detected in the YUV domain. . If the first image and the second image are RGB images, the first image and the second image can be converted from the RGB domain to the YUV domain respectively before performing spot detection on the first image and the second image, then Spot detection can be completed using the Y channel (ie, brightness channel) of the first image and the second image.
  • the Y channel ie, brightness channel
  • methods such as brightness threshold, energy function, and deep learning can be used for spot detection.
  • light spot detection can be performed on the first image and the second image respectively in the following manner: pixels with brightness higher than the first brightness threshold in the first image are determined as first light spot pixels. , and determine at least one connected domain composed of the first light spot pixels as the first light spot area; determine the pixels with a brightness higher than the second brightness threshold in the second image as the second light spot pixels, and determine At least one connected area composed of the second light spot pixels is determined as the second light spot area.
  • the brightness value of the pixel is the value of the pixel in the Y channel.
  • the pixel with a brightness value higher than the first brightness threshold can be determined as the first light spot pixel, and the other pixels can be determined as non-first light spot pixels;
  • the pixels with a brightness value higher than the second brightness threshold can be determined as the second light spot pixels, and the other pixels can be determined as non-second light spot pixels.
  • a connected domain division standard of a four-connected connected domain or an eight-connected connected domain can be used to determine the connected domain composed of the first light spot pixels and the connected domain composed of the second light spot pixels. Then each independent connected domain in the first image can be assigned a unique label value, such as a number, etc. This label value is also the label value of the first light spot area determined by the connected domain; each independent connected domain in the second image can be assigned Each independent connected domain is assigned a unique label value, such as a number, etc., and the label value is also the label value of the second light spot area determined by the connected domain. Label values can be assigned to connected domains using two-pass or seed-fill methods.
  • the large-area light source out of focus does not form a light source. Therefore, when determining the first light spot area and the second light spot area, the area of the connected domain can be further measured, the connected domain with an area that is too large is excluded, and the connected domain with an area within a reasonable range is determined as the light spot area, so that large areas can be excluded
  • the light source is mistakenly identified as a spot area, which improves the accuracy of spot area detection.
  • the area of the connected domain can be characterized by the number of pixels in the connected domain, then at least one connected domain composed of the first light spot pixels and the number of pixels within the preset number range can be determined as the first light spot area; At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as the second light spot area.
  • the preset quantity range can be set in advance, and can be less than a preset quantity threshold (for example, 300), etc.
  • the image acquisition device may have a position change when acquiring the first image and acquiring the second image. For example, the user may shake when holding the terminal device to take pictures. Changes in the position of the image acquisition device will cause the first image and the second image not to completely overlap, but to have some deviations. Therefore, before determining the effective light spot area, the first image and the second image can be aligned.
  • One channel of the Y channel in the YUV domain can be used to complete the alignment process of the first image and the second image, that is, Align the Y channel of the first image with the Y channel of the second image, thereby completing the alignment process of the first image and the second image.
  • first increase the brightness of the second image through the Y channel histogram of the second image then use optical flow alignment to align the first image and the second image after increasing the brightness, and finally complete the first image according to the alignment result.
  • Alignment processing of an image and a second image For example, after increasing the brightness, the second image is aligned with the first image by shifting 15 pixels upward and 20 pixels to the right. Then the second image can be shifted upward. 15 pixels and offset 20 pixels to the right to align with the first image. Through the alignment process, it can be ensured that pixels at the same position on the first image and the second image correspond to the same real-life scene, thereby improving the screening accuracy of the effective spot area.
  • each first light spot area belonging to a first intersection in the at least one first light spot area can be determined as an effective light spot area, wherein the first intersection is the at least one The intersection of the first light spot area and the at least one second light spot area.
  • the first intersection can be determined based on the position coordinates of the first light spot area in the first image and the position coordinates of the second light spot area in the second image, that is, the first light spot area and the second light spot with the same position coordinates The area joins the first intersection.
  • the first image marked with the first light spot area and the second image marked with the second light spot area are superimposed, and the first light spot area with an overlapping second light spot area on the second image is determined as an effective light spot. area.
  • step S103 a blur rendering process is performed on the first image according to the effective light spot area.
  • the depth information of the current imaging scene can be calculated through the multi-camera system or deep learning algorithm of the terminal device, and then the blur corresponding to the pixels outside the focus plane is calculated based on different depth information. radius, and finally generate a picture with a blur effect based on the blur radius corresponding to each pixel.
  • the image processing method since the first image is a normal exposure image and the second image is an underexposure image, light spot areas that are mistakenly recognized such as light-colored substances in the normal exposure image will not be detected in the underexposure image. Identified as a light spot area, therefore the effective light spot area screened out in the first light spot area by using the second light spot area is more accurate, and the first light spot area that was mistakenly identified is removed, so that the third light spot area can be determined on the basis of determining the effective light spot area.
  • An image is rendered with blur, imitating the physical blur function of a professional camera. If this method is applied to the camera program of the terminal device, the functions of the camera program can be enriched and the camera effect can be closer to that of a professional camera.
  • the present disclosure can perform light spot detection on the first image and the second image respectively by acquiring the first image and the second image collected by the image acquisition device for the same scene, thereby obtaining at least one first light spot in the first image. area and at least one second spot area in the second image, and then these second spot areas in the second image can be used to determine the effective spot area among the first spot areas in the first image, that is, in the first image Some or all of these first light spot areas are determined as effective light spot areas, and finally the first image can be blurred and rendered based on these effective light spot areas.
  • the first image is a normal exposure image and the second image is an underexposure image
  • the light spot areas mistakenly recognized such as light-colored substances in the normal exposure image will not be recognized as light spot areas in the underexposure image. Therefore, the second image is used
  • the effective light spot area screened out in the first light spot area is more accurate, and the mistakenly identified first light spot area is removed, thereby achieving a better blur rendering effect and avoiding blurring of non-light spot areas. Rendered with a spot effect. If this method is applied to the camera program of the terminal device, the camera program can accurately identify the spot area in the image to be blurred, so that the camera program can achieve a better blurred rendering effect.
  • the color information in the spot area is easily lost, especially the color information in the overexposed spot area.
  • the U and V channels in the overexposed spot area will have patchy distribution, that is, N, The V channel values are all close to 128, so the color information in the effective spot area in the first image may be lost. Therefore, after determining the effective light spot area in the first light spot area in the at least one first image based on at least one second light spot area in the second image, the effective light spot area can be determined in the following manner.
  • the color information is restored: first, determine the target pixel point of each effective light spot area, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area, and the pixel point with the highest color saturation It mostly exists near the boundary of the effective light spot area, that is, within the halo. For example, the color saturation of each pixel can be judged based on the value of each pixel in the U and V channels in each effective light spot area. degree; next, in each effective light spot area, determine the color parameter of each pixel according to the color parameter of the target pixel, wherein the color parameter is used to blur the first image Rendering processing.
  • the color parameters of each pixel in the i effective light spot areas are adjusted according to the color parameters of the target pixels, where i is an integer greater than 0 and not greater than N, and the N is an integer in the first image.
  • the total number of effective spot areas This is because in this case, the original color in the effective light spot area is one color, that is, the color of the target pixel. Therefore, the color parameters of each pixel can be adjusted using the color parameters of the target pixel.
  • the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, determine the jth The color parameter of each pixel in the effective light spot area remains unchanged, where j is an integer greater than 0 and not greater than N. This is because in this case, it means that the original colors in the effective spot area are at least two colors. If the color parameters of each pixel are adjusted using the color parameters of the target pixel, it will make part of the effective spot area Pixels are adjusted to a different color than their original color.
  • the lost color information of the pixels in the effective light spot area can be restored, so that in the image rendered according to the blurring of the effective light spot area, the color information in the light spot area can be restored. Colors stay true and color loss is avoided.
  • the effective light spot area can be enhanced in the following manner Brightness level: First, determine the target brightness parameter of each effective spot area, where the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area.
  • a certain pixel The brightness parameter of a point is the value of the pixel on the Y channel; next, in each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness The parameters are used to perform blur rendering processing on the first image.
  • the brightness parameters of the pixels can be adjusted through the following formula that characterizes the gamma curve:
  • Y' is the brightness parameter adjustment result of the pixel
  • Y is the brightness parameter of the pixel
  • Y min is the target brightness parameter
  • its energy response value in the Y channel is remapped to enrich the brightness level, so that in the image rendered according to the blurring of the effective spot area, the light spot area
  • the brightness is realistic and layered.
  • FIG. 2 exemplarily shows the complete flow of the image processing method provided by the present disclosure.
  • first obtain the normal exposure frame EV0 and underexposure frame EV- collected by the image acquisition device and then perform YUV domain conversion on EV0 and EV- respectively, that is, convert to the YUV domain, and then convert EV0 and EV- Perform image alignment, then perform intensity threshold detection on EV0 to obtain the first light spot pixel point, then perform connected domain detection to obtain the first light spot area, and then perform the same detection as EV0 on EV- to obtain the second light spot area, and use the third light spot area
  • the second light spot area filters the first light spot area, that is, the intersection of the two is retained, and then it is judged whether the remaining first light spot area enhances the color.
  • the overexposed spot is color enhanced, and finally the remaining third light spot area is enhanced.
  • the light spot energy value is remapped in the first light spot area, thereby improving the brightness level of the first light spot area, and obtaining the image to be rendered.
  • the image to be rendered can be blurred and rendered.
  • an image processing device is provided. Please refer to FIG. 3.
  • the device includes:
  • the acquisition module 301 is used to acquire the first image and the second image collected by the image acquisition device for the same scene, where the first image is a normal exposure image and the second image is an underexposure image;
  • Determining module 302 configured to determine an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;
  • the rendering module 303 is configured to perform blur rendering processing on the first image according to the effective light spot area.
  • the determining module is specifically used to:
  • the first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
  • a color module is also included for:
  • determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image determining a target pixel for each of the effective light spot areas. point, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;
  • the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
  • the color module is used to determine the color parameters of each pixel according to the color parameters of the target pixel in each of the effective light spot areas, specifically for:
  • the i-th effective light spot When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;
  • the jth effective light spot area is determined
  • the color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
  • a brightness module is also included for:
  • determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image determining a target brightness of each effective light spot area.
  • the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area
  • the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
  • a detection module is also included for:
  • the detection module is specifically used to:
  • the detection module is specifically used to:
  • Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
  • the detection module when used to determine at least one connected domain composed of the first light spot pixels as the first light spot area, it is specifically used to:
  • Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:
  • At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
  • the device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.
  • the device 400 may include one or more of the following components: a processing component 402, a memory 404, a power supply component 406, a multimedia component 408, an audio component 410, an input/output (I/O) interface 412, a sensor component 414, and communications component 416.
  • Processing component 402 generally controls the overall operations of device 400, such as operations associated with display, phone calls, data communications, camera program operations, and recording operations.
  • the processing element 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above method.
  • processing component 402 may include one or more modules that facilitate interaction between processing component 402 and other components.
  • processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.
  • Memory 404 is configured to store various types of data to support operations at device 400 . Examples of such data include instructions for any application or method operating on device 400, contact data, phonebook data, messages, pictures, videos, etc.
  • Memory 404 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EEPROM erasable programmable read-only memory
  • EPROM Programmable read-only memory
  • PROM programmable read-only memory
  • ROM read-only memory
  • magnetic memory flash memory, magnetic or optical disk.
  • Power component 406 provides power to various components of device 400 .
  • Power components 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 400 .
  • Multimedia component 408 includes a screen that provides an output interface between the device 400 and the user.
  • the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user.
  • the touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor may not only sense the boundary of the touch or sliding operation, but also detect the duration and pressure associated with the touch or sliding operation.
  • multimedia component 408 includes a front-facing camera and/or a rear-facing camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data.
  • Each front-facing camera and rear-facing camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.
  • Audio component 410 is configured to output and/or input audio signals.
  • audio component 410 includes a microphone (MIC) configured to receive external audio signals when device 400 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or sent via communication component 416 .
  • audio component 410 also includes a speaker for outputting audio signals.
  • the I/O interface 412 provides an interface between the processing component 402 and a peripheral interface module, which may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.
  • Sensor component 414 includes one or more sensors for providing various aspects of status assessment for device 400 .
  • the sensor component 414 can detect the open/closed state of the device 400, the relative positioning of components, such as the display and keypad of the device 400, and the sensor component 414 can also detect a change in position of the device 400 or a component of the device 400. , the presence or absence of user contact with the device 400 , device 400 orientation or acceleration/deceleration and temperature changes of the device 400 .
  • Sensor assembly 414 may also include a proximity sensor configured to detect the presence of nearby objects without any physical contact.
  • Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications.
  • the sensor component 414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
  • Communication component 416 is configured to facilitate wired or wireless communication between apparatus 400 and other devices.
  • the device 400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G or 5G or a combination thereof.
  • the communication component 416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel.
  • the communications component 416 also includes a near field communications (NFC) module to facilitate short-range communications.
  • NFC near field communications
  • the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
  • RFID radio frequency identification
  • IrDA infrared data association
  • UWB ultra-wideband
  • Bluetooth Bluetooth
  • apparatus 400 may be configured by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are implemented for executing the power supply method of the above electronic device.
  • ASICs application specific integrated circuits
  • DSPs digital signal processors
  • DSPDs digital signal processing devices
  • PLDs programmable logic devices
  • FPGA field programmable Gate array
  • controller microcontroller, microprocessor or other electronic components are implemented for executing the power supply method of the above electronic device.
  • the present disclosure also provides a non-transitory computer-readable storage medium including instructions, such as a memory 404 including instructions.
  • the instructions can be executed by the processor 420 of the device 400 to complete the above electronic tasks.
  • the method of powering the device may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

The present disclosure relates to an image processing method and apparatus, an electronic device, and a storage medium. The method comprises: acquiring a first image and a second image which are collected by an image collection device for the same scene, wherein the first image is a normally exposed image, and the second image is an underexposed image; determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image; and performing blurring and rendering processing on the first image according to the effective light spot area.

Description

图像处理方法、装置、电子设备和存储介质Image processing methods, devices, electronic equipment and storage media 技术领域Technical field
本公开涉及图像处理技术领域,具体涉及一种图像处理方法、装置、电子设备和存储介质。The present disclosure relates to the technical field of image processing, and specifically to an image processing method, device, electronic device and storage medium.
背景技术Background technique
近年来,终端设备的功能越来越丰富,各项功能的性能也逐渐提高,例如终端设备的相机程序能够提供多种拍照模式,从而使其具有专业相机的各种功能,以满足用户在各种场景下的拍照需求。但是终端设备的相机程序与专业相机相比,还是存在一定的差距。以专业相机的物理虚化为例,专业相机在拍摄时对对焦物体所在深度的物体保持清晰,而对其他深度的物体进行模糊和虚化,从而突出拍摄主体。相关技术中,主要通过专业相机的专业镜头来实现物理虚化功能的拍照。In recent years, the functions of terminal devices have become more and more abundant, and the performance of various functions has gradually improved. For example, the camera program of the terminal device can provide a variety of photo modes, so that it has various functions of a professional camera to satisfy users in various situations. Photography needs in various scenarios. However, there is still a certain gap between the camera program of the terminal device and the professional camera. Take the physical blur of professional cameras as an example. When shooting, professional cameras keep objects at the depth of the focus object clear, while blurring and blurring objects at other depths to highlight the subject. In related technologies, photography with a physical blur function is mainly realized through professional lenses of professional cameras.
发明内容Contents of the invention
为克服相关技术中存在的问题,本公开实施例提供一种图像处理方法、装置、电子设备和存储介质,用以解决相关技术中的缺陷。In order to overcome the problems existing in the related technology, embodiments of the present disclosure provide an image processing method, device, electronic device and storage medium to solve the defects in the related technology.
根据本公开实施例的第一方面,提供一种图像处理方法,所述方法包括:According to a first aspect of an embodiment of the present disclosure, an image processing method is provided, the method including:
获取图像采集设备针对相同场景采集的第一图像和第二图像,其中,所述第一图像为正常曝光图像,第二图像为欠曝光图像;Obtaining a first image and a second image collected by the image acquisition device for the same scene, wherein the first image is a normal exposure image and the second image is an underexposure image;
根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域;determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;
根据所述有效光斑区域对所述第一图像进行虚化渲染处理。The first image is blurred and rendered according to the effective light spot area.
在一个实施例中,所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域,包括:In one embodiment, determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image includes:
将所述至少一个第一光斑区域中属于第一交集的第一光斑区域,确定为有效光斑区域,其中,所述第一交集为所述至少一个第一光斑区域和所述至少一个第二光斑区域的交集。The first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
在一个实施例中,在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域之后,还包括:In one embodiment, after determining the effective light spot area in at least one first light spot area in the first image based on the at least one second light spot area in the second image, the method further includes:
确定每个所述有效光斑区域的目标像素点,其中,所述目标像素点为所述有效光斑区域内颜色饱和度最高的像素点;Determine the target pixel point of each effective light spot area, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;
在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数,其中,所述颜色参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
在一个实施例中,所述在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数,包括:In one embodiment, determining the color parameters of each pixel point in each effective light spot area according to the color parameters of the target pixel point includes:
在第i个有效光斑区域内的每个像素点,与第i个有效光斑区域的所述目标像素点间的颜色参数差值均小于预设差值阈值的情况下,在第i个有效光斑区域内根据所述目标像素点的颜色参数对每个像素点的颜色参数进行调整,其中,i为大于0,且不大于N的整数,所述N为所述第一图像中的有效光斑区域的总数量;When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;
在第j个有效光斑区域内的至少一个像素点,与第j个有效光斑区域的所述目标像素点间的颜色参数差值大于预设差值阈值的情况下,确定第j个有效光斑区域内的每个像素点的颜色参数不变,其中,j为大于0,且不大于N的整数。In the case where the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, the jth effective light spot area is determined The color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
在一个实施例中,在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域之后,还包括:In one embodiment, after determining the effective light spot area in at least one first light spot area in the first image based on the at least one second light spot area in the second image, the method further includes:
确定每个所述有效光斑区域的目标亮度参数,其中,所述目标亮度参数为所述有效光斑区域内亮度参数最小的像素点的亮度参数;Determine the target brightness parameter of each effective light spot area, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective light spot area;
在每个所述有效光斑区域内,根据所述目标亮度参数对每个像素点的亮度参数进行调整,其中,所述亮度参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
在一个实施例中,在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第二光斑区域中确定有效光斑区域之前,还包括:In one embodiment, before determining the effective light spot area in the at least one second light spot area in the first image based on the at least one second light spot area in the second image, the method further includes:
分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image.
在一个实施例中,分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域,包括:In one embodiment, light spot detection is performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot in the second image. areas, including:
在YUV域下,分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。In the YUV domain, perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. .
在一个实施例中,所述分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域,包括:In one embodiment, light spot detection is performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one first light spot area in the second image. The two-spot area includes:
将所述第一图像中亮度高于第一亮度阈值的像素点确定为第一光斑像素点,并将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域;Determine the pixels in the first image whose brightness is higher than the first brightness threshold as the first light spot pixels, and determine at least one connected domain composed of the first light spot pixels as the first light spot area;
将所述第二图像中亮度高于第二亮度阈值的像素点确定为第二光斑像素点,并将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域。Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
在一个实施例中,所述将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域,包括:In one embodiment, determining at least one connected domain composed of the first light spot pixels as the first light spot area includes:
将所述第一光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通域确定为第一光斑区域;Determine at least one connected domain composed of the first light spot pixels and with a number of pixels within a preset number range as the first light spot area;
所述将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域,包括:Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:
将所述第二光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通区域确定为第二光斑区域。At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
根据本公开实施例的第二方面,提供一种图像处理装置,所述装置包括:According to a second aspect of an embodiment of the present disclosure, an image processing device is provided, and the device includes:
获取模块,用于获取图像采集设备针对相同场景采集的第一图像和第二图像,其中,所述第一图像为正常曝光图像,第二图像为欠曝光图像;An acquisition module, configured to acquire the first image and the second image collected by the image acquisition device for the same scene, where the first image is a normal exposure image and the second image is an underexposure image;
确定模块,用于根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域;a determining module configured to determine an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;
渲染模块,用于根据所述有效光斑区域对所述第一图像进行虚化渲染处理。A rendering module, configured to perform blur rendering processing on the first image according to the effective light spot area.
在一个实施例中,所述确定模块具体用于:In one embodiment, the determining module is specifically used to:
将所述至少一个第一光斑区域中属于第一交集的第一光斑区域,确定为有效光斑区域,其中,所述第一交集为所述至少一个第一光斑区域和所述至少一个第二光斑区域的交集。The first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
在一个实施例中,还包括颜色模块,用于:In one embodiment, a color module is also included for:
在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域之后,确定每个所述有效光斑区域的目标像素点,其中,所述目标像素点为所述有效光斑区域内颜色饱和度最高的像素点;After determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image, determining a target pixel for each of the effective light spot areas. point, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;
在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数,其中,所述颜色参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
在一个实施例中,所述颜色模块用于在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数时,具体用于:In one embodiment, the color module is used to determine the color parameters of each pixel according to the color parameters of the target pixel in each of the effective light spot areas, specifically for:
在第i个有效光斑区域内的每个像素点,与第i个有效光斑区域的所述目标像素点间的颜色参数差值均小于预设差值阈值的情况下,在第i个有效光斑区域内根据所述目标像素点的颜色参数对每个像素点的颜色参数进行调整,其中,i为大于0,且不大于N的整数,所述N为所述第一图像中的有效光斑 区域的总数量;When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;
在第j个有效光斑区域内的至少一个像素点,与第j个有效光斑区域的所述目标像素点间的颜色参数差值大于预设差值阈值的情况下,确定第j个有效光斑区域内的每个像素点的颜色参数不变,其中,j为大于0,且不大于N的整数。In the case where the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, the jth effective light spot area is determined The color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
在一个实施例中,还包括亮度模块,用于:In one embodiment, a brightness module is also included for:
在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域之后,确定每个所述有效光斑区域的目标亮度参数,其中,所述目标亮度参数为所述有效光斑区域内亮度参数最小的像素点的亮度参数;After determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image, determining a target brightness of each effective light spot area. Parameter, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area;
在每个所述有效光斑区域内,根据所述目标亮度参数对每个像素点的亮度参数进行调整,其中,所述亮度参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
在一个实施例中,还包括检测模块,用于:In one embodiment, it also includes a detection module for:
分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域;Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image;
在一个实施例中,所述检测模块具体用于:In one embodiment, the detection module is specifically used to:
在YUV域下,分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。In the YUV domain, perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. .
在一个实施例中,所述检测模块具体用于:In one embodiment, the detection module is specifically used to:
将所述第一图像中亮度高于第一亮度阈值的像素点确定为第一光斑像素点,并将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域;Determine the pixels in the first image whose brightness is higher than the first brightness threshold as the first light spot pixels, and determine at least one connected domain composed of the first light spot pixels as the first light spot area;
将所述第二图像中亮度高于第二亮度阈值的像素点确定为第二光斑像素点,并将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域。Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
在一个实施例中,所述检测模块用于将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域时,具体用于:In one embodiment, when the detection module is used to determine at least one connected domain composed of the first light spot pixels as the first light spot area, it is specifically used to:
将所述第一光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通域确定为第一光斑区域;Determine at least one connected domain composed of the first light spot pixels and with a number of pixels within a preset number range as the first light spot area;
所述将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域,包括:Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:
将所述第二光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通区域确定为第二光斑区域。At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
根据本公开实施例的第三方面,提供一种电子设备,所述电子设备包括存储器、处理器,所述存储器用于存储可在处理器上运行的计算机指令,所述处理器用于在执行所述计算机指令时基于第一方面所述的图像处理方法。According to a third aspect of an embodiment of the present disclosure, an electronic device is provided. The electronic device includes a memory and a processor. The memory is used to store computer instructions executable on the processor. The processor is used to execute the The computer instructions are based on the image processing method described in the first aspect.
根据本公开实施例的第四方面,提供一种计算机可读存储介质,其上存储有计算机程序,所述程序被处理器执行时实现第一方面所述的方法。According to a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.
本公开的实施例提供的技术方案可以包括以下有益效果:The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:
本公开所提供的图像处理方法,由于第一图像为正常曝光图像,第二图像为欠曝光图像,因此正常曝光图像中的浅色物质等误识别的光斑区域,在欠曝光图像中不会被识别为光斑区域,因此利用第二光斑区域在第一光斑区域中所筛选出的有效光斑区域较为准确,去除了误识别到的第一光斑区域,从而可以在确定有效光斑区域的基础上对第一图像进行虚化渲染处理,模仿专业相机的物理虚化功能。若将该方法应用于终端设备的相机程序中,则可以使得相机程序的功能更加丰富,更加贴近专业相机的拍照效果。In the image processing method provided by the present disclosure, since the first image is a normal exposure image and the second image is an underexposure image, light spot areas that are mistakenly recognized such as light-colored substances in the normal exposure image will not be detected in the underexposure image. Identified as a light spot area, therefore the effective light spot area screened out in the first light spot area by using the second light spot area is more accurate, and the first light spot area that was mistakenly identified is removed, so that the third light spot area can be determined on the basis of determining the effective light spot area. An image is rendered with blur, imitating the physical blur function of a professional camera. If this method is applied to the camera program of the terminal device, the functions of the camera program can be enriched and the camera effect can be closer to that of a professional camera.
附图说明Description of the drawings
此处的附图被并入说明书中并构成本说明书的一部分,示出了符合本发明的实施例,并与说明书一起用于解释本发明的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.
图1是本公开一示例性实施例示出的图像处理方法的流程图;Figure 1 is a flow chart of an image processing method according to an exemplary embodiment of the present disclosure;
图2是本公开另一示例性实施例示出的图像处理方法的流程图;FIG. 2 is a flowchart of an image processing method according to another exemplary embodiment of the present disclosure;
图3是本公开一示例性实施例示出的图像处理装置的结构示意图;Figure 3 is a schematic structural diagram of an image processing device according to an exemplary embodiment of the present disclosure;
图4是本公开一示例性实施例示出的电子设备的结构框图。FIG. 4 is a structural block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
具体实施方式Detailed ways
这里将详细地对示例性实施例进行说明,其示例表示在附图中。下面的描述涉及附图时,除非另有表示,不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反,它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with aspects of the disclosure as detailed in the appended claims.
在本公开使用的术语是仅仅出于描述特定实施例的目的,而非旨在限制本公开。在本公开和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式,除非上下文清楚地表示其他含义。还应当理解,本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the" and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
应当理解,尽管在本公开可能采用术语第一、第二、第三等来描述各种信息,但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如,在不脱离本公开范围的情况下,第一信息也可以被称为第二信息,类似地,第二信息也可以被称为第一信息。取决于语境,如在此所使用的词语“如果”可以被解释成为“在……时”或“当……时”或“响应于确定”。It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of the present disclosure, the first information may also be called second information, and similarly, the second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when" or "in response to determining."
当使用专业的相机拍摄图片时,如果采用长焦或大光圈镜头,那么会得到景深较小的图片,对焦物体及其所在深度的其他物体会保持清晰,而前景和背景会有不同程度的模糊和虚化,这可以起到突出摄影主体的作用。其中在被虚化的背景或前景中,点状光源往往由于其更高的能力密度,会在成像平面被虚化成为光斑。一般来说,点状光源的亮度越大,距离对焦平面越远,所形成的光斑半径也会越大。When using a professional camera to take pictures, if you use a telephoto or large aperture lens, you will get a picture with a smaller depth of field. The focus object and other objects at the depth will remain clear, while the foreground and background will be blurred to varying degrees. and blur, which can play a role in highlighting the subject of the photograph. Among them, in the blurred background or foreground, point light sources are often blurred into light spots on the imaging plane due to their higher power density. Generally speaking, the greater the brightness of the point light source and the farther it is from the focus plane, the larger the radius of the spot formed will be.
当前由于智能手机等终端设备便携性与成本的要求,往往会采用尺寸较小的摄像头,这导致手机摄影很难拍摄出具有虚化效果的图片,因此在智能手机的相机程序中引入软件算法来模拟物理虚化,也就是将采集到的原始图像进行虚化渲染等处理,但是当采集到的原始图像中存在光斑时,光斑区域 的识别不够准确,而且虚化渲染的结果中亮度层次较差,颜色信息较少。Currently, due to the portability and cost requirements of terminal devices such as smartphones, smaller cameras are often used, which makes it difficult to take pictures with a blurred effect in mobile phone photography. Therefore, software algorithms are introduced into the camera programs of smartphones. Simulate physical blur, that is, perform blur rendering and other processing on the original image collected. However, when there are light spots in the original image collected, the identification of the spot area is not accurate enough, and the brightness level of the blur rendering result is poor. , less color information.
第一方面,本公开至少一个实施例提供了一种图像处理方法,请参照附图1,其示出了该方法的流程,包括步骤S101和步骤S103。In a first aspect, at least one embodiment of the present disclosure provides an image processing method. Please refer to FIG. 1 , which shows the flow of the method, including step S101 and step S103.
其中,该方法可以应用于终端设备,例如应用于终端设备的相机程序中的模拟物理虚化的算法中。终端设备可以具有摄像头等图像采集设备,这些图像采集设备可以采集图像,而且终端设备的相机程序可以控制图像采集设备采集图像过程中的各项参数。该方法可以应用于终端设备的相机程序拍摄图像的场景下,即利用该方法对图像采集设备采集的图形进行虚化渲染难处理,从而得到相机程序输出的图像,即用户利用相机程序拍照时所得到的图像。The method can be applied to a terminal device, for example, to an algorithm that simulates physical blur in a camera program of the terminal device. The terminal device may have an image acquisition device such as a camera. These image acquisition devices can acquire images, and the camera program of the terminal device can control various parameters in the image acquisition process of the image acquisition device. This method can be applied to the scene where the camera program of the terminal device captures images. That is, this method is used to blur and render the graphics collected by the image acquisition device, thereby obtaining the image output by the camera program, which is what the user does when taking pictures with the camera program. the resulting image.
在步骤S101中,获取图像采集设备针对相同场景采集的第一图像和第二图像,其中,所述第一图像为正常曝光图像,第二图像为欠曝光图像。In step S101, a first image and a second image collected by the image capture device for the same scene are obtained, where the first image is a normal exposure image and the second image is an underexposure image.
其中,在终端设备的相机程序被启动,且用户通过操作触发相机程序的拍摄功能时,图像采集设备可以针对相同场景连续采集第一图像和第二图像。相同场景即为用户拍照所针对的场景,即图像采集设备的视野中的现实场景。可以理解的是,本步骤并未限定第一图像和第二图像的采集顺序,即可以先采集第一图像再采集第二图像,也可以先采集第二图像再采集第一图像,还可以通过图像采集设备中的不同子摄像头同时采集第一图像和第二图像。Wherein, when the camera program of the terminal device is started and the user triggers the shooting function of the camera program through operation, the image acquisition device can continuously collect the first image and the second image for the same scene. The same scene is the scene that the user takes pictures of, that is, the real scene in the field of view of the image acquisition device. It can be understood that this step does not limit the order in which the first image and the second image are collected, that is, the first image can be collected first and then the second image, or the second image can be collected first and then the first image, or the first image can be collected first and then the first image. Different sub-cameras in the image collection device collect the first image and the second image simultaneously.
第一图像为正常曝光图像,即在相机程序拍摄照片时的正常曝光量(即默认曝光量)的情况下采集的图像,第二图像为欠曝光图像,即曝光量小于相机程序拍摄照片时的正常曝光量的情况下采集的图像。可以理解的是,欠曝光图像采集时的曝光量与正常曝光图像采集时的曝光量比例关系可以预先设置,例如80%、75%、60%等;可以通过控制曝光时间来控制曝光量。The first image is a normal exposure image, that is, the image collected under the normal exposure (ie, the default exposure) when the camera program takes the photo. The second image is an underexposed image, that is, the exposure is smaller than the exposure when the camera program takes the photo. Images captured under normal exposure conditions. It can be understood that the proportional relationship between the exposure amount when collecting under-exposed images and the exposure amount when collecting normal-exposed images can be set in advance, such as 80%, 75%, 60%, etc.; the exposure amount can be controlled by controlling the exposure time.
在步骤S102中,根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域。In step S102, an effective light spot area is determined in at least one first light spot area in the first image according to at least one second light spot area in the second image.
第一图像中的第一光斑区域和第二图像中的第二光斑区域,可以预先采集得到。即在步骤S102之前,可以分别对所述第一图像和所述第二图像进行 光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。The first light spot area in the first image and the second light spot area in the second image can be collected in advance. That is, before step S102, light spot detection can be performed on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. Spot area.
光斑检测的目的是检测到图像中的光斑区域,而光斑区域与其他区域的区别主要体现在亮度方面,因此可以在YUV域下,分别对所述第一图像和所述第二图像进行光斑检测。若第一图像和第二图像为RGB图像,则可以在对第一图像和第二图像进行光斑检测前,分别将所述第一图像和所述第二图像由RGB域转换至YUV域,则可以利用第一图像和第二图像的Y通道(即亮度通道)来完成光斑检测。The purpose of spot detection is to detect the spot area in the image, and the difference between the spot area and other areas is mainly reflected in the brightness. Therefore, the first image and the second image can be separately detected in the YUV domain. . If the first image and the second image are RGB images, the first image and the second image can be converted from the RGB domain to the YUV domain respectively before performing spot detection on the first image and the second image, then Spot detection can be completed using the Y channel (ie, brightness channel) of the first image and the second image.
可选的,可以采用亮度阈值、能量函数、深度学习等方法进行光斑检测。以亮度阈值的方法为例,可以按照下述方式分别对第一图像和第二图像进行光斑检测:将所述第一图像中亮度高于第一亮度阈值的像素点确定为第一光斑像素点,并将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域;将所述第二图像中亮度高于第二亮度阈值的像素点确定为第二光斑像素点,并将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域。Optionally, methods such as brightness threshold, energy function, and deep learning can be used for spot detection. Taking the method of brightness threshold as an example, light spot detection can be performed on the first image and the second image respectively in the following manner: pixels with brightness higher than the first brightness threshold in the first image are determined as first light spot pixels. , and determine at least one connected domain composed of the first light spot pixels as the first light spot area; determine the pixels with a brightness higher than the second brightness threshold in the second image as the second light spot pixels, and determine At least one connected area composed of the second light spot pixels is determined as the second light spot area.
其中,像素点的亮度值即为Y通道中像素点的值。通过遍历第一图像的每个像素点的亮度值,可以将亮度值高于第一亮度阈值的像素点确定为第一光斑像素点,将其他像素点确定为非第一光斑像素点;通过遍历第二图像的每个像素点的亮度值,可以将亮度值高于第二亮度阈值的像素点确定为第二光斑像素点,将其他像素点确定为非第二光斑像素点。Among them, the brightness value of the pixel is the value of the pixel in the Y channel. By traversing the brightness value of each pixel of the first image, the pixel with a brightness value higher than the first brightness threshold can be determined as the first light spot pixel, and the other pixels can be determined as non-first light spot pixels; by traversing For the brightness value of each pixel of the second image, the pixels with a brightness value higher than the second brightness threshold can be determined as the second light spot pixels, and the other pixels can be determined as non-second light spot pixels.
可以采用四连接连通域或八连接连通域的连通域划分标准来确定第一光斑像素点组成的连通域和第二光斑像素点组成的连通域。然后可以对第一图像中的每个独立的连通域赋予唯一的标签值,例如编号等,该标签值也就是连通域所确定的第一光斑区域的标签值;可以对第二图像中的每个独立的连通域赋予唯一的标签值,例如编号等,该标签值也就是连通域所确定的第二光斑区域的标签值。可以采用two-pass、或seed-fill等方式对连通域赋予标签值。A connected domain division standard of a four-connected connected domain or an eight-connected connected domain can be used to determine the connected domain composed of the first light spot pixels and the connected domain composed of the second light spot pixels. Then each independent connected domain in the first image can be assigned a unique label value, such as a number, etc. This label value is also the label value of the first light spot area determined by the connected domain; each independent connected domain in the second image can be assigned Each independent connected domain is assigned a unique label value, such as a number, etc., and the label value is also the label value of the second light spot area determined by the connected domain. Label values can be assigned to connected domains using two-pass or seed-fill methods.
由于光斑是由于焦外点状光源形成的,而焦外的大面积光源并不会形成光源。因此在确定第一光斑区域和第二光斑区域时,可以进一步衡量连通域的面积,将面积过大的连通域排除,将面积在合理范围内的连通域确定为光斑区域,这样可以排除大面积光源误识别为光斑区域,提高光斑区域检测的准确性。连通域的面积可以采用连通域内像素点的数量进行表征,则可以将所述第一光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通域确定为第一光斑区域;将所述第二光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通区域确定为第二光斑区域。预设数量范围可以预先设置,可以为小于预先设置的数量阈值(例如300)等。Because the light spot is formed by the out-of-focus point light source, the large-area light source out of focus does not form a light source. Therefore, when determining the first light spot area and the second light spot area, the area of the connected domain can be further measured, the connected domain with an area that is too large is excluded, and the connected domain with an area within a reasonable range is determined as the light spot area, so that large areas can be excluded The light source is mistakenly identified as a spot area, which improves the accuracy of spot area detection. The area of the connected domain can be characterized by the number of pixels in the connected domain, then at least one connected domain composed of the first light spot pixels and the number of pixels within the preset number range can be determined as the first light spot area; At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as the second light spot area. The preset quantity range can be set in advance, and can be less than a preset quantity threshold (for example, 300), etc.
图像采集设备在采集第一图像和采集第二图像时可能存在位置变化,例如用户手持终端设备进行拍照时发生抖动等。图像采集设备的位置变化会使得第一图像和第二图像并非完全成重合,而是存在一些偏差的。因此在确定有效光斑区域前,可以对所述第一图像和所述第二图像进行对齐处理,可以采用YUV域下的Y通道一个通道来完成第一图像和第二图像的对齐处理,也就是将第一图像的Y通道第二图像的Y通道对齐,从而完成第一图像和第二图像的对齐处理。示例性的,先通过第二图像的Y通道直方图来提高第二图像的亮度,然后采用光流对齐的方式对第一图像和提高亮度之后的第二图像进行对齐,最后根据对齐结果完成第一图像和第二图像的对齐处理,例如提高亮度之后的第二图像通过向上偏移15个像素点和向右偏移20个像素点与第一图像对齐,则可以将第二图像向上偏移15个像素点,并向右偏移20个像素点,从而与第一图像对齐。通过对齐处理,可以保证第一图像和第二图像上相同位置的像素对应相同的现实场景,从而可以提高有效光斑区域的筛选准确度。The image acquisition device may have a position change when acquiring the first image and acquiring the second image. For example, the user may shake when holding the terminal device to take pictures. Changes in the position of the image acquisition device will cause the first image and the second image not to completely overlap, but to have some deviations. Therefore, before determining the effective light spot area, the first image and the second image can be aligned. One channel of the Y channel in the YUV domain can be used to complete the alignment process of the first image and the second image, that is, Align the Y channel of the first image with the Y channel of the second image, thereby completing the alignment process of the first image and the second image. For example, first increase the brightness of the second image through the Y channel histogram of the second image, then use optical flow alignment to align the first image and the second image after increasing the brightness, and finally complete the first image according to the alignment result. Alignment processing of an image and a second image. For example, after increasing the brightness, the second image is aligned with the first image by shifting 15 pixels upward and 20 pixels to the right. Then the second image can be shifted upward. 15 pixels and offset 20 pixels to the right to align with the first image. Through the alignment process, it can be ensured that pixels at the same position on the first image and the second image correspond to the same real-life scene, thereby improving the screening accuracy of the effective spot area.
在一个可能的实施例中,可以将所述至少一个第一光斑区域中属于第一交集的每个第一光斑区域,均确定为有效光斑区域,其中,所述第一交集为所述至少一个第一光斑区域和所述至少一个第二光斑区域的交集。示例性的,可以根据第一光斑区域在第一图像中的位置坐标和第二光斑区域在第二图像 中的位置坐标来确定第一交集,即将位置坐标相同的第一光斑区域和第二光斑区域加入第一交集。再示例性的,将标注第一光斑区域的第一图像和标注第二光斑区域的第二图像进行叠加,将在第二图像上存在重叠的第二光斑区域的第一光斑区域确定为有效光斑区域。In a possible embodiment, each first light spot area belonging to a first intersection in the at least one first light spot area can be determined as an effective light spot area, wherein the first intersection is the at least one The intersection of the first light spot area and the at least one second light spot area. For example, the first intersection can be determined based on the position coordinates of the first light spot area in the first image and the position coordinates of the second light spot area in the second image, that is, the first light spot area and the second light spot with the same position coordinates The area joins the first intersection. As another example, the first image marked with the first light spot area and the second image marked with the second light spot area are superimposed, and the first light spot area with an overlapping second light spot area on the second image is determined as an effective light spot. area.
在步骤S103中,根据所述有效光斑区域对所述第一图像进行虚化渲染处理。In step S103, a blur rendering process is performed on the first image according to the effective light spot area.
对第一图像进行虚化渲染处理时,可以通过终端设备的多摄系统或深度学习算法来计算出当前成像场景的深度信息,然后根据不同深度信息来计算出对焦平面外像素点所对应的模糊半径,最后根据各个像素点所对应的模糊半径,生成具有虚化效果的图片。When performing blur rendering processing on the first image, the depth information of the current imaging scene can be calculated through the multi-camera system or deep learning algorithm of the terminal device, and then the blur corresponding to the pixels outside the focus plane is calculated based on different depth information. radius, and finally generate a picture with a blur effect based on the blur radius corresponding to each pixel.
本公开所提供的图像处理方法,由于第一图像为正常曝光图像,第二图像为欠曝光图像,因此正常曝光图像中的浅色物质等误识别的光斑区域,在欠曝光图像中不会被识别为光斑区域,因此利用第二光斑区域在第一光斑区域中所筛选出的有效光斑区域较为准确,去除了误识别到的第一光斑区域,从而可以在确定有效光斑区域的基础上对第一图像进行虚化渲染处理,模仿专业相机的物理虚化功能。若将该方法应用于终端设备的相机程序中,则可以使得相机程序的功能更加丰富,更加贴近专业相机的拍照效果。In the image processing method provided by the present disclosure, since the first image is a normal exposure image and the second image is an underexposure image, light spot areas that are mistakenly recognized such as light-colored substances in the normal exposure image will not be detected in the underexposure image. Identified as a light spot area, therefore the effective light spot area screened out in the first light spot area by using the second light spot area is more accurate, and the first light spot area that was mistakenly identified is removed, so that the third light spot area can be determined on the basis of determining the effective light spot area. An image is rendered with blur, imitating the physical blur function of a professional camera. If this method is applied to the camera program of the terminal device, the functions of the camera program can be enriched and the camera effect can be closer to that of a professional camera.
具体来说,本公开通过获取图像采集设备针对相同场景采集的第一图像和第二图像,可以分别对第一图像和第二图像进行光斑检测,从而得到第一图像中的至少一个第一光斑区域和第二图像中的至少一个第二光斑区域,然后可以利用第二图像中的这些第二光斑区域,在第一图像中的这些第一光斑区域中确定有效光斑区域,即将第一图像中的这些第一光斑区域中的部分或全部确定为有效光斑区域,最后可以根据这些有效光斑区域对第一图像进行虚化渲染处理。由于第一图像为正常曝光图像,第二图像为欠曝光图像,因此正常曝光图像中的浅色物质等误识别的光斑区域,在欠曝光图像中不会被识别为光斑区域,因此利用第二光斑区域在第一光斑区域中所筛选出的有效光斑区域较为准确,去除了误识别到的第一光斑区域,从而取得较好的虚化 渲染的效果,还能够避免了将非光斑区域虚化渲染为光斑效果。若将该方法应用于终端设备的相机程序中,则可以使得相机程序对待虚化渲染图像中的光斑区域进行准确识别,从而使相机程序取得较好的虚化渲染效果。Specifically, the present disclosure can perform light spot detection on the first image and the second image respectively by acquiring the first image and the second image collected by the image acquisition device for the same scene, thereby obtaining at least one first light spot in the first image. area and at least one second spot area in the second image, and then these second spot areas in the second image can be used to determine the effective spot area among the first spot areas in the first image, that is, in the first image Some or all of these first light spot areas are determined as effective light spot areas, and finally the first image can be blurred and rendered based on these effective light spot areas. Since the first image is a normal exposure image and the second image is an underexposure image, the light spot areas mistakenly recognized such as light-colored substances in the normal exposure image will not be recognized as light spot areas in the underexposure image. Therefore, the second image is used The effective light spot area screened out in the first light spot area is more accurate, and the mistakenly identified first light spot area is removed, thereby achieving a better blur rendering effect and avoiding blurring of non-light spot areas. Rendered with a spot effect. If this method is applied to the camera program of the terminal device, the camera program can accurately identify the spot area in the image to be blurred, so that the camera program can achieve a better blurred rendering effect.
本公开的一些实施例中,光斑区域内的颜色信息容易丢失,尤其是过曝光的光斑区域内的颜色信息更容易丢失,过曝光的光斑区域的U、V通道会出现斑状分布,即N、V通道值均接近128,因此第一图像中的有效光斑区域内的颜色信息存在丢失的可能。因此可以在所述根据所述第二图像中的至少一个第二光斑区域,在所述至少一个第一图像中的第一光斑区域中确定有效光斑区域之后,按照下述方式对有效光斑区域内的颜色信息进行恢复:首先,确定每个所述有效光斑区域的目标像素点,其中,所述目标像素点为所述有效光斑区域内颜色饱和度最高的像素点,颜色饱和度最高的像素点多存在于有效光斑区域靠近边界的位置,即光晕内,示例性的,可以根据每个有效光斑区域内的每个像素点在U、V通道上的值来判断每个像素点的颜色饱和度;接下来,在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数,其中,所述颜色参数用于对所述第一图像进行虚化渲染处理。In some embodiments of the present disclosure, the color information in the spot area is easily lost, especially the color information in the overexposed spot area. The U and V channels in the overexposed spot area will have patchy distribution, that is, N, The V channel values are all close to 128, so the color information in the effective spot area in the first image may be lost. Therefore, after determining the effective light spot area in the first light spot area in the at least one first image based on at least one second light spot area in the second image, the effective light spot area can be determined in the following manner. The color information is restored: first, determine the target pixel point of each effective light spot area, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area, and the pixel point with the highest color saturation It mostly exists near the boundary of the effective light spot area, that is, within the halo. For example, the color saturation of each pixel can be judged based on the value of each pixel in the U and V channels in each effective light spot area. degree; next, in each effective light spot area, determine the color parameter of each pixel according to the color parameter of the target pixel, wherein the color parameter is used to blur the first image Rendering processing.
示例性的,在第i个有效光斑区域内的每个像素点,与第i个有效光斑区域的所述目标像素点间的颜色参数差值均小于预设差值阈值的情况下,在第i个有效光斑区域内根据所述目标像素点的颜色参数对每个像素点的颜色参数进行调整,其中,i为大于0,且不大于N的整数,所述N为所述第一图像中的有效光斑区域的总数量。这是因为这种情况下说明该有效光斑区域内原本的颜色为一种颜色,即目标像素点的颜色,因此可以采用目标像素点的颜色参数对每个像素点的颜色参数进行调整。具体调整时,可以直接使用目标像素点在U通道上的值替换每个像素点在U通道上的值,使用目标像素点在V通道上的值替换每个像素点在V通道上的值;或者,也可以针对每个像素点这样操作:随机使用目标像素点在U通道上的值某个像素点在U通道上的值之间的某个值,替换该像素点在U通道上的值,随机使用目标像素点在V通 道上的值某个像素点在V通道上的值之间的某个值,替换该像素点在V通道上的值。For example, when the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, in the The color parameters of each pixel in the i effective light spot areas are adjusted according to the color parameters of the target pixels, where i is an integer greater than 0 and not greater than N, and the N is an integer in the first image. The total number of effective spot areas. This is because in this case, the original color in the effective light spot area is one color, that is, the color of the target pixel. Therefore, the color parameters of each pixel can be adjusted using the color parameters of the target pixel. When making specific adjustments, you can directly use the value of the target pixel on the U channel to replace the value of each pixel on the U channel, and use the value of the target pixel on the V channel to replace the value of each pixel on the V channel; Alternatively, you can also do this for each pixel: randomly use a value between the value of the target pixel on the U channel and a value between the values of a certain pixel on the U channel to replace the value of the pixel on the U channel. , randomly replace the value of the pixel on the V channel with a value between the value of the target pixel on the V channel and the value of a certain pixel on the V channel.
示例性的,在第j个有效光斑区域内的至少一个像素点,与第j个有效光斑区域的所述目标像素点间的颜色参数差值大于预设差值阈值的情况下,确定第j个有效光斑区域内的每个像素点的颜色参数不变,其中,j为大于0,且不大于N的整数。这是因为这种情况下说明该有效光斑区域内原本的颜色为至少两种颜色,若使用目标像素点的颜色参数对每个像素点的颜色参数进行调整,则会使得有效光斑区域内的部分像素调整为与其原本颜色不同的颜色。Exemplarily, when the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, determine the jth The color parameter of each pixel in the effective light spot area remains unchanged, where j is an integer greater than 0 and not greater than N. This is because in this case, it means that the original colors in the effective spot area are at least two colors. If the color parameters of each pixel are adjusted using the color parameters of the target pixel, it will make part of the effective spot area Pixels are adjusted to a different color than their original color.
本实施例中,通过确定有效光斑区域内的目标像素点,可以对有效光斑区域内的像素点丢失的颜色信息进行恢复,从而使得根据有效光斑区域所虚化渲染得到的图像中,光斑区域内颜色保持真实状态,避免出现颜色丢失。In this embodiment, by determining the target pixels in the effective light spot area, the lost color information of the pixels in the effective light spot area can be restored, so that in the image rendered according to the blurring of the effective light spot area, the color information in the light spot area can be restored. Colors stay true and color loss is avoided.
本公开的一些实施例中,由于终端设备的成像系统的宽容度比较低,因此有效光斑区域内很难根据实际亮度产生亮度层次。因此可以在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域之后,按照下述方式增强有效光斑区域内的亮度层次:首先,确定每个所述有效光斑区域的目标亮度参数,其中,所述目标亮度参数为所述有效光斑区域内亮度参数最小的像素点的亮度参数,示例性的,某个像素点的亮度参数即为该像素在Y通道上的值;接下来,在每个所述有效光斑区域内,根据所述目标亮度参数对每个像素点的亮度参数进行调整,其中,所述亮度参数用于对所述第一图像进行虚化渲染处理。示例性的,由于人眼对亮度的感受是非线性的,因此可以通过表征gamma曲线的下述公式对像素点的亮度参数进行调整:In some embodiments of the present disclosure, due to the relatively low latitude of the imaging system of the terminal device, it is difficult to generate brightness levels based on actual brightness in the effective spot area. Therefore, after the effective light spot area is determined in the at least one first light spot area in the first image based on the at least one second light spot area in the second image, the effective light spot area can be enhanced in the following manner Brightness level: First, determine the target brightness parameter of each effective spot area, where the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area. For example, a certain pixel The brightness parameter of a point is the value of the pixel on the Y channel; next, in each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness The parameters are used to perform blur rendering processing on the first image. For example, since the human eye's perception of brightness is non-linear, the brightness parameters of the pixels can be adjusted through the following formula that characterizes the gamma curve:
Figure PCTCN2022098240-appb-000001
Figure PCTCN2022098240-appb-000001
其中,Y’为像素点的亮度参数调整结果,Y为像素点的亮度参数,Y min为目标亮度参数。 Among them, Y' is the brightness parameter adjustment result of the pixel, Y is the brightness parameter of the pixel, and Y min is the target brightness parameter.
本实施例中,根据有效光斑区域内像素点的亮度参数,将其在Y通道的能量响应值重新映射来丰富亮度层次,从而使得根据有效光斑区域所虚化渲染得到的图像中,光斑区域内亮度具有真实感和层次感。In this embodiment, according to the brightness parameters of the pixels in the effective spot area, its energy response value in the Y channel is remapped to enrich the brightness level, so that in the image rendered according to the blurring of the effective spot area, the light spot area The brightness is realistic and layered.
请参照附图2,其示例性的示出了本公开所提供的图像处理方法的完整流程。从图中可以看出,首先获取图像采集设备所采集的正常曝光帧EV0和欠曝光帧EV-,然后分别对EV0和EV-进行YUV域转换,即转换至YUV域,然后将EV0和EV-进行图像对齐,然后对EV0进行强度阈值检测,得到第一光斑像素点,然后进行连通域检测得到第一光斑区域,然后对EV-进行与EV0相同的检测,得到第二光斑区域,并使用第二光斑区域对第一光斑区域进行筛选,即保留二者的交集,然后判断剩余的第一光斑区域是否增强颜色,在具有光爆光斑的情况对过曝光斑进行颜色增强,最后对剩余的第一光斑区域进行光斑能量值重映射,从而提高第一光斑区域的亮度层次,得到待渲染图像,最后便可以对待渲染图像进行虚化渲染处理。Please refer to FIG. 2 , which exemplarily shows the complete flow of the image processing method provided by the present disclosure. As can be seen from the figure, first obtain the normal exposure frame EV0 and underexposure frame EV- collected by the image acquisition device, and then perform YUV domain conversion on EV0 and EV- respectively, that is, convert to the YUV domain, and then convert EV0 and EV- Perform image alignment, then perform intensity threshold detection on EV0 to obtain the first light spot pixel point, then perform connected domain detection to obtain the first light spot area, and then perform the same detection as EV0 on EV- to obtain the second light spot area, and use the third light spot area The second light spot area filters the first light spot area, that is, the intersection of the two is retained, and then it is judged whether the remaining first light spot area enhances the color. In the case of a light-exposure spot, the overexposed spot is color enhanced, and finally the remaining third light spot area is enhanced. The light spot energy value is remapped in the first light spot area, thereby improving the brightness level of the first light spot area, and obtaining the image to be rendered. Finally, the image to be rendered can be blurred and rendered.
根据本公开实施例的第二方面,提供一种图像处理装置,请参照附图3,所述装置包括:According to a second aspect of the embodiment of the present disclosure, an image processing device is provided. Please refer to FIG. 3. The device includes:
获取模块301,用于获取图像采集设备针对相同场景采集的第一图像和第二图像,其中,所述第一图像为正常曝光图像,第二图像为欠曝光图像;The acquisition module 301 is used to acquire the first image and the second image collected by the image acquisition device for the same scene, where the first image is a normal exposure image and the second image is an underexposure image;
确定模块302,用于根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域;Determining module 302, configured to determine an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;
渲染模块303,用于根据所述有效光斑区域对所述第一图像进行虚化渲染处理。The rendering module 303 is configured to perform blur rendering processing on the first image according to the effective light spot area.
在本公开的一些实施例中,所述确定模块具体用于:In some embodiments of the present disclosure, the determining module is specifically used to:
将所述至少一个第一光斑区域中属于第一交集的第一光斑区域,确定为有效光斑区域,其中,所述第一交集为所述至少一个第一光斑区域和所述至少一个第二光斑区域的交集。The first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
在本公开的一些实施例中,还包括颜色模块,用于:In some embodiments of the present disclosure, a color module is also included for:
在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像 中的至少一个第一光斑区域中确定有效光斑区域之后,确定每个所述有效光斑区域的目标像素点,其中,所述目标像素点为所述有效光斑区域内颜色饱和度最高的像素点;After determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image, determining a target pixel for each of the effective light spot areas. point, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;
在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数,其中,所述颜色参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
在本公开的一些实施例中,所述颜色模块用于在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数时,具体用于:In some embodiments of the present disclosure, the color module is used to determine the color parameters of each pixel according to the color parameters of the target pixel in each of the effective light spot areas, specifically for:
在第i个有效光斑区域内的每个像素点,与第i个有效光斑区域的所述目标像素点间的颜色参数差值均小于预设差值阈值的情况下,在第i个有效光斑区域内根据所述目标像素点的颜色参数对每个像素点的颜色参数进行调整,其中,i为大于0,且不大于N的整数,所述N为所述第一图像中的有效光斑区域的总数量;When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;
在第j个有效光斑区域内的至少一个像素点,与第j个有效光斑区域的所述目标像素点间的颜色参数差值大于预设差值阈值的情况下,确定第j个有效光斑区域内的每个像素点的颜色参数不变,其中,j为大于0,且不大于N的整数。In the case where the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, the jth effective light spot area is determined The color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
在本公开的一些实施例中,还包括亮度模块,用于:In some embodiments of the present disclosure, a brightness module is also included for:
在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域之后,确定每个所述有效光斑区域的目标亮度参数,其中,所述目标亮度参数为所述有效光斑区域内亮度参数最小的像素点的亮度参数;After determining an effective light spot area in at least one first light spot area in the first image based on at least one second light spot area in the second image, determining a target brightness of each effective light spot area. Parameter, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area;
在每个所述有效光斑区域内,根据所述目标亮度参数对每个像素点的亮度参数进行调整,其中,所述亮度参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
在本公开的一些实施例中,还包括检测模块,用于:In some embodiments of the present disclosure, a detection module is also included for:
分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域;Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image;
在本公开的一些实施例中,所述检测模块具体用于:In some embodiments of the present disclosure, the detection module is specifically used to:
在YUV域下,分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。In the YUV domain, perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. .
在本公开的一些实施例中,所述检测模块具体用于:In some embodiments of the present disclosure, the detection module is specifically used to:
将所述第一图像中亮度高于第一亮度阈值的像素点确定为第一光斑像素点,并将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域;Determine the pixels in the first image whose brightness is higher than the first brightness threshold as the first light spot pixels, and determine at least one connected domain composed of the first light spot pixels as the first light spot area;
将所述第二图像中亮度高于第二亮度阈值的像素点确定为第二光斑像素点,并将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域。Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
在本公开的一些实施例中,所述检测模块用于将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域时,具体用于:In some embodiments of the present disclosure, when the detection module is used to determine at least one connected domain composed of the first light spot pixels as the first light spot area, it is specifically used to:
将所述第一光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通域确定为第一光斑区域;Determine at least one connected domain composed of the first light spot pixels and with a number of pixels within a preset number range as the first light spot area;
所述将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域,包括:Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:
将所述第二光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通区域确定为第二光斑区域。At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
关于上述实施例中的装置,其中各个模块执行操作的具体方式已经在第一方面有关该方法的实施例中进行了详细描述,此处将不做详细阐述说明。Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the first aspect of the embodiment of the method, and will not be described in detail here.
根据本公开实施例的第三方面,请参照附图4,其示例性的示出了一种电子设备的框图。例如,装置400可以是移动电话,计算机,数字广播终端,消息收发设备,游戏控制台,平板设备,医疗设备,健身设备,个人数字助理等。According to a third aspect of an embodiment of the present disclosure, please refer to FIG. 4 , which exemplarily shows a block diagram of an electronic device. For example, the device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.
参照图4,装置400可以包括以下一个或多个组件:处理组件402,存储器404,电源组件406,多媒体组件408,音频组件410,输入/输出(I/O)的 接口412,传感器组件414,以及通信组件416。Referring to Figure 4, the device 400 may include one or more of the following components: a processing component 402, a memory 404, a power supply component 406, a multimedia component 408, an audio component 410, an input/output (I/O) interface 412, a sensor component 414, and communications component 416.
处理组件402通常控制装置400的整体操作,诸如与显示,电话呼叫,数据通信,相机程序操作和记录操作相关联的操作。处理元件402可以包括一个或多个处理器420来执行指令,以完成上述的方法的全部或部分步骤。此外,处理组件402可以包括一个或多个模块,便于处理组件402和其他组件之间的交互。例如,处理部件402可以包括多媒体模块,以方便多媒体组件408和处理组件402之间的交互。 Processing component 402 generally controls the overall operations of device 400, such as operations associated with display, phone calls, data communications, camera program operations, and recording operations. The processing element 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 402 may include one or more modules that facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.
存储器404被配置为存储各种类型的数据以支持在设备400的操作。这些数据的示例包括用于在装置400上操作的任何应用程序或方法的指令,联系人数据,电话簿数据,消息,图片,视频等。存储器404可以由任何类型的易失性或非易失性存储设备或者它们的组合实现,如静态随机存取存储器(SRAM),电可擦除可编程只读存储器(EEPROM),可擦除可编程只读存储器(EPROM),可编程只读存储器(PROM),只读存储器(ROM),磁存储器,快闪存储器,磁盘或光盘。 Memory 404 is configured to store various types of data to support operations at device 400 . Examples of such data include instructions for any application or method operating on device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.
电力组件406为装置400的各种组件提供电力。电力组件406可以包括电源管理系统,一个或多个电源,及其他与为装置400生成、管理和分配电力相关联的组件。 Power component 406 provides power to various components of device 400 . Power components 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 400 .
多媒体组件408包括在所述装置400和用户之间的提供一个输出接口的屏幕。在一些实施例中,屏幕可以包括液晶显示器(LCD)和触控面板(TP)。如果屏幕包括触控面板,屏幕可以被实现为触控屏,以接收来自用户的输入信号。触控面板包括一个或多个触控传感器以感测触控、滑动和触控面板上的手势。所述触控传感器可以不仅感测触控或滑动动作的边界,而且还检测与所述触控或滑动操作相关的持续时间和压力。在一些实施例中,多媒体组件408包括一个前置摄像头和/或后置摄像头。当装置400处于操作模式,如拍摄模式或视频模式时,前置摄像头和/或后置摄像头可以接收外部的多媒体数据。每个前置摄像头和后置摄像头可以是一个固定的光学透镜系统或具有焦距和光学变焦能力。 Multimedia component 408 includes a screen that provides an output interface between the device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor may not only sense the boundary of the touch or sliding operation, but also detect the duration and pressure associated with the touch or sliding operation. In some embodiments, multimedia component 408 includes a front-facing camera and/or a rear-facing camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front-facing camera and rear-facing camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.
音频组件410被配置为输出和/或输入音频信号。例如,音频组件410包括一个麦克风(MIC),当装置400处于操作模式,如呼叫模式、记录模式和语音识别模式时,麦克风被配置为接收外部音频信号。所接收的音频信号可以被进一步存储在存储器404或经由通信组件416发送。在一些实施例中,音频组件410还包括一个扬声器,用于输出音频信号。 Audio component 410 is configured to output and/or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when device 400 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or sent via communication component 416 . In some embodiments, audio component 410 also includes a speaker for outputting audio signals.
I/O接口412为处理组件402和外围接口模块之间提供接口,上述外围接口模块可以是键盘,点击轮,按钮等。这些按钮可包括但不限于:主页按钮、音量按钮、启动按钮和锁定按钮。The I/O interface 412 provides an interface between the processing component 402 and a peripheral interface module, which may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.
传感器组件414包括一个或多个传感器,用于为装置400提供各个方面的状态评估。例如,传感器组件414可以检测到装置400的打开/关闭状态,组件的相对定位,例如所述组件为装置400的显示器和小键盘,传感器组件414还可以检测装置400或装置400一个组件的位置改变,用户与装置400接触的存在或不存在,装置400方位或加速/减速和装置400的温度变化。传感器组件414还可以包括接近传感器,被配置用来在没有任何的物理接触时检测附近物体的存在。传感器组件414还可以包括光传感器,如CMOS或CCD图像传感器,用于在成像应用中使用。在一些实施例中,该传感器组件414还可以包括加速度传感器,陀螺仪传感器,磁传感器,压力传感器或温度传感器。 Sensor component 414 includes one or more sensors for providing various aspects of status assessment for device 400 . For example, the sensor component 414 can detect the open/closed state of the device 400, the relative positioning of components, such as the display and keypad of the device 400, and the sensor component 414 can also detect a change in position of the device 400 or a component of the device 400. , the presence or absence of user contact with the device 400 , device 400 orientation or acceleration/deceleration and temperature changes of the device 400 . Sensor assembly 414 may also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 414 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
通信组件416被配置为便于装置400和其他设备之间有线或无线方式的通信。装置400可以接入基于通信标准的无线网络,如WiFi,2G或3G,4G或5G或它们的组合。在一个示例性实施例中,通信部件416经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中,所述通信部件416还包括近场通信(NFC)模块,以促进短程通信。例如,在NFC模块可基于射频识别(RFID)技术,红外数据协会(IrDA)技术,超宽带(UWB)技术,蓝牙(BT)技术和其他技术来实现。 Communication component 416 is configured to facilitate wired or wireless communication between apparatus 400 and other devices. The device 400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G or 5G or a combination thereof. In one exemplary embodiment, the communication component 416 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communications component 416 also includes a near field communications (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
在示例性实施例中,装置400可以被一个或多个应用专用集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理设备(DSPD)、可编程逻 辑器件(PLD)、现场可编程门阵列(FPGA)、控制器、微控制器、微处理器或其他电子元件实现,用于执行上述电子设备的供电方法。In an exemplary embodiment, apparatus 400 may be configured by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable Gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are implemented for executing the power supply method of the above electronic device.
第四方面,本公开在示例性实施例中,还提供了一种包括指令的非临时性计算机可读存储介质,例如包括指令的存储器404上述指令可由装置400的处理器420执行以完成上述电子设备的供电方法。例如,所述非临时性计算机可读存储介质可以是ROM、随机存取存储器(RAM)、CD-ROM、磁带、软盘和光数据存储设备等。In a fourth aspect, in an exemplary embodiment, the present disclosure also provides a non-transitory computer-readable storage medium including instructions, such as a memory 404 including instructions. The instructions can be executed by the processor 420 of the device 400 to complete the above electronic tasks. The method of powering the device. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.
本领域技术人员在考虑说明书及实践这里公开的公开后,将容易想到本公开的其它实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化,这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的,本公开的真正范围和精神由下面的权利要求指出。Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the disclosure that follow the general principles of the disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
应当理解的是,本公开并不局限于上面已经描述并在附图中示出的精确结构,并且可以在不脱离其范围进行各种修改和改变。本公开的范围仅由所附的权利要求来限制。It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the disclosure is limited only by the appended claims.

Claims (20)

  1. 一种图像处理方法,其特征在于,所述方法包括:An image processing method, characterized in that the method includes:
    获取图像采集设备针对相同场景采集的第一图像和第二图像,其中,所述第一图像为正常曝光图像,第二图像为欠曝光图像;Obtaining a first image and a second image collected by the image acquisition device for the same scene, wherein the first image is a normal exposure image and the second image is an underexposure image;
    根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域;determining an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;
    根据所述有效光斑区域对所述第一图像进行虚化渲染处理。The first image is blurred and rendered according to the effective light spot area.
  2. 根据权利要求1所述的图像处理方法,其特征在于,所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域,包括:The image processing method according to claim 1, wherein the effective light spot is determined in at least one first light spot area in the first image according to at least one second light spot area in the second image. areas, including:
    将所述至少一个第一光斑区域中属于第一交集的第一光斑区域,确定为有效光斑区域,其中,所述第一交集为所述至少一个第一光斑区域和所述至少一个第二光斑区域的交集。The first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
  3. 根据权利要求1所述的图像处理方法,其特征在于,在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域之后,还包括:The image processing method according to claim 1, characterized in that, according to at least one second light spot area in the second image, it is determined to be effective in at least one first light spot area in the first image. After the light spot area, it also includes:
    确定每个所述有效光斑区域的目标像素点,其中,所述目标像素点为所述有效光斑区域内颜色饱和度最高的像素点;Determine the target pixel point of each effective light spot area, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;
    在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数,其中,所述颜色参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
  4. 根据权利要求3所述的图像处理方法,其特征在于,所述在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数,包括:The image processing method according to claim 3, characterized in that, in each effective spot area, determining the color parameters of each pixel according to the color parameters of the target pixels includes:
    在第i个有效光斑区域内的每个像素点,与第i个有效光斑区域的所述目标像素点间的颜色参数差值均小于预设差值阈值的情况下,在第i个有效光斑区域内根据所述目标像素点的颜色参数对每个像素点的颜色参数进行调整,其中,i为大于0,且不大于N的整数,所述N为所述第一图像中的有效光斑区域的总数量;When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;
    在第j个有效光斑区域内的至少一个像素点,与第j个有效光斑区域的所述目标像素点间的颜色参数差值大于预设差值阈值的情况下,确定第j个有效光斑区域内的每个像素点的颜色参数不变,其中,j为大于0,且不大于N的整数。In the case where the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, the jth effective light spot area is determined The color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
  5. 根据权利要求1所述的图像处理方法,其特征在于,在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第二光斑区域中确定有效光斑区域之后,还包括:The image processing method according to claim 1, characterized in that, based on at least one second light spot area in the second image, determining the effective value in at least one second light spot area in the first image After the light spot area, it also includes:
    确定每个所述有效光斑区域的目标亮度参数,其中,所述目标亮度参数为所述有效光斑区域内亮度参数最小的像素点的亮度参数;Determine the target brightness parameter of each effective light spot area, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective light spot area;
    在每个所述有效光斑区域内,根据所述目标亮度参数对每个像素点的亮度参数进行调整,其中,所述亮度参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
  6. 根据权利要求1所述的图像处理方法,其特征在于,在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第二光斑区域中确定有效光斑区域之前,还包括:The image processing method according to claim 1, characterized in that, based on at least one second light spot area in the second image, determining the effective value in at least one second light spot area in the first image Before the light spot area, it also includes:
    分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image.
  7. 根据权利要求6所述的图像处理方法,其特征在于,分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第 一光斑区域和所述第二图像中的至少一个第二光斑区域,包括:The image processing method according to claim 6, characterized in that, performing light spot detection on the first image and the second image respectively, to obtain at least one first light spot area and the third light spot area in the first image. At least one second light spot area in the two images includes:
    在YUV域下,分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。In the YUV domain, perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. .
  8. 根据权利要求6所述的图像处理方法,其特征在于,所述分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域,包括:The image processing method according to claim 6, characterized in that: performing light spot detection on the first image and the second image respectively to obtain at least one first light spot area and the first light spot area in the first image. At least one second light spot area in the second image includes:
    将所述第一图像中亮度高于第一亮度阈值的像素点确定为第一光斑像素点,并将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域;Determine the pixels in the first image whose brightness is higher than the first brightness threshold as the first light spot pixels, and determine at least one connected domain composed of the first light spot pixels as the first light spot area;
    将所述第二图像中亮度高于第二亮度阈值的像素点确定为第二光斑像素点,并将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域。Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
  9. 根据权利要求8所述的图像处理方法,其特征在于,所述将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域,包括:The image processing method according to claim 8, wherein determining at least one connected domain composed of the first light spot pixels as the first light spot area includes:
    将所述第一光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通域确定为第一光斑区域;Determine at least one connected domain composed of the first light spot pixels and with a number of pixels within a preset number range as the first light spot area;
    所述将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域,包括:Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:
    将所述第二光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通区域确定为第二光斑区域。At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
  10. 一种图像处理装置,其特征在于,所述装置包括:An image processing device, characterized in that the device includes:
    获取模块,用于获取图像采集设备针对相同场景采集的第一图像和第二图像,其中,所述第一图像为正常曝光图像,第二图像为欠曝光图像;An acquisition module, configured to acquire the first image and the second image collected by the image acquisition device for the same scene, where the first image is a normal exposure image and the second image is an underexposure image;
    确定模块,用于根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第一光斑区域中确定有效光斑区域;a determining module configured to determine an effective light spot area in at least one first light spot area in the first image according to at least one second light spot area in the second image;
    渲染模块,用于根据所述有效光斑区域对所述第一图像进行虚化渲染处理。A rendering module, configured to perform blur rendering processing on the first image according to the effective light spot area.
  11. 根据权利要求10所述的图像处理装置,其特征在于,所述确定模块具体用于:The image processing device according to claim 10, characterized in that the determining module is specifically used to:
    将所述至少一个第一光斑区域中属于第一交集的第一光斑区域,确定为有效光斑区域,其中,所述第一交集为所述至少一个第一光斑区域和所述至少一个第二光斑区域的交集。The first light spot area belonging to the first intersection among the at least one first light spot area is determined as the effective light spot area, wherein the first intersection is the at least one first light spot area and the at least one second light spot. intersection of regions.
  12. 根据权利要求10所述的图像处理装置,其特征在于,还包括颜色模块,用于:The image processing device according to claim 10, further comprising a color module for:
    在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第二光斑区域中确定有效光斑区域之后,确定每个所述有效光斑区域的目标像素点,其中,所述目标像素点为所述有效光斑区域内颜色饱和度最高的像素点;After determining the effective light spot area in the at least one second light spot area in the first image based on the at least one second light spot area in the second image, determining the target pixel of each effective light spot area. point, wherein the target pixel point is the pixel point with the highest color saturation in the effective light spot area;
    在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数,其中,所述颜色参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the color parameter of each pixel is determined according to the color parameter of the target pixel, wherein the color parameter is used to perform a blur rendering process on the first image.
  13. 根据权利要求12所述的图像处理装置,其特征在于,所述颜色模块用于在每个所述有效光斑区域内,根据所述目标像素点的颜色参数确定每个像素点的颜色参数时,具体用于:The image processing device according to claim 12, wherein the color module is used to determine the color parameter of each pixel according to the color parameter of the target pixel in each of the effective light spot areas, Specifically used for:
    在第i个有效光斑区域内的每个像素点,与第i个有效光斑区域的所述目标像素点间的颜色参数差值均小于预设差值阈值的情况下,在第i个有效光斑区域内根据所述目标像素点的颜色参数对每个像素点的颜色参数 进行调整,其中,i为大于0,且不大于N的整数,所述N为所述第一图像中的有效光斑区域的总数量;When the color parameter difference between each pixel in the i-th effective light spot area and the target pixel point in the i-th effective light spot area is less than the preset difference threshold, the i-th effective light spot The color parameters of each pixel in the area are adjusted according to the color parameters of the target pixel, where i is an integer greater than 0 and not greater than N, and N is the effective spot area in the first image. total quantity;
    在第j个有效光斑区域内的至少一个像素点,与第j个有效光斑区域的所述目标像素点间的颜色参数差值大于预设差值阈值的情况下,确定第j个有效光斑区域内的每个像素点的颜色参数不变,其中,j为大于0,且不大于N的整数。In the case where the color parameter difference between at least one pixel in the jth effective light spot area and the target pixel point in the jth effective light spot area is greater than the preset difference threshold, the jth effective light spot area is determined The color parameters of each pixel within are unchanged, where j is an integer greater than 0 and not greater than N.
  14. 根据权利要求10所述的图像处理装置,其特征在于,还包括亮度模块,用于:The image processing device according to claim 10, further comprising a brightness module for:
    在所述根据所述第二图像中的至少一个第二光斑区域,在所述第一图像中的至少一个第二光斑区域中确定有效光斑区域之后,确定每个所述有效光斑区域的目标亮度参数,其中,所述目标亮度参数为所述有效光斑区域内亮度参数最小的像素点的亮度参数;After determining the effective light spot area in the at least one second light spot area in the first image based on the at least one second light spot area in the second image, determining the target brightness of each of the effective light spot areas. Parameter, wherein the target brightness parameter is the brightness parameter of the pixel with the smallest brightness parameter in the effective spot area;
    在每个所述有效光斑区域内,根据所述目标亮度参数对每个像素点的亮度参数进行调整,其中,所述亮度参数用于对所述第一图像进行虚化渲染处理。In each effective light spot area, the brightness parameter of each pixel is adjusted according to the target brightness parameter, wherein the brightness parameter is used to perform blur rendering processing on the first image.
  15. 根据权利要求10所述的图像处理装置,其特征在于,还包括检测模块,用于:The image processing device according to claim 10, further comprising a detection module for:
    分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。Perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image.
  16. 根据权利要求15所述的图像处理装置,其特征在于,所述检测模块具体用于:The image processing device according to claim 15, characterized in that the detection module is specifically used for:
    在YUV域下,分别对所述第一图像和所述第二图像进行光斑检测,得到所述第一图像中的至少一个第一光斑区域和所述第二图像中的至少一个第二光斑区域。In the YUV domain, perform light spot detection on the first image and the second image respectively to obtain at least one first light spot area in the first image and at least one second light spot area in the second image. .
  17. 根据权利要求15所述的图像处理装置,其特征在于,所述检测模块具体用于:The image processing device according to claim 15, characterized in that the detection module is specifically used for:
    将所述第一图像中亮度高于第一亮度阈值的像素点确定为第一光斑像素点,并将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域;Determine the pixels in the first image whose brightness is higher than the first brightness threshold as the first light spot pixels, and determine at least one connected domain composed of the first light spot pixels as the first light spot area;
    将所述第二图像中亮度高于第二亮度阈值的像素点确定为第二光斑像素点,并将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域。Pixels in the second image with brightness higher than the second brightness threshold are determined as second light spot pixels, and at least one connected area composed of the second light spot pixels is determined as a second light spot area.
  18. 根据权利要求17所述的图像处理装置,其特征在于,所述检测模块用于将所述第一光斑像素点组成的至少一个连通域确定为第一光斑区域时,具体用于:The image processing device according to claim 17, wherein the detection module is used to determine at least one connected domain composed of the first light spot pixel points as the first light spot area, specifically for:
    将所述第一光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通域确定为第一光斑区域;Determine at least one connected domain composed of the first light spot pixels and with a number of pixels within a preset number range as the first light spot area;
    所述将所述第二光斑像素点组成的至少一个连通区域确定为第二光斑区域,包括:Determining at least one connected area composed of the second light spot pixel points as the second light spot area includes:
    将所述第二光斑像素点组成的,像素点数量在预设数量范围内的至少一个连通区域确定为第二光斑区域。At least one connected area composed of the second light spot pixels and with a number of pixels within a preset number range is determined as a second light spot area.
  19. 一种电子设备,其特征在于,所述电子设备包括存储器、处理器,所述存储器用于存储可在处理器上运行的计算机指令,所述处理器用于在执行所述计算机指令时基于权利要求1至9中任一项所述的图像处理方法。An electronic device, characterized in that the electronic device includes a memory and a processor, the memory is used to store computer instructions that can be run on the processor, and the processor is used to execute the computer instructions based on the claims. The image processing method according to any one of 1 to 9.
  20. 一种计算机可读存储介质,其上存储有计算机程序,其特征在于,所述程序被处理器执行时实现权利要求1至9中任一项所述的方法。A computer-readable storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the method of any one of claims 1 to 9 is implemented.
PCT/CN2022/098240 2022-06-10 2022-06-10 Image processing method and apparatus, electronic device, and storage medium WO2023236209A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2022/098240 WO2023236209A1 (en) 2022-06-10 2022-06-10 Image processing method and apparatus, electronic device, and storage medium
CN202280004273.6A CN117616777A (en) 2022-06-10 2022-06-10 Image processing method, device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2022/098240 WO2023236209A1 (en) 2022-06-10 2022-06-10 Image processing method and apparatus, electronic device, and storage medium

Publications (1)

Publication Number Publication Date
WO2023236209A1 true WO2023236209A1 (en) 2023-12-14

Family

ID=89117464

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/098240 WO2023236209A1 (en) 2022-06-10 2022-06-10 Image processing method and apparatus, electronic device, and storage medium

Country Status (2)

Country Link
CN (1) CN117616777A (en)
WO (1) WO2023236209A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103916610A (en) * 2013-01-07 2014-07-09 通用汽车环球科技运作有限责任公司 Glaring reduction for dynamic rearview mirror
CN105635593A (en) * 2014-10-13 2016-06-01 广达电脑股份有限公司 Multiple exposure imaging system and white balance method thereof
CN107197146A (en) * 2017-05-31 2017-09-22 广东欧珀移动通信有限公司 Image processing method and related product
US20220005169A1 (en) * 2018-11-29 2022-01-06 Samsung Electronics Co., Ltd. Image processing method and electronic device supporting same
CN114565517A (en) * 2021-12-29 2022-05-31 骨圣元化机器人(深圳)有限公司 Image denoising method and device for infrared camera and computer equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103916610A (en) * 2013-01-07 2014-07-09 通用汽车环球科技运作有限责任公司 Glaring reduction for dynamic rearview mirror
CN105635593A (en) * 2014-10-13 2016-06-01 广达电脑股份有限公司 Multiple exposure imaging system and white balance method thereof
CN107197146A (en) * 2017-05-31 2017-09-22 广东欧珀移动通信有限公司 Image processing method and related product
US20220005169A1 (en) * 2018-11-29 2022-01-06 Samsung Electronics Co., Ltd. Image processing method and electronic device supporting same
CN114565517A (en) * 2021-12-29 2022-05-31 骨圣元化机器人(深圳)有限公司 Image denoising method and device for infrared camera and computer equipment

Also Published As

Publication number Publication date
CN117616777A (en) 2024-02-27

Similar Documents

Publication Publication Date Title
WO2019183813A1 (en) Image capture method and device
CN109379572B (en) Image conversion method, image conversion device, electronic equipment and storage medium
KR101800101B1 (en) Method, apparatus, program, and recording medium for adjustmenting shooting parameter
CN109345485B (en) Image enhancement method and device, electronic equipment and storage medium
WO2020034737A1 (en) Imaging control method, apparatus, electronic device, and computer-readable storage medium
CN111586282B (en) Shooting method, shooting device, terminal and readable storage medium
CN108154465B (en) Image processing method and device
KR101930460B1 (en) Photographing apparatusand method for controlling thereof
US9025050B2 (en) Digital photographing apparatus and control method thereof
JP2012199675A (en) Image processing apparatus, image processing method, and program
CN108040204B (en) Image shooting method and device based on multiple cameras and storage medium
WO2021136035A1 (en) Photographing method and apparatus, storage medium, and electronic device
WO2023071933A1 (en) Camera photographing parameter adjustment method and apparatus and electronic device
WO2023220957A1 (en) Image processing method and apparatus, and mobile terminal and storage medium
CN111586280B (en) Shooting method, shooting device, terminal and readable storage medium
CN108156381B (en) Photographing method and device
WO2023236209A1 (en) Image processing method and apparatus, electronic device, and storage medium
WO2016074414A1 (en) Image processing method and apparatus
KR102512787B1 (en) Method and apparatus for displaying captured preview image, and medium
CN112016595A (en) Image classification method and device, electronic equipment and readable storage medium
WO2023236215A1 (en) Image processing method and apparatus, and storage medium
WO2023231009A1 (en) Focusing method and apparatus, and storage medium
KR102686103B1 (en) Method and apparatus for processing image, mobile terminal and storage medium
US20240005521A1 (en) Photographing method and apparatus, medium and chip
WO2024159546A1 (en) Method for taking pictures with a device equipped with cameras directed in opposite directions, and associated device

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 202280004273.6

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22945350

Country of ref document: EP

Kind code of ref document: A1