WO2022067761A1 - Procédé et appareil de traitement d'image, dispositif de capture, plateforme mobile, et support de stockage lisible par ordinateur - Google Patents

Procédé et appareil de traitement d'image, dispositif de capture, plateforme mobile, et support de stockage lisible par ordinateur Download PDF

Info

Publication number
WO2022067761A1
WO2022067761A1 PCT/CN2020/119656 CN2020119656W WO2022067761A1 WO 2022067761 A1 WO2022067761 A1 WO 2022067761A1 CN 2020119656 W CN2020119656 W CN 2020119656W WO 2022067761 A1 WO2022067761 A1 WO 2022067761A1
Authority
WO
WIPO (PCT)
Prior art keywords
white balance
parameter
target
color temperature
image
Prior art date
Application number
PCT/CN2020/119656
Other languages
English (en)
Chinese (zh)
Inventor
吴伟霖
胡涛
李琛
滕文猛
Original Assignee
深圳市大疆创新科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市大疆创新科技有限公司 filed Critical 深圳市大疆创新科技有限公司
Priority to PCT/CN2020/119656 priority Critical patent/WO2022067761A1/fr
Publication of WO2022067761A1 publication Critical patent/WO2022067761A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/88—Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control

Definitions

  • the present application relates to the technical field of image processing, and in particular, to an image processing method, an apparatus, a photographing device, a movable platform, and a computer-readable storage medium.
  • Color temperature refers to the color radiated by a black body (absolute black body) as the temperature increases after being heated. As the temperature increases, the black body first emits red light, and as the temperature continues to increase, it becomes brighter and brighter to yellow, white, and blue light.
  • the color temperature of the light source it is considered that the color of the light emitted by the light source is the same as the color of the light radiated by the black body at a certain temperature, and the black body temperature at this time is called the color temperature of the light source, thus introducing the concept of correlated color temperature.
  • Correlated color temperature mainly refers to the temperature of the most similar black body radiator with the same luminance stimulus.
  • color constancy mainly refers to the perceptual characteristics that people's perception of the color of the surface of the object remains unchanged when the color light irradiated on the surface of the object changes, that is, the visual perception of the color change of the object. invariance. Human beings have a psychological tendency to not change the color judgment of a specific object due to light source or external environmental factors. This tendency is color constancy.
  • AVB Auto White Balance
  • the present application provides an image processing method, an apparatus, a photographing device, a movable platform and a computer-readable storage medium to solve the problem of poor white balance effect in the related art.
  • an image processing method including:
  • the target parameter is related to the brightness of the shooting scene of the image
  • a reference parameter associated with the target parameter from a preset reference database, and obtain a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter; wherein the reference parameter and the shooting scene of the image Brightness is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter;
  • White balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.
  • an image processing apparatus includes a processor and a memory, the memory stores instructions, and the processor implements the following operations when executing the instructions:
  • the target parameter is related to the brightness of the shooting scene of the image
  • a reference parameter associated with the target parameter from a preset reference database, and obtain a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter; wherein the reference parameter and the shooting scene of the image Brightness is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter;
  • White balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.
  • a shooting device including:
  • a lens assembly arranged inside the casing
  • a sensor assembly disposed inside the housing for sensing light passing through the lens assembly and generating electrical signals
  • the image processing apparatus according to the second aspect.
  • a movable platform including:
  • a power system mounted within the body for powering the movable platform
  • the image processing apparatus according to the second aspect.
  • a computer-readable storage medium where several computer instructions are stored thereon, and when the computer instructions are executed, the operations of the method of the first aspect are implemented.
  • the reference database in this solution contains multiple reference parameters, as well as the reference color temperature line and the white balance statistical area corresponding to each reference parameter; different reference parameters are set to simulate the possible occurrences in the actual shooting scene. Therefore, the reference data suitable for the actual environment with different brightness and darkness can be configured; thus, the reference data that matches the brightness of the shooting scene of the image can be selected for white balance during image processing; especially In a low-brightness environment, since the corresponding reference data can be selected, an image captured in a low-brightness environment can also obtain a good white balance effect.
  • FIG. 1 is a schematic diagram of images captured by an imaging system under different lighting conditions according to an embodiment of the present application.
  • FIG. 2A is a flowchart of an image processing method according to an embodiment of the present application.
  • FIG. 2B is a schematic diagram of a white balance reference data.
  • FIG. 2C , FIG. 2D and FIG. 2E are a plurality of reference parameters according to an embodiment of the present application, and schematic diagrams of reference color temperature lines and white balance statistical regions corresponding to each reference parameter.
  • FIG. 2F is a schematic diagram illustrating an offset between a reference color temperature line relative to a preset standard color temperature line based on FIG. 2E in an embodiment of the present application.
  • FIG. 2G is a schematic diagram of a fusion ratio according to an embodiment of the present application.
  • FIG. 2H is a schematic flowchart of another image processing method shown in this embodiment.
  • FIG. 2I is a schematic flowchart of another image processing method shown in this embodiment.
  • FIG. 3 is a schematic structural diagram of a device for implementing the image processing method of this embodiment.
  • FIG. 4 is a block diagram of a mobile platform according to an embodiment of the present application.
  • FIG. 5 is a block diagram of a camera according to an embodiment of the present application.
  • color constancy mainly refers to the perception that the color of the object surface remains unchanged when the color light irradiated on the surface of the object changes. Characteristics, that is, the invariance of visual perception to the color change of objects. When a specific object has a very different reflection spectrum due to the environment (especially the lighting environment is within a certain range of change), the reflection spectrum of the object will be very different, but the visual recognition system of the human eye can recognize this change, and can judge It is found that the change is caused by the change of the lighting environment.
  • the human recognition mechanism will consider the surface color of the object to be constant within this range of change.
  • Another example is a piece of white A4 paper, over the course of the day you would think that the color of the sunlight changes over time while the A4 paper is still white.
  • the image acquisition device will appear color reproduction distortion, that is, the color of the image is either reddish or bluish.
  • the color of the image output by the camera is reddish; in a light environment with a high color temperature, the color of the image output by the camera is blue.
  • Figure 1 it is a schematic diagram of the image captured by the imaging system under different lighting conditions, where the inconstancy of color is reflected.
  • AVB Auto White Balance
  • the white balance in this state is equivalent to the CCT process in the perception environment, and the effect of the color matrix under the CCT is equivalent to the restoration of various surface colors in the environment and the color constancy of human perception.
  • the resulting image obtained by the imaging system is close to the perceptual constancy of the human eye.
  • the current white balance algorithms mainly include: maximum brightness method (Bright Surface First), gray world method (Gray World), improved gray world method, color gamut limit method, light source prediction method, etc.
  • the sensor data of the scene is used to calculate the gain values of the R, G, and B channels of AWB (Automatic white balance) and the CCT (Correlated color temperature) value of the current scene.
  • AWB Automatic white balance
  • CCT Correlated color temperature
  • the sensor needs to use a larger gain to perceive the image. In this state, there will be more bright and dark dead pixels, and the output signal-to-noise ratio of the sensor will also become worse. The sensor may still be unable to use a larger gain.
  • the signal of the image exhibits normal luminance values. The image is underexposed, and the bright and dark dead pixels and low signal-to-noise ratio of the image cause the white point in the scene to drift on the image, so that the ordinary white balance algorithm cannot identify the white point, and thus cannot.
  • the calculation result of the white balance is obtained, which results in that the image is prone to appear greenish, purpleish, etc., resulting in a poor experience for the user.
  • the present application provides an image processing solution, in which the reference database contains multiple reference parameters, as well as the reference color temperature line and the white balance statistical area corresponding to each reference parameter; different reference parameters are set for the purpose of simulating Various environments with different brightness and darkness levels that may appear in the actual shooting scene, so that the reference data suitable for the actual environment with different brightness and darkness levels can be configured; thus, the reference data that matches the brightness of the shooting scene of the image can be selected during image processing. Perform white balance; especially in a low-brightness environment, since the corresponding reference data can be selected, the images captured in a low-brightness environment can also obtain a good white balance effect.
  • the color temperature line can represent the corresponding Planck line of the device under different brightness environments.
  • the solution of this embodiment can be applied to photographing devices such as cameras or video cameras; it can also be applied to electronic devices equipped with cameras, and the electronic devices here may include devices such as movable platforms or smart phones.
  • the camera has a built-in ISP (Image Signal Processing) unit, which is mainly used to process the output signal of the front-end image sensor.
  • the ISP completes the effect processing of the digital image through a series of digital image processing algorithms. Including 3A (auto exposure, auto focus, auto white balance), dead pixel correction, denoising, glare suppression, backlight compensation, color enhancement, lens shading correction, etc.
  • 3A auto exposure, auto focus, auto white balance
  • dead pixel correction denoising, glare suppression, backlight compensation, color enhancement, lens shading correction, etc.
  • the solution of this embodiment can be applied to an ISP unit in a camera to realize automatic white balance processing of an image.
  • the image for white balance processing in this embodiment may be the original image raw collected by the built-in image sensor of the photographing device, or may be the image generated by the ISP unit in the process of processing the image, such as a YUV or RGB image.
  • the solution of this embodiment can also be applied to image processing software, which can run on tablet computers, smart phones, personal digital assistants (PDAs), laptop computers, desktop computers, or media content players, etc.
  • image processing software may apply the image processing method provided in this embodiment to perform white balance processing on a specified image.
  • FIG. 2A is a flowchart of an image processing method provided by an embodiment of the present application. The method includes the following operations:
  • step 202 an image is acquired, and a target parameter corresponding to the image is determined, and the target parameter is related to the brightness of the shooting scene of the image;
  • a reference parameter associated with the target parameter is selected from a preset reference database, and a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter are obtained; wherein the reference parameter is the same as the selected reference parameter.
  • the brightness of the shooting scene of the image is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter;
  • step 206 white balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.
  • the white point in the image means that the color component of the R channel, the color component of the G channel, and the color component of the B channel of the pixel are equal.
  • this embodiment divides the image into blocks, and each block contains multiple pixels. For the pixels of each block, the brightness values of the R channel of each block are accumulated and averaged, the brightness values of the G channel are accumulated and averaged, and the brightness values of the B channel are accumulated and averaged; then, the G channel is accumulated and averaged.
  • Rgain Ravg/Gavg
  • Bgain Bavg/Gavg.
  • the image is divided into blocks, and the Rgain and Bgain of the image block are known.
  • the white balance reference data As shown in FIG. 2B, a schematic diagram of a white balance reference data is shown, the white balance reference data characterizes the color temperature line and the white balance statistical area, and the Rgain and Bgain of each block are used to match the white balance in the white balance reference data.
  • the balance statistics area is compared, if the Rgain and Bgain of the block fall within the white balance statistics area, that is, the block is a white point, and if it is not within the white balance statistics area, the block is not a white point.
  • an Rgain value is obtained according to the Rgain weighted average of each image block belonging to the white point.
  • the Bgain weighted average of each image block belonging to the white point Get a Bgain value; blocks that do not fall within the white balance statistical area are not white points and do not need to be added to the calculation.
  • the white balance reference data is described by taking Rgain and Bgain as an example for the gain of the pixel.
  • Rgain and Bgain can be used to determine the white balance statistical area.
  • the two gains of Rgain and Ggain can be used to determine the white balance statistical area.
  • use the two gains of Bgain and Ggain to determine the white balance statistical area, or use four-channel gain values, such as R, B, G R gain, and G B gain, to determine the white balance statistical area, or It may also be white balance based on other color spaces, which is not limited in this embodiment.
  • the setting of white balance reference data is one of the key factors affecting the effect of white balance processing.
  • fixed white balance reference data is usually set, and the setting method is usually an empirical value.
  • the white balance reference data is usually set in an ideal state (such as ideal brightness) and is suitable for normal scenes. In a scene with low brightness, the white balance of the low-brightness image is performed by the white balance reference data suitable for normal brightness, which obviously cannot obtain a better processing effect.
  • the reference database contains multiple reference parameters, as well as the reference color temperature line and white balance statistical area corresponding to each reference parameter; different reference parameters are set to simulate various environments with different degrees of brightness and darkness that may appear in the actual shooting scene. , so that reference data suitable for actual environments with different brightness and darkness levels can be configured. These reference data may be generated by collecting images of multiple different standard light sources under different reference parameters in advance, and using the collected images. Next, an embodiment will be used to describe the process of how to obtain the data in the reference database.
  • M kinds of standard light sources and N kinds of different reference parameters can be used to collect images with the device; wherein, the standard light sources may refer to the light sources specified by the International Commission on Illumination for lighting in uniform color measurement.
  • the standard light sources the Determine the corresponding color temperature.
  • a standard light source can be placed on a standard gray card, and the device collects images containing the standard gray card, so that images with different color temperatures and different reference parameters can be collected.
  • process the collected images extract the light source white points of these images, and map them to the Rgain/Bgain space (it can also be Rgain and Ggain, Bgain and Ggain, or four-channel gain value, etc. ), after removing a small number of very special light sources, the distribution of the light sources is used to form an area where the white point is located, that is, the white balance statistics area.
  • the reference parameter may be a parameter related to the brightness of the shooting scene, which represents the brightness of the shooting scene of the image.
  • it may include any one or more of the following: scene brightness parameter, image signal-to-noise ratio, image The exposure parameter of the sensor or the dead pixel parameter of the image sensor.
  • scene brightness parameter LV light value
  • the data in the reference database can be generated from images collected under a variety of different standard light sources and under a variety of different brightnesses. In practical applications, other reference parameters can be used to generate data in the reference database.
  • the generated reference database contains multiple reference parameters, as well as the reference color temperature line and white balance statistical area corresponding to each reference parameter. Different reference parameters correspond to the reference color temperature line and white balance statistical area suitable for the brightness of different shooting scenes. .
  • the scene luminance parameter LV is taken as an example for the reference parameters, and 10 reference parameters are used as an example to show the corresponding reference color temperature line and white balance under the 10 reference parameters.
  • Statistics area wherein the white balance statistics area refers to the area surrounded by irregular graphics, and the curve in this area is the color temperature line; it can be seen that this embodiment implements a three-dimensional white balance calculation method.
  • the number of reference parameters may be flexibly configured as required during specific implementation, which is not limited in this embodiment. As an example, if the hardware computing power of the device is good, multiple reference parameters can be set to match various different brightness environments faced during actual shooting.
  • Fig. 2D shows 4 reference data by taking a two-dimensional space as an example, wherein the reference data PreLV located on the top layer, PreLV refers to the reference data in an ideal state, and the specific reference parameters are not shown in Fig. 2D; in some examples , PreLV can also be included in the reference database as a kind of reference data.
  • Figure 2E shows 4 color temperature lines in the Rgain/Bgain space, which are the color temperature lines corresponding to LV of -2, 3 and 0 respectively, and also includes the color temperature line PreLV under ideal brightness, and the corresponding color temperature lines are not shown in Figure 2E white balance statistics area.
  • white balance can be performed using reference data that is consistent with the brightness of the shooting scene of the image to be processed; specifically, for the image to be processed, the target parameter corresponding to the image is determined;
  • the target parameter selects the reference parameter associated with the target parameter from the reference database, selects the associated reference parameter, and correspondingly selects the reference color temperature line and the white balance statistical area, according to the target parameter, the selected reference parameter and its corresponding
  • the reference color temperature line and the white balance statistical area of are used to perform white balance processing on the image.
  • the target parameters of the image may be the same as or different from the reference parameters in the reference database.
  • the value of the selected reference parameter is the same as the value of the target parameter; in other examples, there are two selected reference parameters; among the plurality of reference parameters, the two reference parameters The value of the parameter has the smallest difference from the value of the target parameter.
  • the reference parameters in the reference database include 10 integers between the interval [-2, 7], assuming that the target parameter LV of the image is 1.0, and the value of the target parameter 1.0 is the same as the reference value in the reference database. If the parameter value 1 is the same, the reference parameter 1.0 is selected, so as to obtain the reference color temperature line and the white balance statistical area corresponding to the reference parameter 1.0, and use the reference color temperature line and the white balance statistical area to white balance the image.
  • the method of selecting the reference parameter can be flexibly configured as needed, for example, one reference parameter with the smallest difference can be selected; this embodiment can also be selected
  • the two with the smallest difference that is, among the multiple reference parameters, the values of the selected two reference parameters have the smallest difference with the value of the target parameter, for example, the reference parameters 1 and 1 with the smallest value difference from the target parameter 1.2 2. Since two reference parameters are selected, the reference color temperature line and the white balance statistics area corresponding to the two reference references are obtained respectively. When the image is white balanced, the reference color temperature line and white balance statistics corresponding to the two reference references are obtained respectively. area is processed.
  • white balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.
  • performing white balance processing on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical region includes:
  • the first target white balance result is determined according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.
  • white balance statistical area corresponding to the selected reference parameter based on the white balance statistical area, white pixels can be found from the image, and then the white balance gain can be calculated by using the found white pixels, and the calculation can be used as needed.
  • the obtained white balance gain, as well as the target parameters, the selected reference parameters and their corresponding reference color temperature lines, further perform white balance.
  • two pieces of reference data are obtained for the aforementioned case of selecting two reference parameters.
  • this embodiment uses the two pieces of reference data to perform white balance, and in the processing process, the two pieces of reference data are fused by fusing the two pieces of reference data. the way the data is processed.
  • the reference color temperature line includes a first reference color temperature line and a second reference color temperature line
  • the two reference parameters include a first reference parameter and a second reference parameter. Parameters and their corresponding reference color temperature lines and white balance statistical areas to determine the first target white balance result, including:
  • the first reference white balance result may be determined from the white point found in the image based on the white balance statistical area corresponding to the first reference color temperature line; the second reference white balance result may be based on the second reference color temperature
  • the white balance statistics area corresponding to the line is determined from the white point found in the image.
  • the first reference white balance result and the second reference white balance result are fused to determine the first target white balance result.
  • the values of the two selected reference parameters are 1 and 2, which are referred to as the first reference parameter and the second reference parameter in this embodiment;
  • the first reference white balance result is determined with reference to the color temperature line and the white balance statistics area;
  • the second reference white balance result is determined by using the reference color temperature line and the white balance statistics area corresponding to the second reference parameter.
  • the first target white balance result of the image is obtained by fusing the first reference white balance result and the second reference white balance result.
  • the fusion ratio of the first reference white balance result and the second reference white balance result is determined based on the respective weights of the first reference parameter and the second reference parameter relative to the target parameter.
  • the weight is determined based on differences between the first reference parameter and the second reference parameter and the target parameter, respectively.
  • AWBGain_tmp AWBgain0*lvratio+AWBgain1*(1-lvratio);
  • AWBGain_tmp represents the first target white balance result
  • AWBgain0 represents the first reference white balance result
  • AWBgain1 represents the second reference white balance result
  • lvratio represents the weight between the first reference parameter and the target parameter
  • (1-lvratio) represents the second The weights of the reference parameters and the target parameters actually represent the fusion ratio of AWBgain0 and AWBgain1.
  • lvratio can be calculated by normalizing the difference between the target parameter and the first reference parameter and the second reference parameter, or it can be calculated by using a weighting method; in practical applications, other parameters can also be configured as required.
  • the implementation manner is not limited in this embodiment.
  • the values of the two selected reference parameters are 1 and 2
  • the difference between the target parameter 1.2 and the first reference parameter 1 is 0.2
  • the target parameter 1.2 and the second reference parameter The difference of 1 is 0.8 of the difference of 2.
  • the weight of the white balance result can be set to 0.2.
  • the first target white balance result is obtained, and in some examples, the first target white balance result may be output as the final result.
  • the reference data can be used to find the exact white point, that is, the light source part in the image, and the light source belongs to the brighter area; but there are many other non-light source parts in the image, which are relatively dark, because White point drift may occur. If the first target white balance result is directly used, it may not be possible to perform better color restoration on the non-light source part in the image. Based on this, this embodiment may further process the first target white balance result to obtain a second target white balance result with a better processing effect.
  • a second target white balance result is determined according to the offset of the reference color temperature line relative to the preset standard color temperature line; wherein the preset standard color temperature line is at The color temperature line obtained under the ideal brightness of the shooting scene, such as the aforementioned ideal brightness PreLV; in this embodiment, the offset between the reference color temperature line and the preset standard color temperature line is used to determine the second target white balance result.
  • the second target white balance result may be determined according to the operational relationship between the first target white balance result and the offset.
  • FIG. 2F is used as an example for illustration.
  • FIG. 2F shows the offset of the reference color temperature line relative to the preset standard color temperature line. If the selected reference parameter LV is -2, that is, FIG. 2F In the reference color temperature line corresponding to lv-2, it can be seen from FIG. 2F that there is an offset between the reference color temperature line corresponding to lv-2 and the preset standard color temperature line.
  • the offset includes a color temperature mapping coefficient.
  • the offset between the reference color temperature line and the preset standard color temperature line may be realized by using a color temperature mapping coefficient.
  • the first target white balance result obtained by processing represents the grayscale gain of the image pixel, such as Rgain and Bgain;
  • the color temperature can be determined by Rgain and Bgain, for example, the ratio of Rgain and Bgain falls on the reference color temperature line , the point on the reference color temperature line represents the color temperature; if it does not fall on the reference color temperature line, draw a line with the ratio of Rgain and Bgain as the starting point and perpendicular to the reference color temperature line, the intersection of the line and the reference color temperature line is the color temperature.
  • the points on each color temperature line are the color temperature points.
  • FIG. 2F shows the mapping between the color temperature points on the reference color temperature line corresponding to lv-2 and the corresponding color temperature points on the preset standard color temperature line. Mapping, the color temperature mapping coefficient can be determined according to the mapping relationship between the two.
  • the first reference white balance result includes a first reference correlated color temperature
  • the second reference white balance result includes a second reference correlated color temperature
  • the second target white balance result includes a target correlated color temperature, according to the reference
  • the offset of the color temperature line relative to the preset standard color temperature line determines the second target white balance result, including:
  • the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient are fused to determine the target color temperature mapping coefficient
  • the second target white balance result is determined according to the target color temperature mapping coefficient.
  • Remap represents the target color temperature mapping coefficient obtained by fusion
  • Remap0 represents the first reference color temperature mapping coefficient
  • Remap1 represents the second reference color temperature mapping coefficient
  • the second target white balance result is determined based on the target color temperature mapping coefficient, for example, the second target white balance result is determined according to the product of the first target white balance result and the target color temperature mapping coefficient.
  • the second target white balance result is obtained based on the above processing.
  • the second target white balance result may be output as the final result; in other examples, further processing may be performed according to actual shooting requirements.
  • the brightness may change greatly during the video shooting. For example, the user shoots a brighter scene at night, and then shoots a darker scene; The difference between the scenes is large. If only the current scene is considered to white balance the video image, it may cause the color jump of the front and back video pictures to be large and have a sudden feeling.
  • the image processing method may further include:
  • the second target white balance result is fused with the pixel correction result of the historical image to determine a third target white balance result.
  • the pixel correction result of the historical image includes: the pixel correction result of the previous frame of the image, or the time domain pixel correction result of the historical image.
  • the shooting scene may be a night scene, usually based on the night scene, a night scene light source parameter is preset, and the night scene light source parameter is used to correct the image captured in the night scene, in order to make the image match the night scene and prevent the image The color is more obtrusive.
  • the preset night scene light source parameters can also be superimposed when the image is white balanced.
  • the image processing method further includes: in response to determining the For the second target white balance result, the second target white balance result is fused with the preset night scene light source parameters to determine the fourth target white balance result.
  • whether the current scene is a night scene is determined by the scene brightness parameter, wherein the threshold can be set as required.
  • the scene brightness of the image is lower than the set threshold, it can be determined that the shooting scene is a night scene, which needs to be superimposed
  • the light source parameters of the night scene are fused to avoid the abrupt feeling of the image color.
  • the fusion ratio of the second target white balance result and the pixel correction result of the historical image is preset. In some examples, the fusion ratio of the second target white balance result and the preset night scene light source parameters is preset. In some examples, the fusion ratio corresponds to the target parameter, and in practical applications, the fusion ratio can be preset as required, for example, the fusion ratio is set based on different target parameters.
  • AWBGain_t is the pixel correction result of the historical image or the night light source parameter
  • AWBGain-Final is the second target white balance result
  • (1-Blend_Ratio) is the fusion ratio of AWBGain_t
  • Blend_ratio is the fusion ratio of AWBGain-Final.
  • FIG. 2G a schematic diagram of a blending ratio (Blend_ratio) is shown. In this embodiment, corresponding blending ratios are preset for different reference parameters.
  • FIG. 2H it is a schematic flowchart of another image processing method shown in this embodiment, including the following operations:
  • Step 211 the image sensor outputs RAW data
  • it can also be an image generated by the ISP unit in the process of processing the image.
  • Step 212 calculating the target parameters of the image through the RAW data
  • the target parameter includes any one or more of the following: scene brightness parameter, image signal-to-noise ratio, image sensor exposure parameter or image sensor dead pixel parameter; this embodiment takes the scene brightness parameter LV value as an example for description.
  • the scene brightness parameter of the image may be determined by the exposure parameter of the image sensor and the average brightness of the image.
  • luma is the weighted average brightness of the raw image
  • dgain and shutter are the exposure parameters of the AE module
  • fnum is the aperture size of the raw data
  • k and b are constants, for example, k is 0.3 and b is 5.1.
  • step 214 the calculated AWB result is fused with the night light source parameter or the correction result of the historical image.
  • Step 215 Transfer the final AWB result obtained by fusion to the ISP system.
  • step 213 as shown in Figure 2I, the following operations are included:
  • Step 2131 obtain the image and the target parameter LV of the image
  • the first reference parameter LV Index0 and the second reference parameter LV Index1 that are associated with the target parameter LV of the image are selected.
  • Step 2132 obtain LV Index0 and LV Index1;
  • the corresponding color temperature curve and white balance statistical area are extracted through LV Index0 and LV Index1.
  • Step 2133 input the color temperature curve and the white balance statistical area corresponding to LV Index0 and LV Index1 respectively as the condition for the AWB algorithm to detect the white point, and perform the AWB calculation respectively;
  • the AWB algorithm can be the gray-scale world method, the maximum brightness method, the improved gray-scale world method, the color gamut boundary method, the frame area segmentation method, the light source prediction method, the perfect reflection method, the dynamic threshold method, and the fuzzy logic method.
  • Step 2134 Calculate the fusion ratio lvratio
  • the calculation process of the fusion ratio lvratio may be: calculating the difference between the target parameter lv and the first reference parameter LV Index0, and calculating the difference between the target reference parameter lv and the second reference parameter LV Index1, and normalizing the calculated two differences
  • the normalization can also be calculated by adopting a weighted method. That is, lvratio represents the weight of the target parameter lv from the first reference parameter LV Index0 (upper boundary), and (1-lvratio) represents the weight of the target parameter lv from the first reference parameter LV Index1 (lower boundary).
  • Step 2135 Calculate the fused target correlated color temperature CCT-final and the first target white balance result AWBGain_tmp according to lvratio;
  • CCT-final CCT0*lvratio+CCT1*(1-lvratio)
  • AWBGain_tmp AWBGain0*lvratio+AWBgain1*(1-lvratio).
  • Step 2136 Calculate the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient
  • the second reference color temperature mapping coefficient Remap1 (rratio, bratio) is determined.
  • Step 2137 calculate the target color temperature mapping coefficient Remap
  • Remap Remap0*lvratio+Remap1(1-lvratio).
  • Step 2138 calculate and obtain the second target white balance result AWBGain-final
  • step 215AWB and night light source parameters or pixel correction results of historical images may be:
  • Blend_Ratio is obtained in Figure 2G;
  • AWBGain is equal to AWBGain_t in time domain (night light source parameter or pixel correction result of historical image) times (1-Blend_Ratio) plus AWBGain-Final*Blend_ratio.
  • the image processing method of this embodiment can better demonstrate the white balance restoration capability of low-light scenes compared with ordinary white balance algorithms, and provide a more favorable guarantee for the image effect experience of super night scenes .
  • the foregoing method embodiments may be implemented by software, and may also be implemented by hardware or a combination of software and hardware.
  • a device in a logical sense is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory for operation by the image processing processor where it is located.
  • FIG. 3 it is a hardware structure diagram of an image processing apparatus 300 for implementing the image processing method of this embodiment. Except for the processor 301 and the memory 302 shown in FIG. 3 , in the embodiment
  • the image processing device used to implement the image processing method generally may also include other hardware according to the actual function of the image processing device, which will not be repeated here.
  • the processor 301 implements the following operations when executing the computer program:
  • the target parameter is related to the brightness of the shooting scene of the image
  • a reference parameter associated with the target parameter from a preset reference database, and obtain a reference color temperature line and a white balance statistical area corresponding to the selected reference parameter; wherein the reference parameter and the shooting scene of the image Brightness is related, and the reference database includes: a plurality of reference parameters, and a reference color temperature line and a white balance statistical area corresponding to each reference parameter;
  • White balance processing is performed on the image according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.
  • the multiple reference parameters included in the reference database, as well as the reference color temperature line and the white balance statistical area corresponding to each reference parameter are obtained by pre-collecting multiple images of different standard light sources under different reference parameters, using generated from the acquired images.
  • the value of the selected reference parameter is the same as the value of the target parameter.
  • two reference parameters are selected; among the plurality of reference parameters, the values of the two reference parameters and the value of the target parameter have the smallest difference.
  • the processor performs white balance processing operations on the image according to the target parameters, the selected reference parameters and their corresponding reference color temperature lines and white balance statistical regions, including:
  • the first target white balance result is determined according to the target parameter, the selected reference parameter and its corresponding reference color temperature line and white balance statistical area.
  • the reference color temperature line includes a first reference color temperature line and a second reference color temperature line
  • the two reference parameters include a first reference parameter and a second reference parameter
  • the processor processes the target parameters according to the target parameters.
  • the selected reference parameter and its corresponding reference color temperature line and white balance statistical area determine the operation of the first target white balance result, including:
  • the first reference white balance result and the second reference white balance result are fused to determine the first target white balance result.
  • the weight is determined based on differences between the first reference parameter and the second reference parameter and the target parameter, respectively.
  • the processor when executing the instructions, further implements the following operations:
  • a second target white balance result is determined according to the offset of the reference color temperature line relative to the preset standard color temperature line.
  • the processor performs the operation of determining the second target white balance result according to the offset of the reference color temperature line relative to the preset standard color temperature line, including:
  • a second target white balance result is determined according to the operational relationship between the first target white balance result and the offset.
  • the offset includes a color temperature mapping coefficient.
  • the first reference white balance result includes a first reference correlated color temperature
  • the second reference white balance result includes a second reference correlated color temperature
  • the second target white balance result includes a target correlated color temperature
  • the The processor performs the operation of determining the second target white balance result according to the offset of the reference color temperature line relative to the preset standard color temperature line, including:
  • the first reference color temperature mapping coefficient and the second reference color temperature mapping coefficient are fused to determine the target color temperature mapping coefficient
  • the second target white balance result is determined according to the target color temperature mapping coefficient.
  • the processor performing the operation of determining the second target white balance result according to the target color temperature mapping coefficient includes:
  • the second target white balance result is determined according to the product of the first target white balance result and the target color temperature mapping coefficient.
  • the image is one frame of images in a continuous shooting scene
  • the processor further implements the following operations when executing the instruction:
  • the second target white balance result is fused with the pixel correction result of the historical image to determine a third target white balance result.
  • the pixel correction result of the historical image includes: the pixel correction result of the previous frame of the image, or the time domain pixel correction result of the historical image.
  • the processor further performs the following operations:
  • the second target white balance result is fused with preset night scene light source parameters to determine a fourth target white balance result.
  • the fusion ratio of the second target white balance result and the pixel correction result of the historical image is preset.
  • the fusion ratio of the second target white balance result and the preset night scene light source parameters is preset.
  • the fusion ratio corresponds to a target parameter.
  • the target parameters include any one or more of the following: a scene brightness parameter, a signal-to-noise ratio of an image, an exposure parameter of the image sensor, or a dead pixel parameter of the image sensor.
  • the scene brightness parameter of the image is determined by the exposure parameter of the image sensor and the average brightness of the image.
  • an embodiment of the application further provides a photographing device 400 , including: a casing 401 ; a lens assembly 402 , which is arranged inside the casing 401 ; and a sensor assembly 403 , which is arranged inside the casing 401 for sensing Pass the light of the lens assembly 402 and generate an electrical signal; and the image processing apparatus 300 according to any one of the embodiments.
  • an embodiment of the present application further provides a movable platform 500, including: a body 501; a power system 502 installed in the body 501 and used to provide power for the movable platform; and any The image processing apparatus 300 described in the embodiment.
  • the movable platform 500 is a vehicle, a drone or a mobile robot.
  • the embodiments of this specification further provide a computer-readable storage medium, where several computer instructions are stored on the readable storage medium, and when the computer instructions are executed, the operations of the image processing method described in any one of the embodiments are performed.
  • Embodiments of the present specification may take the form of a computer program product embodied on one or more storage media having program code embodied therein, including but not limited to disk storage, CD-ROM, optical storage, and the like.
  • Computer-usable storage media includes permanent and non-permanent, removable and non-removable media, and storage of information can be accomplished by any method or technology.
  • Information may be computer readable instructions, data structures, modules of programs, or other data.
  • Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.
  • PRAM phase-change memory
  • SRAM static random access memory
  • DRAM dynamic random access memory
  • RAM random access memory
  • ROM read only memory
  • EEPROM Electrically Erasable Programmable Read Only Memory
  • Flash Memory or other memory technology
  • CD-ROM Compact Disc Read Only Memory
  • CD-ROM Compact Disc Read Only Memory
  • DVD Digital Versatile Disc
  • Magnetic tape cassettes magnetic tape magnetic disk storage or other magnetic storage devices or any other non-

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Television Image Signal Generators (AREA)
  • Processing Of Color Television Signals (AREA)

Abstract

L'invention concerne un procédé et un appareil de traitement d'image, un dispositif de capture (400), une plateforme mobile (500), et un support de stockage lisible par ordinateur. Une base de données de référence comprend de multiples paramètres de référence, et des lignes de température de couleur de référence et des zones statistiques d'équilibrage des blancs correspondant aux paramètres de référence ; différents paramètres de référence sont définis pour simuler différents environnements ayant différents degrés de luminosité et d'obscurité qui peuvent apparaître dans des scènes de capture réelles, de manière à configurer des données de référence qui sont applicables à des environnements réels ayant différents degrés de luminosité et d'obscurité, et ainsi, pendant un traitement d'image, les données de référence ayant une luminosité adaptée de la scène de capture de l'image peuvent être sélectionnées pour un équilibrage des blancs ; en particulier dans un environnement à faible luminosité, étant donné que des données de référence correspondantes peuvent être sélectionnées, l'image capturée dans l'environnement à faible luminosité peut également avoir un bon effet d'équilibrage des blancs.
PCT/CN2020/119656 2020-09-30 2020-09-30 Procédé et appareil de traitement d'image, dispositif de capture, plateforme mobile, et support de stockage lisible par ordinateur WO2022067761A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/119656 WO2022067761A1 (fr) 2020-09-30 2020-09-30 Procédé et appareil de traitement d'image, dispositif de capture, plateforme mobile, et support de stockage lisible par ordinateur

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2020/119656 WO2022067761A1 (fr) 2020-09-30 2020-09-30 Procédé et appareil de traitement d'image, dispositif de capture, plateforme mobile, et support de stockage lisible par ordinateur

Publications (1)

Publication Number Publication Date
WO2022067761A1 true WO2022067761A1 (fr) 2022-04-07

Family

ID=80949414

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/119656 WO2022067761A1 (fr) 2020-09-30 2020-09-30 Procédé et appareil de traitement d'image, dispositif de capture, plateforme mobile, et support de stockage lisible par ordinateur

Country Status (1)

Country Link
WO (1) WO2022067761A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114710653A (zh) * 2022-04-15 2022-07-05 浙江大华技术股份有限公司 一种白平衡调整方法、装置、电子设备及存储介质
CN114979608A (zh) * 2022-05-06 2022-08-30 维沃移动通信有限公司 白平衡调整方法、装置、电子设备和存储介质
CN116055699A (zh) * 2022-07-28 2023-05-02 荣耀终端有限公司 一种图像处理方法及相关电子设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050286097A1 (en) * 2004-06-25 2005-12-29 Szepo Hung Automatic white balance method and apparatus
CN104935903A (zh) * 2014-03-18 2015-09-23 韩华泰科株式会社 白平衡校正设备和白平衡校正方法
CN109729332A (zh) * 2018-12-12 2019-05-07 珠海亿智电子科技有限公司 一种自动白平衡矫正方法和系统
CN111698493A (zh) * 2020-06-02 2020-09-22 Oppo广东移动通信有限公司 白平衡处理方法和装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050286097A1 (en) * 2004-06-25 2005-12-29 Szepo Hung Automatic white balance method and apparatus
CN104935903A (zh) * 2014-03-18 2015-09-23 韩华泰科株式会社 白平衡校正设备和白平衡校正方法
CN109729332A (zh) * 2018-12-12 2019-05-07 珠海亿智电子科技有限公司 一种自动白平衡矫正方法和系统
CN111698493A (zh) * 2020-06-02 2020-09-22 Oppo广东移动通信有限公司 白平衡处理方法和装置

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114710653A (zh) * 2022-04-15 2022-07-05 浙江大华技术股份有限公司 一种白平衡调整方法、装置、电子设备及存储介质
CN114979608A (zh) * 2022-05-06 2022-08-30 维沃移动通信有限公司 白平衡调整方法、装置、电子设备和存储介质
CN116055699A (zh) * 2022-07-28 2023-05-02 荣耀终端有限公司 一种图像处理方法及相关电子设备
CN116055699B (zh) * 2022-07-28 2023-10-20 荣耀终端有限公司 一种图像处理方法及相关电子设备

Similar Documents

Publication Publication Date Title
WO2020207239A1 (fr) Procédé et appareil de traitement d'images
US10949958B2 (en) Fast fourier color constancy
US9996913B2 (en) Contrast based image fusion
WO2022067761A1 (fr) Procédé et appareil de traitement d'image, dispositif de capture, plateforme mobile, et support de stockage lisible par ordinateur
CN111028190A (zh) 图像处理方法、装置、存储介质及电子设备
US20070047803A1 (en) Image processing device with automatic white balance
CN112565636B (zh) 图像处理方法、装置、设备和存储介质
WO2021143300A1 (fr) Procédé et appareil de traitement d'images, dispositif électronique et support de stockage
WO2019019904A1 (fr) Procédé et appareil de traitement d'équilibrage de blancs, et terminal
WO2022067762A1 (fr) Procédé et appareil de traitement d'image, dispositif de photographie, plateforme mobile et support de stockage lisible par ordinateur
CN105163047A (zh) 一种基于色彩空间转换的hdr图像生成方法、系统及拍摄终端
CN112907497B (zh) 图像融合方法以及图像融合装置
US20180025476A1 (en) Apparatus and method for processing image, and storage medium
US7885458B1 (en) Illuminant estimation using gamut mapping and scene classification
CN107682611B (zh) 对焦的方法、装置、计算机可读存储介质和电子设备
US11115637B2 (en) Image processing apparatus, image capturing apparatus, image processing method, and storage medium
JP2007311895A (ja) 撮像装置、画像処理装置、画像処理方法及び画像処理プログラム
CN114286000B (zh) 图像色彩处理方法、装置和电子设备
JP2007221678A (ja) 撮像装置、画像処理装置、画像処理方法及び画像処理プログラム
CN112995633B (zh) 图像的白平衡处理方法、装置、电子设备和存储介质
CN109345602A (zh) 图像处理方法和装置、存储介质、电子设备
CN114125408A (zh) 图像处理方法及装置、终端和可读存储介质
JP2007293686A (ja) 撮像装置、画像処理装置、画像処理方法及び画像処理プログラム
CN109447925B (zh) 图像处理方法和装置、存储介质、电子设备
Jo et al. Adaptive white point extraction based on dark channel prior for automatic white balance

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20955784

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20955784

Country of ref document: EP

Kind code of ref document: A1