CN112995634B

CN112995634B - Image white balance processing method and device, electronic equipment and storage medium

Info

Publication number: CN112995634B
Application number: CN202110427639.8A
Authority: CN
Inventors: 刘闯
Original assignee: Seashell Housing Beijing Technology Co Ltd
Current assignee: Seashell Housing Beijing Technology Co Ltd
Priority date: 2021-04-21
Filing date: 2021-04-21
Publication date: 2021-07-20
Anticipated expiration: 2041-04-21
Also published as: CN112995634A

Abstract

The embodiment of the disclosure discloses a white balance processing method and device for an image, an electronic device and a storage medium. The white balance processing method of the image comprises the following steps: acquiring an image to be processed in a first color space; performing light source estimation on an image to be processed to obtain a first light source estimation result; performing first white balance processing on the image to be processed based on the first light source estimation result to generate a first white balance processed image; converting the color space of the image subjected to the first white balance processing into a second color space to obtain a converted image; performing light source estimation on the converted image to obtain a second light source estimation result; and performing second white balance processing on the image to be processed based on the first light source estimation result and the second light source estimation result to generate a second white-balance-processed image. The embodiment of the disclosure can perform white balance processing on the image to be processed based on the first light source estimation result and the second light source estimation result, thereby improving the accuracy of the white balance processing of the image.

Description

Image white balance processing method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to image processing technologies, and in particular, to a method and an apparatus for processing white balance of an image, an electronic device, and a storage medium.

Background

White Balance (WB), an index describing the accuracy of White in an image. White balance is generated along with the reproduction of colors of electronic images, and is earlier applied in the field of professional photography. Home electronics (home video cameras, digital cameras) are also widely used.

In practice, many users encounter such a problem in capturing images using an image capturing apparatus: the image shot in the room of the fluorescent lamp appears green, the scenery shot under the indoor tungsten lamp light is yellowish, and the picture shot in the sunlight shadow is inexplicably bluish, which is caused by the processing mode of the white balance algorithm.

Currently, white balance is challenged by indoor environments represented by their rich colors, complex light source environments, large-area monochromatic scenes, and the like. Moreover, in the process of ISP (Internet Service Provider), various links have coupling effects of different degrees, so that under the complicated indoor environment shooting condition, the mutual influence and the mutual coupling effect are amplified. For example, after conversion by CCM (color correction parameter matrix), it will couple with white balance to cause influence, which greatly affects the accuracy of white balance processing. Therefore, in the above-mentioned scenario, how to improve the accuracy of the image white balance processing is a significant technical problem.

Disclosure of Invention

The embodiment of the disclosure provides a white balance processing method and device for an image, an electronic device and a storage medium, so as to improve the accuracy of the white balance processing of the image.

According to a first aspect of the embodiments of the present disclosure, there is provided a white balance processing method for an image, including:

acquiring an image to be processed in a first color space;

performing light source estimation on the image to be processed to obtain a first light source estimation result;

performing first white balance processing on the image to be processed based on the first light source estimation result to generate a first white balance processed image;

converting the color space of the first white balance processed image into a second color space to obtain a converted image;

performing light source estimation on the converted image to obtain a second light source estimation result;

and performing second white balance processing on the image to be processed based on the first light source estimation result and the second light source estimation result to generate a second white-balance processed image.

Optionally, in the method according to any embodiment of the present disclosure, the performing, based on the first illuminant estimation result and the second illuminant estimation result, second white balance processing on the image to be processed to generate a second white-balance-processed image includes:

determining a first conversion matrix based on the first illuminant estimation result, wherein the first conversion matrix is used for converting the image to be processed into a white balance processed image of the image to be processed;

determining a second conversion matrix based on the second illuminant estimation result, wherein the second conversion matrix is used for converting the converted image into a white balance processed image of the converted image;

and performing third white balance processing on the image to be processed based on the first conversion matrix and the second conversion matrix to generate a second white-balance processed image.

Optionally, in the method according to any embodiment of the present disclosure, the performing, on the basis of the first conversion matrix and the second conversion matrix, third white balance processing on the image to be processed to generate a second white-balance-processed image includes:

and performing fourth white balance processing on the image to be processed based on the first conversion matrix, the second conversion matrix and a third conversion matrix to generate a second white-balance processed image, wherein the third conversion matrix is used for converting the image in the first color space into the image in the second color space.

Optionally, in the method according to any embodiment of the present disclosure, the performing, on the basis of the first conversion matrix, the second conversion matrix, and the third conversion matrix, fourth white balance processing on the image to be processed to generate a second white-balanced processed image includes:

determining a color cast limit matrix for the first transformation matrix based on the first, second, and third transformation matrices;

and performing fifth white balance processing on the image to be processed based on the color cast limiting matrix to generate a second white-balance processed image.

Optionally, in the method of any embodiment of the present disclosure, the first illuminant estimation result includes a color value of an illuminant; and

converting the color space of the first white balance processed image into a second color space to obtain a converted image, including:

carrying out normalization processing on the color value of the light source to obtain a normalized color value of the light source;

converting the normalized light source color value into a light source color value in the first color space to obtain a converted light source color value;

calculating a color temperature corresponding to the converted light source color value;

determining a fourth conversion matrix based on the color temperature, wherein the fourth conversion matrix is used for converting the color space of the white balance processed image of the image in the first color space into a second color space;

and converting the color space of the first white balance processed image into a second color space based on the fourth conversion matrix to obtain a converted image.

Optionally, in the method according to any embodiment of the present disclosure, the determining a fourth conversion matrix based on the color temperature includes:

determining a result of multiplying the color space conversion matrix corresponding to the preset first color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being less than a preset first color temperature;

determining a result of multiplying a color space conversion matrix corresponding to a preset second color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being equal to the preset second color temperature;

determining a result of multiplying a color space conversion matrix corresponding to a preset third color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being greater than the preset third color temperature;

in response to the color temperature being greater than the preset second color temperature and less than or equal to the preset third color temperature, performing a first operation of:

recalculating the converted light source color value;

setting the color temperature corresponding to the recalculated converted light source color value as a first target color temperature;

responding to the fact that the difference between the color temperature obtained by the last calculation and the color temperature obtained by the last calculation is smaller than a preset value, and determining a fourth conversion matrix based on the size relation between the first target color temperature and the preset second color temperature and the preset third color temperature;

in response to the color temperature being greater than or equal to the preset first color temperature and less than the preset second color temperature, performing a second operation of:

recalculating the converted light source color value;

setting the color temperature corresponding to the recalculated converted light source color value as a second target color temperature;

and determining a fourth conversion matrix based on the magnitude relation between the second target color temperature and the preset first color temperature and the preset third color temperature in response to the difference between the color temperature obtained by the last calculation and the color temperature obtained by the last calculation being smaller than a preset value.

Optionally, in the method of any embodiment of the present disclosure, the determining a fourth conversion matrix based on the color temperature further includes at least one of:

when the color temperature is greater than the preset second color temperature and less than or equal to the preset third color temperature, responding to that the difference between the color temperature obtained by the last calculation and the color temperature obtained by the last calculation is greater than or equal to the preset value, and continuing to execute the first operation:

and when the color temperature is greater than or equal to the preset first color temperature and less than the preset second color temperature, continuing to execute the second operation.

Optionally, in the method according to any embodiment of the present disclosure, the acquiring an image to be processed in a first color space includes:

acquiring an original image;

carrying out corrosion operation on an image mask of the original image to obtain an image to be processed, and taking an image area corresponding to the image to be processed in the original image as the image to be processed;

and converting the image to be processed into the image to be processed in the first color space.

Optionally, in the method according to any embodiment of the present disclosure, the converting the image to be processed into the image to be processed in the first color space includes:

determining an exposure magnification for exposure processing based on a gray-scale image corresponding to the image to be processed;

exposing the image to be processed by adopting the exposure multiplying power to generate a first exposed image;

and generating an image to be processed in a first color space based on the first exposed image.

Optionally, in the method according to any embodiment of the present disclosure, the determining an exposure magnification for performing exposure processing based on a grayscale image corresponding to the image to be processed includes:

sequentially selecting exposure multiplying powers according to the order from small to large aiming at the exposure multiplying powers in a predetermined exposure multiplying power set, and executing the following first exposure multiplying power determination operation based on the selected exposure multiplying powers:

carrying out exposure processing on the image to be processed by adopting the selected exposure multiplying power to generate a second exposed image of the image to be processed aiming at the exposure multiplying power;

converting the second exposed image of the image to be processed aiming at the exposure multiplying power into a gray image, and generating the gray image of the image to be processed aiming at the exposure multiplying power;

determining whether the mean value of the brightness of the pixels in the gray level image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold value;

and in response to the fact that the mean value of the brightness of the pixels in the gray-scale image of the exposure magnification of the image to be processed is larger than or equal to the preset brightness threshold, taking the adopted exposure magnification as the exposure magnification for exposure processing.

Optionally, in the method of any embodiment of the present disclosure, the method further includes:

and in response to that the mean value of the brightness of the pixels in the gray-scale image of the exposure magnification of the image to be processed is smaller than the preset brightness threshold, continuing to reselect the exposure magnification from the exposure magnification set according to the sequence from small to large, and executing the first exposure magnification determination operation based on the reselected exposure magnification.

selecting an exposure magnification from a predetermined exposure magnification interval, and performing a second exposure magnification determination operation based on the selected exposure magnification, the second exposure magnification determination operation being:

and in response to that the mean value of the brightness of the pixels in the gray-scale image of the exposure magnification of the image to be processed is smaller than the preset brightness threshold, reselecting the unselected exposure magnification from the exposure magnification interval, and executing the second exposure magnification determination operation based on the reselected exposure magnification.

Optionally, in the method according to any embodiment of the present disclosure, the generating, based on the first exposed image, an image to be processed in a first color space includes:

and performing image division on the first exposed image based on the image size of the original image to obtain an image area set, and taking at least one image area in the image area set as an image to be processed.

Optionally, in the method according to any embodiment of the present disclosure, the performing fifth white balance processing on the image to be processed based on the color cast limiting matrix to generate a second white-balanced processed image includes:

determining a target pixel corresponding to each image to be processed based on the color cast limiting matrix, wherein the target pixel is a pixel to be subjected to white balance processing;

and performing sixth white balance processing on the image to be processed based on the determined target pixels to generate a second white-balance-processed image.

taking image pixels in the image to be processed as effective pixels; and

the taking at least one image area in the image area set as an image to be processed includes:

and taking a target image area in the image area set as an image to be processed, wherein the product of the number of all pixels in the target image area and a preset percentage is less than or equal to the number of effective pixels in the target image area.

Optionally, in the method according to any embodiment of the present disclosure, the performing sixth white balance processing on the image to be processed based on the determined target pixels to generate a second white-balance-processed image includes:

for each determined target pixel, determining a value of a first coordinate and a value of a second coordinate of the target pixel based on the color values of three channels of the target pixel;

calculating the distance from the target pixel to a preset color temperature curve in a preset coordinate system based on the value of the first coordinate and the value of the second coordinate of the target pixel, wherein the preset coordinate system comprises the first coordinate and the second coordinate, and the preset color temperature curve represents the corresponding relation between the value of the first coordinate and the value of the second coordinate;

and performing seventh white balance processing on the image to be processed based on the distance from the determined target pixel to the preset color temperature curve to generate a second white-balance-processed image.

Optionally, in the method according to any embodiment of the present disclosure, the performing a seventh white balance processing on the image to be processed based on the determined distance between the target pixel and the predetermined color temperature curve to generate a second white-balanced processed image includes:

determining the weight corresponding to each determined target pixel based on the distance from each determined target pixel to the predetermined color temperature curve;

determining color values of a red channel, a green channel and a blue channel of the target pixels based on the determined weights corresponding to the target pixels;

and performing eighth white balance processing on the image to be processed based on the color value of the red channel, the color value of the green channel and the color value of the blue channel of the target pixel to generate a second white balance processed image.

Optionally, in the method according to any embodiment of the present disclosure, a value of a first coordinate of a target pixel is a ratio of a color value of a blue channel of the target pixel to a color value of a green channel of the target pixel, and a value of a second coordinate of the target pixel is a ratio of a color value of a red channel of the target pixel to a color value of a green channel of the target pixel; and

the method further comprises the following steps:

aiming at each color temperature value in the preset color temperature value set, respectively shooting a preset number of pictures for each standard color card in a predetermined standard color card by adopting the color temperature value;

calculating a value of a first coordinate and a value of a second coordinate of a pixel in a shot picture; the value of the first coordinate of the pixel is the ratio of the color value mean value of the blue channel of the pixel to the color value mean value of the green channel; the value of the second coordinate of the pixel is the ratio of the color value mean value of the red channel of the pixel to the color value mean value of the green channel; the color value average of the red channel of a pixel is: the average value of the color values of the red channels of the pixels corresponding to the pixels in the preset number of pictures of the same standard color card under the same color temperature value; the color value average of the green channel of a pixel is: the average value of the color values of the green channels of the pixels corresponding to the pixels in the preset number of pictures of the same standard color card under the same color temperature value; the color value average of the blue channel of a pixel is: the average value of the color values of the blue channels of the pixels corresponding to the pixels in the preset number of pictures of the same standard color card under the same color temperature value;

and generating a predetermined color temperature curve based on the calculated values of the first coordinate and the second coordinate of the pixel.

According to a second aspect of the embodiments of the present disclosure, there is provided a white balance processing apparatus for an image, including:

an acquisition unit configured to acquire an image to be processed in a first color space;

the first light source estimation unit is configured to perform light source estimation on the image to be processed to obtain a first light source estimation result;

a first generation unit configured to perform first white balance processing on the image to be processed based on the first light source estimation result, and generate a first white-balance-processed image;

a conversion unit configured to convert a color space of the first white balance processed image into a second color space, resulting in a converted image;

the second light source estimation unit is configured to perform light source estimation on the converted image to obtain a second light source estimation result;

and the second generation unit is configured to perform second white balance processing on the image to be processed based on the first light source estimation result and the second light source estimation result to generate a second white balance processed image.

Optionally, in the apparatus of any embodiment of the present disclosure, the second generating unit is further configured to:

Optionally, in the apparatus of any embodiment of the present disclosure, the first illuminant estimation result includes a color value of an illuminant; and

the conversion unit is further configured to:

Optionally, in the apparatus of any embodiment of the present disclosure, the conversion unit is further configured to:

recalculating the converted light source color value;

Optionally, in the apparatus of any embodiment of the present disclosure, the obtaining unit is further configured to:

acquiring an original image;

carrying out corrosion operation on an image mask of the original image to obtain a corroded image, and taking an image area corresponding to the corroded image in the original image as an image to be processed;

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus including:

a memory for storing a computer program;

a processor for executing the computer program stored in the memory, and when the computer program is executed, the method of any embodiment of the white balance processing method of the image of the first aspect of the present disclosure is realized.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable medium, which when executed by a processor, implements the method of any one of the embodiments of the white balance processing method for an image according to the first aspect described above.

According to a fifth aspect of embodiments of the present disclosure, there is provided a computer program comprising computer readable code which, when run on a device, causes a processor in the device to execute instructions for implementing the steps in the method as in any one of the embodiments of the method for white balance processing of an image of the first aspect described above.

Based on the image white balance processing method, the image white balance processing device, the electronic device, and the storage medium provided by the embodiments of the present disclosure, an image to be processed in a first color space may be obtained, then, light source estimation is performed on the image to be processed to obtain a first light source estimation result, then, first white balance processing is performed on the image to be processed based on the first light source estimation result to generate a first white balance processed image, then, a color space of the first white balance processed image is converted into a second color space to obtain a converted image, then, light source estimation is performed on the converted image to obtain a second light source estimation result, and finally, second white balance processing is performed on the image to be processed based on the first light source estimation result and the second light source estimation result to generate a second white balance processed image. The embodiment of the disclosure can combine the first illuminant estimation result and the second illuminant estimation result to realize white balance processing, thereby improving the accuracy of the image white balance processing.

The technical solution of the present disclosure is further described in detail by the accompanying drawings and examples.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.

The present disclosure may be more clearly understood from the following detailed description, taken with reference to the accompanying drawings, in which:

fig. 1 is a flowchart of a first embodiment of a white balance processing method of an image according to the present disclosure.

Fig. 2 is a flowchart of a white balance processing method of an image according to a second embodiment of the disclosure.

Fig. 3 is a flowchart of a white balance processing method of an image according to a third embodiment of the present disclosure.

Fig. 4 is a schematic diagram illustrating a generation process of a color temperature curve in an embodiment of the white balance processing method for an image according to the present disclosure.

Fig. 5 is a schematic structural diagram of an embodiment of a white balance processing apparatus for an image according to the present disclosure.

Fig. 6 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.

Detailed Description

Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

It will be understood by those of skill in the art that the terms "first," "second," and the like in the embodiments of the present disclosure are used merely to distinguish one element from another, and are not intended to imply any particular technical meaning, nor is the necessary logical order between them.

It is also understood that in embodiments of the present disclosure, "a plurality" may refer to two or more and "at least one" may refer to one, two or more.

It is also to be understood that any reference to any component, data, or structure in the embodiments of the disclosure, may be generally understood as one or more, unless explicitly defined otherwise or stated otherwise.

In addition, the term "and/or" in the present disclosure is only one kind of association relationship describing an associated object, and means that three kinds of relationships may exist, for example, a and/or B may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in the present disclosure generally indicates that the former and latter associated objects are in an "or" relationship.

It should also be understood that the description of the various embodiments of the present disclosure emphasizes the differences between the various embodiments, and the same or similar parts may be referred to each other, so that the descriptions thereof are omitted for brevity.

Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

The disclosed embodiments may be applied to at least one of a terminal device, a computer system, and a server, which are operational with numerous other general purpose or special purpose computing system environments or configurations. Examples of well known computing systems, environments, and/or configurations that may be suitable for use with at least one electronic device of a terminal device, computer system, and server include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, hand-held or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, networked personal computers, minicomputer systems, mainframe computer systems, distributed cloud computing environments that include any of the above, and the like.

At least one of the terminal device, the computer system, and the server may be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform particular tasks or implement particular abstract data types. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

Referring to fig. 1, a flow 100 of a first embodiment of a method of white balance processing of an image according to the present disclosure is shown. The white balance processing method of the image comprises the following steps:

101, acquiring an image to be processed in a first color space.

In this embodiment, an execution subject (e.g., a server, a terminal device, an image processing unit with an image processing function, etc.) of the white balance processing method for an image may acquire the image to be processed in the first color space from other electronic devices or locally by a wired connection manner or a wireless connection manner.

The image to be processed may be an image to be subjected to white balance processing. For example, the image to be processed may be an original image (e.g., a raw image file), an image obtained based on the original image, or an image obtained after processing such as erosion operation.

The first color space may be any predetermined color space. As an example, the first color space may be native xyz (original space, i.e. the color space when the image is not transformed at all).

And 102, performing light source estimation on the image to be processed to obtain a first light source estimation result.

In this embodiment, the executing body may perform light source estimation on the to-be-processed image obtained in 101 to obtain a first light source estimation result.

The executing entity may perform light source estimation on the image to be processed obtained in 101 by using various light source estimation algorithms, which is not limited herein.

And 103, performing first white balance processing on the image to be processed based on the first light source estimation result to generate a first white-balance processed image.

In this embodiment, the executing entity may perform white balance processing (i.e., first white balance processing) on the image to be processed obtained in 101 based on the first light source estimation result obtained in 102, and generate a first white-balance-processed image.

As an example, the executing body may execute 103 as follows:

assuming that the first illuminant estimation result is rgb and the matrix of the image to be processed is Ii', the first white-balance-processed image IIi can be obtained as follows:

IIi = Ii '. rgb ', wherein rgb ' = max (rgb)/rgb

It will be appreciated that rgb above may be a matrix comprising 3 elements, max (rgb) being the maximum of the 3 elements r, g, b.

And 104, converting the color space of the image subjected to the first white balance processing into a second color space to obtain a converted image.

In this embodiment, the execution subject may convert the color space of the first white balance processed image into the second color space, resulting in a converted image.

The second color space may be any predetermined color space different from the first color space. As an example, the second color space may be an sRGB (standard Red Green Blue) color space.

Here, the execution body may execute the 104 in various ways, for example, a conversion matrix for converting a color space of the first white balance processed image into a second color space may be used to obtain a converted image.

And 105, performing light source estimation on the converted image to obtain a second light source estimation result.

In this embodiment, the executing entity may perform illuminant estimation on the converted image to obtain a second illuminant estimation result.

The executing body may perform light source estimation on the converted image by using various light source estimation algorithms. The illuminant estimation algorithm employed in 105 can be the same as or different from the illuminant estimation algorithm employed in 102.

And 106, performing second white balance processing on the image to be processed based on the first light source estimation result and the second light source estimation result to generate a second white-balance processed image.

In this embodiment, the executing entity may perform white balance processing (i.e., second white balance processing) on the image to be processed based on the first illuminant estimation result obtained in 102 and the second illuminant estimation result obtained in 105, and generate a second white-balanced processed image.

As an example, the executing body may input the first and second illuminant estimation results and a first white balance processing model trained in advance to the image to be processed, and generate a second white-balance-processed image by the first white balance processing model. The first white balance processing model may be a convolutional neural network obtained by training a training sample including a first sample light source estimation result, a second sample light source estimation result, a sample to-be-processed image, and a second sample white balance processed image.

The white balance processing method for an image according to the embodiment of the present disclosure may obtain an image to be processed in a first color space, perform light source estimation on the image to be processed to obtain a first light source estimation result, perform first white balance processing on the image to be processed based on the first light source estimation result to generate a first white-balance-processed image, convert a color space of the first white-balance-processed image into a second color space to obtain a converted image, perform light source estimation on the converted image to obtain a second light source estimation result, and perform white balance processing (i.e., third white balance processing) on the image to be processed based on the first light source estimation result and the second light source estimation result to generate a second white-balance-processed image. The embodiment of the disclosure can combine the first illuminant estimation result and the second illuminant estimation result to realize white balance processing, thereby improving the accuracy of the image white balance processing.

In some optional implementations of the embodiment, the executing body may execute the step 101 to obtain the to-be-processed image in the first color space by:

step one, acquiring an original image.

The raw image may include, among other things, image data processed from an image sensor of a digital camera, scanner, or motion picture film scanner. It will be appreciated that the original image is typically an image that has not been processed or edited.

And secondly, carrying out corrosion operation on the image mask of the original image to obtain a corroded image, and taking an image area corresponding to the corroded image in the original image as an image to be processed.

Here, the above-mentioned performing body may perform the etching operation in various ways, and is not limited herein. For example, in the case where the image to be processed is an image obtained after being subjected to a process such as a corrosion operation, the image to be processed may be obtained as follows: data of an original image corresponding to a point where a median value of an image mask (a binarized image) corresponding to the original image is 0 is removed, and calculation after the removal is not included. That is, the data of the original image corresponding to the point of the image mask whose median is 0 in the data from which the original image is to be removed is obtained, so as to obtain the image to be processed after the erosion operation.

And step three, converting the image to be processed into an image in the first color space, and taking the converted image as the image to be processed.

Here, the executing body may implement the third step by a conversion matrix for converting the image to be processed into the image to be processed in the first color space; the image to be processed can also be converted into the image to be processed in the first color space through a conversion model trained in advance. The conversion model may be a convolutional neural network model obtained by training based on a training sample including a sample image after corrosion and a sample image to be processed converted into a sample image in a first color space by using a machine learning algorithm.

It can be understood that the image to be processed in the above alternative implementation manner is the image to be processed after the erosion operation, so that the influence of the outer boundary on the white balance processing can be eliminated, and the accuracy of the white balance processing is further improved.

In some application scenarios in the above optional implementation, the executing entity may perform step three as follows to convert the image to be processed into the image to be processed in the first color space:

the first step is to determine an exposure magnification for performing exposure processing based on a grayscale image corresponding to an image to be processed.

The grayscale image corresponding to the image to be processed may be a grayscale image into which the image to be processed is converted. Optionally, the grayscale image corresponding to the image to be processed may also be obtained as follows: the method comprises the steps of carrying out exposure processing on an image to be processed by adopting a certain exposure multiplying power to generate an exposed image of the image to be processed, then converting the exposed image into a gray image, and taking the gray image obtained after conversion as a gray image corresponding to the image to be processed. Here, the exposure magnification may be a predetermined exposure magnification, or an exposure magnification such that the average value of the luminance of each pixel in a grayscale image into which an exposed image after exposure processing is converted is greater than or equal to a preset threshold value.

And a second step of performing exposure processing on the image to be processed by adopting the exposure magnification to generate a first post-exposure image.

As an example, the execution subject may multiply the value of each pixel (for example, a value of a single channel or a value of three channels) in the image to be processed by the exposure magnification to perform exposure processing on the image to be processed, thereby obtaining a first post-exposure image.

And a third step of generating an image to be processed in the first color space based on the first exposed image.

As an example, the executing entity may convert the first exposed image into an image in a first color space, and use the converted image as a to-be-processed image in the first color space.

It can be understood that, in the application scenario, the exposure magnification may be determined based on the grayscale image corresponding to the image to be processed, and the exposure processing is performed on the image to be processed by using the exposure magnification to obtain an exposed image, and then the white balance processing is performed on the image to be processed, so that different exposure magnifications for performing the exposure processing on the image to be processed may be determined for different images to be processed, and then the white balance processing is performed on the image to be processed based on the exposed image, thereby improving the accuracy of the white balance processing of the image.

In some cases in the application scenarios described above, the executing body may execute the first step described above to determine an exposure magnification for performing exposure processing based on a grayscale image corresponding to an image to be processed, in the following manner:

the exposure magnifications are sequentially selected in descending order of magnitude for the exposure magnifications in a predetermined exposure magnification set, and a first exposure magnification determination operation as follows is performed based on the selected exposure magnifications. Wherein the first exposure-magnification determining operation may include a first sub-step, a second sub-step, a third sub-step, and a fourth sub-step as follows.

For example, if the following exposure magnifications are included in the set of exposure magnifications: 1. 4, 16, 64, then, when selecting for the first time, the execution body may select 1 as the selected exposure magnification; in the second selection, the execution main body may select 4 as the selected exposure magnification; in the third selection, the execution body may select 16 as the selected exposure magnification; in the fourth selection, the execution body may select 64 as the selected exposure magnification.

Here, the subsequent steps (i.e., the first sub-step, the second sub-step, the third sub-step, and the fourth sub-step) are performed once for each selection of the exposure magnification, thereby obtaining the exposure magnification for performing the exposure process.

And the first substep, adopting the selected exposure multiplying power to carry out exposure processing on the image to be processed, and generating a second post-exposure image of the image to be processed aiming at the exposure multiplying power.

Here, the manner of performing exposure processing on the image may refer to the above description, and is not described herein again.

And a second sub-step of converting the second exposed image of the image to be processed with respect to the exposure magnification into a gray image, and generating the gray image of the image to be processed with respect to the exposure magnification.

As an example, in the case where the second post-exposure image is an RGB three-channel image, the execution subject described above may generate a grayscale image of the image to be processed for the exposure magnification based on a result of weighted summation of color values of three channels of pixels of the second post-exposure image, or based on an average value of color values of three channels of pixels of the second post-exposure image.

And a third substep of determining whether the mean value of the brightness of the pixels in the gray scale image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold value. The brightness of the pixel points in the gray image can be the gray of the pixel points in the gray image.

And a fourth substep of taking the exposure magnification adopted as the exposure magnification for exposure processing when the mean value of the brightness of the pixels in the gray scale image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold value.

Optionally, in the foregoing case, when the mean luminance value of the pixels in the grayscale image of the exposure magnification of the image to be processed is smaller than the preset luminance threshold, the executing main body may further reselect the exposure magnification from the exposure magnification set in descending order, and execute the first exposure magnification determination operation based on the reselected exposure magnification.

It can be understood that, in the above case, the minimum exposure magnification in the exposure magnification set that makes the luminance average value of the grayscale image corresponding to the image to be processed larger may be selected, so that not only the effect of exposure processing may be achieved, but also the image to be processed may be prevented from being overexposed to some extent, and thus the accuracy and robustness of the image white balance processing may be further improved.

Optionally, in the above case, the executing body further executes the first step in such a manner that an exposure magnification for performing exposure processing is determined based on a grayscale image corresponding to the image to be processed:

selecting an exposure magnification from a predetermined exposure magnification interval, and performing a second exposure magnification determination operation based on the selected exposure magnification, wherein the second exposure magnification determination operation includes sub-step one, sub-step two, sub-step three, and sub-step four.

And the first substep is to adopt the selected exposure multiplying power to carry out exposure processing on the image to be processed and generate a second post-exposure image of the image to be processed aiming at the exposure multiplying power.

Here, the exposure magnification may be randomly selected from the above-described exposure magnification interval, or the selection of the exposure magnification may be performed in order of increasing or decreasing every predetermined value (for example, 1, 2, or the like).

As an example, the executing body may multiply the value of each pixel (for example, the value of a single channel or the value of three channels) in the to-be-processed image of the to-be-processed image acquired in 101 by the selected exposure magnification, thereby generating a second post-exposure image of the to-be-processed image for the exposure magnification.

And a second substep of converting the second exposed image of the image to be processed corresponding to the exposure magnification into a gray image and generating the gray image of the image to be processed corresponding to the exposure magnification.

Here, the execution subject may convert the second post-exposure image of the image to be processed for the exposure magnification into a grayscale image in various ways.

As an example, in the case where the image to be processed is an RGB three-channel image, the execution subject described above may generate a grayscale image of the image to be processed for the exposure magnification based on the result of weighted summation of color values of three channels of pixels of the image to be processed, or based on the average value of color values of three channels of pixels of the image to be processed, and convert it as a grayscale image into which the image to be processed is converted for the second post-exposure image of the exposure magnification.

And a third substep of determining whether the mean value of the brightness of the pixels in the gray scale image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold value.

And a fourth substep of taking the adopted exposure magnification as the exposure magnification for exposure processing when the mean value of the brightness of the pixels in the gray scale image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold.

Optionally, the executing main body may further reselect an unselected exposure magnification from the exposure magnification interval when a luminance average of pixels in the gray-scale image of the exposure magnification of the image to be processed is smaller than a preset luminance threshold, and execute the second exposure magnification determining operation based on the reselected exposure magnification.

It can be understood that, under the condition that the luminance average value of the pixels in the gray-scale image of the exposure magnification of the image to be processed is smaller than the preset luminance threshold, the exposure magnification is not adopted for exposure processing of the image to be processed, but the unselected exposure magnification is reselected until the exposure magnification capable of enabling the luminance average value of the pixels in the gray-scale image of the exposure magnification of the image to be processed to be larger than or equal to the preset luminance threshold is selected, so that after the exposure processing of the image to be processed is carried out by adopting the determined exposure magnification, the obtained gray-scale image corresponding to the post-exposure image has higher luminance, and further, the white balance processing can be carried out on the image to be processed based on the first post-exposure image with the larger pixel value, and the accuracy of the white balance processing can be further improved.

In some cases in the application scenario, the executing entity may execute the third step in a manner as follows to generate the to-be-processed image in the first color space based on the first exposed image:

The number of image divisions can be determined according to actual requirements, for example, the number of image divisions can be positively or negatively correlated with the size of the original image.

Here, a part or all of the image regions in the image region set may be regarded as the image to be processed.

It can be understood that, in the foregoing case, the first post-exposure image is subjected to image division, and the image areas determined as the images to be processed are respectively subjected to white balance processing, so that more targeted white balance processing is performed on different characteristics of each image to be processed, thereby further improving the accuracy of the image white balance processing.

Optionally, after performing the erosion operation to obtain the to-be-processed image, the execution main body may further take, as an invalid pixel, a pixel in an image region where the to-be-processed image is eroded with respect to the original image; and taking the image pixel of the image to be processed which is not corroded relative to the original image as an effective pixel. Based on this, the above taking at least one image area in the image area set as the image to be processed may specifically be: and taking a target image area in the image area set as an image to be processed, wherein the product of the number of all pixels in the target image area and the preset percentage is less than or equal to the number of effective pixels in the target image area.

In some cases in the application scenario, the executing entity may execute the second step in a manner that a target pixel for performing white balance processing is determined based on the determined target image area, where the target pixel is a pixel to be subjected to white balance processing:

and determining pixels meeting preset conditions in the determined target image area as target pixels for white balance processing. Wherein the preset conditions include: the single-channel numerical value of the pixel corresponding to the pixel in the original image is larger than a first preset threshold value, and the sum of the three-channel numerical values of the pixel corresponding to the pixel in the image to be processed is smaller than a second preset threshold value.

It can be understood that, in the above case, the target pixel for performing the white balance processing is determined only based on the pixels meeting the preset condition, and the influence of the pixels not meeting the preset condition on the white balance processing of the image is eliminated, so that the influence of the dark noise and the excessively bright nonlinear region on the white balance can be avoided, and thus, the accuracy of the white balance processing of the image is further improved.

Optionally, the executing body may also execute the second step in the following manner to determine a target pixel for performing white balance processing based on the determined target image area: based on all the target pixels in the determined target image area, a target pixel for performing white balance processing is determined.

It can be understood that, in the application scenario, the target pixel for performing the white balance processing may be determined based on only the pixels in the target image area, and the influence of the pixels in the non-target image area with more invalid pixels on the image white balance processing is eliminated, thereby further improving the accuracy of the image white balance processing.

Optionally, the executing body may further perform white balance processing (i.e., sixth white balance processing) on the image to be processed based on the determined target pixels, so as to generate a second white-balance-processed image:

first, for each determined target pixel, a value of a first coordinate and a value of a second coordinate of the target pixel are determined based on the color values of three channels of the target pixel.

The color values of the three channels of the target pixel may include a color value of a red channel, a color value of a blue channel, and a color value of a green channel.

Here, how to calculate the value of the first coordinate and the value of the second coordinate of the target pixel may be determined according to actual needs.

As an example, the value of the first coordinate may be a ratio of a color value of the red channel to a color value of the blue channel, and the value of the second coordinate may be a ratio of a color value of the red channel to a color value of the green channel. Alternatively, the value of the first coordinate may also be a ratio of the color value of the blue channel to the color value of the red channel, and the value of the second coordinate may also be a ratio of the color value of the green channel to the color value of the red channel, and so on.

Then, a distance of the target pixel to the predetermined color temperature curve in the preset coordinate system is calculated based on the value of the first coordinate and the value of the second coordinate of the target pixel. The preset coordinate system comprises a first coordinate and a second coordinate, and the preset color temperature curve represents the corresponding relation between the value of the first coordinate and the value of the second coordinate.

And finally, performing white balance processing (namely seventh white balance processing) on the image to be processed based on the determined distance between the target pixel and the preset color temperature curve to generate a second white-balance-processed image.

In some cases in the application scenario, the executing entity may execute the third step in a manner that, based on the determined distance between the target pixel and the predetermined color temperature curve, the executing entity performs white balance processing on the image to be processed to generate a white-balanced processed image:

firstly, based on the distance of each determined target pixel to a predetermined color temperature curve, the weight corresponding to each determined target pixel is determined.

For example, the weight corresponding to the target pixel may be positively or negatively correlated with the distance from the target pixel to the predetermined color temperature curve.

Then, based on the determined weights corresponding to the respective target pixels, color values of a red channel, a green channel, and a blue channel of the target pixels are determined.

The target pixel may be a target pixel for white balance processing.

For example, the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel may be determined by the following formulas:

wherein R is the color value of the red channel of the target pixel, G is the color value of the green channel of the target pixel, B is the color value of the blue channel of the target pixel, i is used for uniquely identifying the determined target pixel, W_iCharacterizing the determined weight corresponding to the target pixel identified as i,

a color value characterizing the determined red channel of the target pixel identified as i,

a color value characterizing the determined green channel of the target pixel identified as i,

a color value characterizing the determined blue channel of the target pixel identified as i.

Optionally, the executing entity may further determine, based on the determined weight corresponding to each target pixel, a color value of a red channel, a color value of a green channel, and a color value of a blue channel of the target pixel by:

the color values of the blue channel characterizing the determined target pixels identified as i, N characterizing the total number of the determined respective target pixels.

And then, generating a white balance processed image based on the color value of the red channel, the color value of the green channel and the color value of the blue channel of the target pixel and the image to be processed.

As an example, the execution subject may first determine a color value with the largest numerical value among the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel. Then, the color value with the maximum value is divided by a matrix formed by the color value of the red channel, the color value of the green channel and the color value of the blue channel of the target pixel to obtain a first matrix. And then, taking the convolution result of the image matrix of the image to be processed and the first matrix as the white balance processed image.

Optionally, the executing body may further generate a white balance processed image based on the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel, and the image to be processed, as follows:

firstly, processing an image to be processed based on at least one of an etching operation and an exposure processing to obtain a processed image. Then, a color value with the largest numerical value among the color value of the red channel, the color value of the green channel, and the color value of the blue channel of the target pixel is determined. Then, the color value with the maximum value is divided by a matrix formed by the color value of the red channel, the color value of the green channel and the color value of the blue channel of the target pixel to obtain a first matrix. And then, the result of the convolution operation between the image matrix of the processed image and the first matrix is used as the white balance processed image.

It can be understood that, in the above case, the white balance processing may be performed on the image to be processed based on the determined weight corresponding to each target pixel, and thus, the white balance processing may be performed based on the weight corresponding to each target pixel, and thus, the robustness and accuracy of the target pixel in the finally generated white-balanced processed image may be further improved.

Optionally, the executing main body may also execute the third step in the following manner, so as to perform white balance processing on the image to be processed based on the determined distance between the target pixel and the predetermined color temperature curve, and generate an image after the white balance processing:

and determining the color temperature when the image to be processed is shot based on the determined distance between the target pixel and the preset color temperature curve, and further performing white balance processing on the image to be processed based on the color temperature to generate an image after the white balance processing.

It can be understood that, in the above application scenario, the white balance processing may be performed on the image to be processed based on the determined distance from the target pixel to the predetermined color temperature curve, so that the white balance processing is assisted by the predetermined color temperature curve, and the accuracy of the white balance processing may be further improved.

With further reference to fig. 2, fig. 2 is a flowchart of a second embodiment of the white balance processing method of an image of the present disclosure. The flow 200 of the white balance processing method of the image comprises the following steps:

201, an image to be processed in a first color space is obtained.

In this embodiment, an execution subject (for example, a server, a terminal device, an image processing unit with an image processing function, or the like) of the white balance processing method for an image may obtain an image to be processed in the first color space from another electronic device or locally through a wired connection manner or a wireless connection manner.

And 202, performing light source estimation on the image to be processed to obtain a first light source estimation result.

In this embodiment, the executing body may perform light source estimation on the image to be processed to obtain a first light source estimation result.

And 203, performing first white balance processing on the image to be processed based on the first light source estimation result to generate a first white-balance processed image.

In this embodiment, the executing body may perform the first white balance processing on the image to be processed based on the first illuminant estimation result, and generate the first white-balance-processed image.

And 204, converting the color space of the image subjected to the first white balance processing into a second color space to obtain a converted image.

And 205, performing light source estimation on the converted image to obtain a second light source estimation result.

In this embodiment, 201-205 are substantially the same as 101-105 in the corresponding embodiment of fig. 1, and are not described herein again.

Based on the first illuminant estimation result, a first conversion matrix is determined 206.

In this embodiment, the executing body may determine the first conversion matrix based on the first illuminant estimation result. The first conversion matrix is used for converting the image to be processed into a white balance processed image of the image to be processed.

As an example, if the first illuminant estimation result employs a matrix containing three elements

Characterisation, then a first transformation matrix T₁Can be as follows:

wherein the content of the first and second substances,

as a result of

、

The largest value among the values.

Based on the second illuminant estimation result, a second transformation matrix is determined 207.

In this embodiment, the executing body may determine the second conversion matrix based on the second illuminant estimation result. Wherein the second conversion matrix converts the converted image into a white balance processed image of the converted image.

Characterisation, then, the second transformation matrix T₂Can be as follows:

wherein the content of the first and second substances,

as a result of

、

、

The largest value among the values.

And 208, performing third white balance processing on the image to be processed based on the first conversion matrix and the second conversion matrix to generate a second white-balance processed image.

In this embodiment, the executing body may perform white balance processing (i.e., third white balance processing) on the image to be processed based on the first conversion matrix and the second conversion matrix, and generate a second white-balance-processed image.

In some optional implementations of this embodiment, the executing body may execute 208 to perform white balance processing (i.e., third white balance processing) on the image to be processed based on the first conversion matrix and the second conversion matrix to generate a second white-balance-processed image:

and performing white balance processing (namely fourth white balance processing) on the image to be processed based on the first conversion matrix, the second conversion matrix and the third conversion matrix to generate a second white-balance-processed image. The third conversion matrix is used for converting the image in the first color space into the image in the second color space.

As an example, the execution subject described above may generate the matrix of the second white-balance-processed image in the following manner:

and inputting the first conversion matrix, the second conversion matrix, the third conversion matrix and the image to be processed into a pre-trained second white balance processing model, so as to generate a second white balance processed image through the second white balance processing model. The second white balance processing model may be a convolutional neural network obtained by training a training sample including a first sample transformation matrix, a second sample transformation matrix, a third sample transformation matrix, a sample to-be-processed image, and a second sample white balance processed image.

It is to be understood that, in the above alternative implementation manner, the white balance processing may be performed on the image to be processed based on the first conversion matrix, the second conversion matrix, and the third conversion matrix. In this way, the accuracy of the white balance processing can be further improved.

In some application scenarios of the above optional implementation, the executing body may perform white balance processing on the image to be processed based on the first conversion matrix, the second conversion matrix, and the third conversion matrix in the following manner to generate a second white-balanced processed image:

first, a color cast limit matrix for the first conversion matrix is determined based on the first conversion matrix, the second conversion matrix, and the third conversion matrix. Wherein the color cast restriction matrix may be used to restrict the color cast effect of the first conversion matrix on the image.

As an example, the execution body may determine the color cast limit matrix of the first conversion matrix in the following manner:

firstly, the second light source estimation result is fed back and acted on the first light source estimation result, and the specific calculation process is that the first conversion matrix T is subjected to₁Adding some limitation to make white balance go through the first conversion matrix T after color space conversion₁And a second transformation matrix T₂The result of the co-processing is close to:

wherein the content of the first and second substances,

for the image to be processed, W represents a color cast restriction matrix, T₁Characterizing a first transformation matrix, T₂A second transformation matrix is characterized, and M represents a third transformation matrix.

The reason for this is: if the third conversion matrix M has a large influence on the white balance processing or is sensitive to the result of the white balance processing, it is necessary to perform the white balance processing before it is performed, that is, before the white balance processing is performed

The matrix is preceded by a matrix

To pair

A restriction is made.

The specific solving method of (1) is as follows:

due to the fact that

The matrix is a diagonal matrix, so it is also defined here

Is a diagonal matrix.

，

Is provided with

，

，

By solving the above equation, it can be found that:

wherein the content of the first and second substances,

the diagonal elements in the characterization matrix W,

the elements in the characterization matrix a are,

the elements in matrix B are characterized.

The final image after color space conversion by matrix M

Comprises the following steps:

wherein, in the step (A),

this is equivalent to in use

As a new light source, the accuracy of color recovery after color space conversion after the new light source is relatively insensitive to white balance, and the white balance algorithm is more robust.

Thus, the execution subject may perform white balance processing (i.e., fifth white balance processing) on the image to be processed based on the color cast restriction matrix, and generate a second white-balanced image.

As an example, the executing entity may input the color cast restriction matrix and the image to be processed to a third white balance processing model trained in advance, so as to generate a second white balance processed image by the third white balance processing model. The third white balance processing model may be a convolutional neural network obtained by training using a training sample including a sample color cast limiting matrix, a sample to-be-processed image, and a second sample white balance processed image.

It is to be understood that, in the above alternative implementation, the white balance processing may be performed on the image to be processed based on the color cast limiting matrix. In this way, the accuracy of the white balance processing can be further improved.

In some cases in the application scenario, the executing entity may further perform white balance processing on the image to be processed based on the color cast restriction matrix in the following manner to generate a second white-balanced processed image:

first, a target pixel corresponding to each image to be processed is determined based on the color cast restriction matrix.

Here, each image to be processed may correspond to one color cast restriction matrix, and each image to be processed may correspond to one target pixel.

For example, the RGB color values of the target pixel corresponding to the image to be processed may be determined based on each element in the color cast matrix. For example, when the color cast limits the matrix

Then, the R color value of the target pixel corresponding to the image to be processed may be

The G color value may be

The B color value may be

。

Then, based on the determined respective target pixels, white balance processing (i.e., sixth white balance processing) is performed on the image to be processed, and a second white-balanced processed image is generated.

As an example, the executing body may input the determined target pixels and the image to be processed to a fourth white balance processing model trained in advance, so that the second white-balance-processed image is generated by the fourth white balance processing model. The fourth white balance processing model may be a convolutional neural network obtained by training a training sample including a plurality of sample target pixels, a sample to-be-processed image, and a second sample white balance processed image.

It should be noted that, besides the above-mentioned contents, the embodiment of the present disclosure may also include the same or similar features and effects as the embodiment corresponding to fig. 1, and no further description is provided herein.

As can be seen from fig. 2, the flow 200 of the image white balance processing method in the present embodiment may perform white balance processing on the image to be processed based on the first conversion matrix and the second conversion matrix, thereby improving the accuracy and speed of the white balance processing.

With continuing reference to fig. 3, fig. 3 is a flowchart illustrating a white balance processing method for an image according to a third embodiment of the present disclosure. The flow 300 of the method for processing white balance of an image comprises the following steps:

301, acquiring an image to be processed in a first color space.

And 302, performing light source estimation on the image to be processed to obtain a first light source estimation result.

In this embodiment, the executing body may perform light source estimation on the image to be processed to obtain a first light source estimation result. Wherein the first illuminant estimation result comprises color values of the illuminant.

And 303, performing first white balance processing on the image to be processed based on the first illuminant estimation result to generate a first white-balance processed image.

And 304, carrying out normalization processing on the color value of the light source to obtain a normalized color value of the light source.

In this embodiment, the execution main body may perform normalization processing on the color value of the light source to obtain a normalized color value of the light source.

As an example, the execution subject described above may perform normalization processing on the color values of the light sources in the following manner:

RGB'= RGB / sum(RGB)

wherein, RGB' represents the color value of the light source after normalization, RGB represents the color value of the light source before normalization, and sum (RGB) result is the sum of the color value R, G, B and the color value.

305, converting the normalized illuminant color value into an illuminant color value in the first color space to obtain a converted illuminant color value.

In this embodiment, the execution body may convert the normalized light source color value into the light source color value in the first color space, so as to obtain the converted light source color value.

As an example, when the first color space is an XYZ space, the execution body may convert the normalized light source color value into the light source color value in the first color space in the following manner:

XYZ = CM1 * RGB'

where XYZ represents the converted illuminant color values, RGB' normalized illuminant color values, and CM1 represents a conversion matrix for converting the normalized illuminant color values to illuminant color values in the first color space.

And 306, calculating the color temperature corresponding to the converted light source color value.

In this embodiment, the execution subject may calculate the color temperature corresponding to the converted light source color value.

As an example, the execution subject may determine the color temperature corresponding to the converted light source color value according to a correspondence table established in advance.

Based on the color temperature, a fourth conversion matrix is determined 307.

In this embodiment, the execution subject may determine the fourth conversion matrix based on the color temperature calculated in 306. The fourth conversion matrix is used for converting the color space of the white balance processed image of the image in the first color space into the second color space.

As an example, the color temperature calculated in 306 may be input to a pre-trained matrix determination model, resulting in a fourth transformation matrix. The matrix determination model may be a convolutional neural network trained based on a training sample set including a sample color temperature and a sample fourth conversion matrix.

And 308, converting the color space of the first white balance processed image into a second color space based on a fourth conversion matrix to obtain a converted image.

In this embodiment, the executing body may convert the color space of the first white balance processed image into the second color space based on the fourth conversion matrix, to obtain the converted image.

As an example, the fourth conversion matrix and the first white balance processed image may be input to a conversion model trained in advance, resulting in a converted image. The conversion model may be a convolutional neural network obtained by training based on a training sample set including a sample fourth conversion matrix and a sample first white balance processed image.

309, performing light source estimation on the converted image to obtain a second light source estimation result.

And 310, performing illuminant estimation on the converted image to obtain a second illuminant estimation result.

In this embodiment, 301, 302, 303, 309, and 310 are substantially the same as 101, 102, 103, 105, and 106 in the corresponding embodiment of fig. 1, and are not described herein again.

In some optional implementations of this embodiment, the executing entity may determine the fourth transformation matrix based on the color temperature in the following manner, including:

and determining a result of multiplying the color space conversion matrix corresponding to the preset first color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being less than the preset first color temperature.

And determining a result of multiplying the color space conversion matrix corresponding to the preset second color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being equal to the preset second color temperature.

And determining a result of multiplying the color space conversion matrix corresponding to the preset third color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being greater than the preset third color temperature.

And responding to the color temperature larger than the preset second color temperature and smaller than or equal to the preset third color temperature, and executing the first operation. Wherein, the first operation may include the following steps one and three.

Step one, recalculating the converted light source color value.

And step two, taking the color temperature corresponding to the recalculated converted light source color value as a first target color temperature.

And step three, under the condition that the difference between the color temperature obtained by the last calculation and the color temperature obtained by the last calculation is smaller than a preset value, determining a fourth conversion matrix based on the size relation between the first target color temperature and the preset second color temperature and the preset third color temperature.

In response to the color temperature being greater than or equal to a preset first color temperature and less than a preset second color temperature, performing a second operation, wherein the second operation includes first to third steps of:

the first step, recalculate the converted illuminant color value.

And a second step of setting the color temperature corresponding to the recalculated converted light source color value as a second target color temperature.

And a third step of determining a fourth conversion matrix based on the magnitude relation between the second target color temperature and the preset first color temperature and the preset third color temperature under the condition that the difference between the color temperature obtained by the last calculation and the color temperature obtained by the last calculation is smaller than a preset value.

Optionally, in a case where the color temperature is greater than the preset second color temperature and less than or equal to the preset third color temperature, if a difference between the color temperature calculated last time and the color temperature calculated last time is greater than or equal to a preset value, the executing main body may continue to execute the first operation.

Optionally, the executing main body may continue to execute the second operation when the color temperature is greater than or equal to the preset first color temperature and less than the preset second color temperature.

It can be understood that, in the above alternative implementation manner, the fourth conversion matrix may be determined in different manners by determining a magnitude relationship between the color temperature, the preset first color temperature, the preset second color temperature, and the preset third color temperature, so that the accuracy of the white balance processing is further improved through subsequent steps.

As an example, please further refer to fig. 4, fig. 4 is a schematic diagram illustrating a generation process of a color temperature curve in an embodiment of the white balance processing method for an image according to the present disclosure. In the case that the value of the first coordinate of the target pixel is the ratio of the color value of the blue channel of the target pixel to the color value of the green channel of the target pixel, and the value of the second coordinate of the target pixel is the ratio of the color value of the red channel of the target pixel to the color value of the green channel of the target pixel, the generating process of the color temperature curve includes:

401, for each color temperature value in the predetermined set of color temperature values, taking a predetermined number of pictures for each standard color card in the predetermined standard color cards respectively using the color temperature value.

And 402, calculating values of a first coordinate and a second coordinate of a pixel point in the shot picture.

The value of the first coordinate of the pixel point is the ratio of the color value mean value of the blue channel of the pixel point to the color value mean value of the green channel; the value of the second coordinate of the pixel point is the ratio of the color value mean value of the red channel of the pixel point to the color value mean value of the green channel; the average value of the color values of the red channels of the pixel points is as follows: the average value of the color values of the red channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the green channels of the pixel points is as follows: the average value of the color values of the green channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the blue channels of the pixel points is as follows: and the average value of the color values of the blue channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value.

And 403, generating a preset color temperature curve based on the calculated values of the first coordinate and the second coordinate of the pixel point.

It can be understood that, with the color temperature curve generated by the method shown in fig. 4, the average value of the color values of the RGB three channels of each pixel point corresponding to the pixel in the predetermined number of pictures of the same standard color card with the same color temperature value can be eliminated, so that errors caused by different shooting positions, angles, and the like can be eliminated, and the accuracy of the color temperature curve can be improved.

It should also be understood that fig. 4 is only a schematic diagram of the generation process of the color temperature curve in one embodiment of the white balance processing method of the image of the present disclosure. Besides the color temperature curve generation method shown in fig. 4, the color temperature curve may be generated in other manners, for example, in a case that the value of the first coordinate of the target pixel is a ratio of the color value of the green channel of the target pixel to the color value of the blue channel of the target pixel, and the value of the second coordinate of the target pixel is a ratio of the color value of the green channel of the target pixel to the color value of the red channel of the target pixel, the color temperature curve generation process includes:

firstly, aiming at each color temperature value in a preset color temperature value set, a preset number of pictures are respectively shot for each standard color card in a predetermined standard color card by adopting the color temperature value.

And then, calculating the value of the first coordinate and the value of the second coordinate of the pixel point in the shot picture.

The value of the first coordinate of the pixel point is the ratio of the color value mean value of the green channel of the pixel point to the color value mean value of the blue channel; the value of the second coordinate of the pixel point is the ratio of the color value mean value of the green channel of the pixel point to the color value mean value of the red channel; the average value of the color values of the red channels of the pixel points is as follows: the average value of the color values of the red channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the green channels of the pixel points is as follows: the average value of the color values of the green channels of all the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value; the average value of the color values of the blue channels of the pixel points is as follows: and the average value of the color values of the blue channels of the pixel points corresponding to the pixel in the preset number of pictures of the same standard color card under the same color temperature value.

And finally, generating a preset color temperature curve based on the calculated values of the first coordinate and the second coordinate of the pixel point.

It should be noted that, besides the above-mentioned contents, the embodiments of the present disclosure may also include the same or similar features and effects as the embodiments corresponding to fig. 1 and/or fig. 2, and are not described herein again.

As an application scenario of an embodiment of the white balance processing method for an image of the present disclosure, the white balance processing method for an image may be applied to a VR (Virtual Reality) viewing scene. Typically, to complete an online VR house visit involves the integration of multiple technologies, and the final view presented to the user is a ring that must not be left to processing the photos (e.g., ISP processing). Although devices such as cameras and mobile phones have become popular and involve various related photo processing techniques, various challenges exist with respect to the VR studio customization device. Firstly, the room shooting device is a customized hardware with high integration of various sensor devices and can only be developed autonomously. Secondly, such devices are not usually available in as large a volume as mobile phones, and therefore, are difficult to support by mature ISP manufacturers, and therefore, require self-built ISP capabilities at the same time. Finally, the ISP processing is different from the processing methods of other ISP manufacturers because the method is only limited to the special scene of the shooting room. A general white balance algorithm has a high success rate in a natural scene, but since an indoor environment is represented by rich colors, a complex light source environment, a large-area monochromatic scene, and the like, a great challenge is brought to white balance. And because each link in the ISP processing process presents coupling effects of different degrees, under the complex indoor environment shooting condition, the mutual coupling effect is amplified, and the color recovery becomes more difficult.

The accuracy and robustness of image white balance processing can be improved, so that the method is suitable for being applied to VR house-watching scenes, for example, the method can be directly applied to ISP processing flows of indoor photographing scenes.

Specifically, the color temperature curve, that is, the predetermined color temperature curve described above, may be generated first by the method described above.

For example, standard 24 color cards having color temperatures of 2850, 3100, 3350, 3600, 3850, 4100, 4350, 4600, 4850, 5000, 5250, 5500, 5750, 6000, 6250, 6500 and 10 standard white cards, respectively, can be photographed. Then, the average values of the color values of the three channels of the pixels in the 10 images are respectively calculated to reduce the influence of noise, and finally, calibration data is generated. Besides the change of the color temperature, the calibration scene including the position of the color card cannot be changed, and the uniqueness of the variable is ensured.

Then, the position of the white gray area in the calibration data picture is selected according to the 24-color chart. For example, the middle 80% area of each of the 3 grays at the lower left corner of the 24 color chart can be selected as the range for calculating the white point.

And then, respectively calculating the average value of RGB three-channel data of the selected area for all the calibration data to obtain the color value average value RMean of the red channel, the color value average value GMean of the green channel and the color value average value BMean of the blue channel, and calculating color coordinates (x, y). Wherein x = BMean/GMean, y = rmeean/GMean.

And finally, obtaining a final color temperature calibration curve by using the coordinates of the calculated points.

The white balance processing procedure of the image is entered as follows:

firstly, through corrosion operation, setting image position data corresponding to the boundary data of the panoramic camera to be 0, namely eliminating the influence of the outer boundary on white balance processing. The original calibration boundary pixel is binarized and only includes 0 and 255, the valid pixel may be 255, and the invalid pixel may be 0. And then, carrying out corrosion operation by using a filter kernel with KSize of 71 to obtain an image to be processed.

Then, an automatic exposure process is performed.

Here, although the image to be processed has a high dynamic range, the numerical values of most data (for example, the color values of RGB three channels) are low, which is not beneficial to subsequent white balance and other operations, so that the image needs to be automatically exposed first, and the numerical values of most data are improved, so that the subsequent algorithm can be processed more accurately.

The exposure magnification for exposing the image to be processed may be determined by the method described above, and details are not repeated here. After determining the exposure magnification and carrying out overexposure processing, an exposed image can be obtained.

And then, dividing the image, collecting the image areas, and removing the image areas with the effective data smaller than a threshold value in the image area collection. Here, the number of divided blocks may be decided for different sizes of original images. For example, for a 2774 x 3708 size image, the entire image may be divided into 9 small pictures, i.e., 1/3 with the number of rows and columns being the original data.

For each image area, if the number of effective data (i.e. pixel points with channel values not all 0) is found to be less than half of the total pixels of a single image area, the target pixel of the image area is not calculated.

Subsequently, the target pixels for the remaining image area may be calculated using GrayIndex or other white balance algorithms. Thereby obtaining the first estimated target pixel (i.e. the first illuminant estimation result).

Before the algorithm is used for calculation, the over-bright data points with the single channel numerical value exceeding 245 and the over-dark data points with the sum of the color values of three channels being less than 10 in the original image can be removed, and the influence of dark noise and over-bright nonlinear areas on white balance can be avoided.

And then, taking the image area of each calculation target pixel as an image to be processed, wherein each image to be processed corresponds to one target pixel. Therefore, a CCM matrix is calculated by using the target pixel corresponding to each image to be processed, the image to be processed is processed to an sRGB color space, and a new estimated white point (namely the second light source estimation result) is obtained.

Specifically, CCM may be first calculated using the first estimated target pixel RGB for each image to be processed by:

firstly, calculating corresponding XYZ coordinates by using the RGB value of the target pixel estimated for the first time, calculating color temperature by using the XYZ coordinates, comparing the color temperature with three set standard color temperatures (2850,5000,6500), and reducing color temperature errors through iterative calculation to obtain final color temperature and CCM, wherein the details are as follows:

here, the following matrices (including: FM0, FM1, FM2, FM0, FM1, FM 2) can be obtained by calibration: FM0, CM0 at preset first color temperature t0= 2850; FM1, CM1 at preset second color temperature t1= 5000; FM2, CM2 at preset third color temperature t2= 6500. Wherein CM0, CM1, CM2 may convert data from the camera's color space to XYZ color space; FM0, FM1, FM2 may convert the camera color to the color of the human eye. The matrix M is a color space conversion matrix for converting XYZ color space into sRGB space

Secondly, normalizing the RGB of the target pixel estimated for the first time.

RGB'= RGB / sum(RGB)

Wherein, RGB' represents the color value of the light source after normalization, RGB represents the color value of the light source before normalization, and sum (RGB) is the sum of the color value R, G, B.

And thirdly, converting the normalized light source color value into a light source color value in the first color space to obtain a converted light source color value.

XYZ = CM1 * RGB'

wherein XYZ represents the converted illuminant color value and RGB' normalizes the illuminant color value.

Fourth, the color temperature temper in the current XYZ coordinates is calculated. Here, the calculation method is the same as the color temperature calculation method of the image file in the DNG format.

Then, performing classification calculation, and iteratively solving a final color temperature and CCM matrix:

class one, if temper equals t 1; CCM = FM1 × M;

class two, if temper is less than t 0; CCM = FM0 × M;

class three, if temper is greater than t 2; CCM = FM2 × M;

class four, if t1 < temper < t2: then calculate the ratio:

computing

：

;

Calculating a new color temperature temper 'from the most recent XYZ' calculated in this way until the difference from the last color temperature temper is less than 2, then there are 3 sub-categories:

subdivision class one, if temper > t 2; CCM = FM2 × M;

subdivision class two, if temper < t 1; CCM = FM1 × M;

subdivision three, if t1 < temper < t 2; then:

class three, if t0 < temper < t 1:

calculating a new color temperature temper 'from XYZ' until the difference from the last color temperature temper is less than 2, then there are 3 sub-categories:

subdivision class one, if temper < t 0; CCM = FM0 × M;

subdivision class two, if temper > t 1; CCM = FM1 × M;

subdivision three, if t1 < temper < t 2; then:

at this point, the matrix CCM may be determined.

Then, white balance processing can be carried out on the small picture by utilizing the first estimated target pixel of each image to be processed to obtain a first white balance processed image, then CCM color space conversion is carried out on the white balance processed image to obtain a converted image, and then the pixel position of the estimated white point of the converted image is obtained by utilizing the image to be processed to obtain a second estimated target pixel. Here, the color cast limiting matrix may be determined in the manner as described above, and then the target pixel corresponding to the image to be processed is obtained.

After obtaining the target pixel corresponding to each image to be processed, the final white point coordinate of the whole image with weighted average can be calculated:

B/G, R/G of each target pixel retained above are calculated as xi, yi, respectively. Wherein B represents the color value of the blue channel of the target pixel, G represents the color value of the green channel of the target pixel, R represents the color value of the red channel of the target pixel, and i is used for uniquely identifying each target pixel. Here, the coordinates of the target pixel may be denoted as (xi, yi). Then, the distance di from the target pixel (xi, yi) to the color temperature curve obtained by calibration is calculated, and the corresponding weight Wi of each target pixel may be:

the color value R of the red channel, the color value G of the green channel, and the color value B of the blue channel of the target pixel (R, G, B) may be determined by the above formula, and are not described here again:

and finally, carrying out white balance processing on the picture.

I’ =I*invRGB

Wherein I' represents the white-balanced processed image, I represents the original image, I represents the convolution operation,

。

it is to be understood that, in the above example, the influence of the boundary data can be removed by the erosion operation, the influence of the nonlinear data can be removed by the exposure processing, and the influence of the dark noise can be removed by determining the target image area. By utilizing a blocking scheme and a weighted average scheme, the white point of the whole picture is more robust and more accurate; the color conversion effect generated by the CCM matrix can be reduced: the method comprises the steps of calculating a feedback calibration matrix of the light source fed back to the former light source by estimating the light source of the native XYZ color space and the sRGB color space after the common CCM conversion, and performing final white balance processing on an original picture by using the matrix and the white point of the native XYZ space, so that the accuracy of the white balance processing is improved.

As can be seen from fig. 3, in the flow 300 of the image white balance processing method in the present embodiment, by calculating the color temperature corresponding to the converted light source color value, the target pixel in the image after white balance processing can be more accurate.

Referring now further to fig. 5, as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of an apparatus for white balance processing of an image, the apparatus embodiment corresponds to the method embodiment shown in fig. 1-4, and the apparatus embodiment may include the same or corresponding features as the method embodiment shown in fig. 1-4 in addition to the features described below, and produce the same or corresponding effects as the method embodiment shown in fig. 1-4. The device can be applied to various electronic equipment.

As shown in fig. 5, the white balance processing apparatus 500 of the image of the present embodiment includes: an obtaining unit 501 configured to obtain an image to be processed in a first color space; a first light source estimation unit 502 configured to perform light source estimation on an image to be processed to obtain a first light source estimation result; a first generation unit 503 configured to perform first white balance processing on the image to be processed based on the first light source estimation result, and generate a first white-balance-processed image; a conversion unit 504 configured to convert the color space of the first white balance processed image into a second color space, resulting in a converted image; a second light source estimation unit 505 configured to perform light source estimation on the converted image to obtain a second light source estimation result; and a second generating unit 506 configured to perform second white balance processing on the image to be processed based on the first illuminant estimation result and the second illuminant estimation result, and generate a second white-balance-processed image.

In this embodiment, the acquiring unit 501 of the image white balance processing apparatus 500 may acquire an image to be processed in the first color space.

In this embodiment, the first illuminant estimation unit 502 can perform illuminant estimation on the to-be-processed image acquired by the acquisition unit 501 to obtain a first illuminant estimation result.

In this embodiment, the first generation unit 503 may perform white balance processing on the image to be processed acquired by the acquisition unit 501 based on the first light source estimation result obtained by the first light source estimation unit 502 to generate a first white-balance-processed image.

In this embodiment, the conversion unit 504 may convert the color space of the first white balance processed image generated by the first generation unit 503 into the second color space, resulting in a converted image.

In this embodiment, the second illuminant estimation unit 505 may perform illuminant estimation on the converted image obtained by the conversion unit 504 to obtain a second illuminant estimation result.

In this embodiment, the second generating unit 506 may perform the second white balance processing on the image to be processed acquired by the acquiring unit 501 based on the first illuminant estimation result obtained by the first illuminant estimation unit 502 and the second illuminant estimation result obtained by the second illuminant estimation unit 505, and generate the second white-balance-processed image.

In some optional implementations of this embodiment, the second generating unit is further configured to:

determining a first conversion matrix based on the first light source estimation result, wherein the first conversion matrix is used for converting the image to be processed into a white balance processed image of the image to be processed;

determining a second conversion matrix based on the second light source estimation result, wherein the converted image of the second conversion matrix is converted into a white balance processed image of the converted image;

and performing third white balance processing on the image to be processed based on the first conversion matrix and the second conversion matrix to generate a second white-balance-processed image.

In some optional implementations of the present embodiment, the second generating unit 506 is further configured to:

and performing fourth white balance processing on the image to be processed based on the first conversion matrix, the second conversion matrix and the third conversion matrix to generate a second white-balance-processed image, wherein the third conversion matrix is used for converting the image in the first color space into the image in the second color space.

determining a color cast limit matrix of the first conversion matrix based on the first conversion matrix, the second conversion matrix and the third conversion matrix;

In some optional implementations of this embodiment, the first illuminant estimation result includes a color value of the illuminant; and

the above-mentioned conversion unit 504 is further configured to:

converting the normalized light source color value into a light source color value in a first color space to obtain a converted light source color value;

and converting the color space of the first white balance processed image into a second color space based on a fourth conversion matrix to obtain a converted image.

In some optional implementations of the present embodiment, the converting unit 504 is further configured to:

determining a result of multiplying a color space conversion matrix corresponding to the preset first color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being less than the preset first color temperature;

determining a result of multiplying a color space conversion matrix corresponding to the preset second color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being equal to the preset second color temperature;

determining a result of multiplying the color space conversion matrix corresponding to the preset third color temperature by the third conversion matrix as a fourth conversion matrix in response to the color temperature being greater than the preset third color temperature;

recalculating the converted light source color value;

responding to the difference between the color temperature obtained by the last calculation and the color temperature obtained by the last calculation being smaller than a preset value, and determining a fourth conversion matrix based on the size relation between the first target color temperature and a preset second color temperature and a preset third color temperature;

recalculating the converted light source color value;

and under the condition that the color temperature is greater than the preset second color temperature and less than or equal to the preset third color temperature, responding to the difference between the color temperature obtained by the last calculation and the color temperature obtained by the last calculation being greater than or equal to a preset value, and continuing to execute a first operation:

and continuing to execute the second operation under the condition that the color temperature is greater than or equal to the preset first color temperature and less than the preset second color temperature.

In some optional implementations of this embodiment, the obtaining unit 501 is further configured to:

acquiring an original image;

determining an exposure magnification for exposure processing based on a gray image corresponding to an image to be processed;

carrying out exposure processing on an image to be processed by adopting an exposure multiplying power to generate a first exposed image;

carrying out exposure processing on the image to be processed by adopting the selected exposure multiplying power to generate a second post-exposure image of the image to be processed aiming at the exposure multiplying power;

determining whether the mean value of the brightness of pixels in the gray level image of the exposure magnification of the image to be processed is greater than or equal to a preset brightness threshold value;

and in response to the fact that the mean value of the brightness of the pixels in the gray-scale image of the exposure magnification of the image to be processed is larger than or equal to a preset brightness threshold value, taking the adopted exposure magnification as the exposure magnification for exposure processing.

In some optional implementations of this embodiment, the apparatus 500 further includes:

and a first selecting unit (not shown in the figure) configured to continue to select the exposure magnification from the set of exposure magnifications in descending order in response to the luminance average value of the to-be-processed image for the pixels in the grayscale image of the exposure magnification being less than a preset luminance threshold, and perform a first exposure magnification determining operation based on the selected exposure magnification.

and a second selecting unit (not shown in the figure) configured to reselect an unselected exposure magnification from the exposure magnification interval in response to a luminance average value of the to-be-processed image for pixels in the grayscale image of the exposure magnification being less than a preset luminance threshold, and perform a second exposure magnification determination operation based on the reselected exposure magnification.

taking pixels in an image area of the image to be processed, which is corroded relative to the original image, as invalid pixels; taking the image pixels of the image to be processed, which are not corroded relative to the original image, as effective pixels; and

taking at least one image area in the image area set as an image to be processed, comprising:

and taking a target image area in the image area set as an image to be processed, wherein the product of the number of all pixels in the target image area and the preset percentage is less than or equal to the number of effective pixels in the target image area.

and performing seventh white balance processing on the image to be processed based on the determined distance between the target pixel and the preset color temperature curve to generate a second white-balance-processed image.

determining the weight corresponding to each determined target pixel based on the distance from each determined target pixel to a preset color temperature curve;

In some optional implementations of this embodiment, the value of the first coordinate of the target pixel is a ratio of a color value of a blue channel of the target pixel to a color value of a green channel of the target pixel, and the value of the second coordinate of the target pixel is a ratio of a color value of a red channel of the target pixel to a color value of a green channel of the target pixel; and

the apparatus 500 further comprises:

a photographing unit (not shown in the figure) configured to take a predetermined number of pictures for each of predetermined standard color cards respectively using a color temperature value for each of a predetermined set of color temperature values;

a calculation unit (not shown in the figure) configured to calculate a value of a first coordinate and a value of a second coordinate of a pixel in a captured picture; the value of the first coordinate of the pixel is the ratio of the color value mean value of the blue channel of the pixel to the color value mean value of the green channel; the value of the second coordinate of the pixel is the ratio of the color value mean value of the red channel of the pixel to the color value mean value of the green channel; the color value average of the red channel of a pixel is: the average value of the color values of the red channels of the pixels corresponding to the pixels in the preset number of pictures of the same standard color card under the same color temperature value; the color value average of the green channel of a pixel is: the average value of the color values of the green channels of the pixels corresponding to the pixels in the preset number of pictures of the same standard color card under the same color temperature value; the color value average of the blue channel of a pixel is: the average value of the color values of the blue channels of the pixels corresponding to the pixels in the preset number of pictures of the same standard color card under the same color temperature value;

a third generating unit (not shown in the figure) configured to generate a predetermined color temperature curve based on the calculated values of the first and second coordinates of the pixel.

In the white balance processing apparatus 500 for an image provided in the above embodiment of the present disclosure, the obtaining unit 501 may obtain an image to be processed in a first color space, then the first illuminant estimation unit 502 may perform illuminant estimation on the image to be processed to obtain a first illuminant estimation result, then the first generating unit 503 may perform first white balance processing on the image to be processed based on the first illuminant estimation result to generate a first white-balanced processed image, next, the converting unit 504 may convert the color space of the first white-balanced processed image into a second color space to obtain a converted image, then, the second illuminant estimation unit 505 may perform illuminant estimation on the converted image to obtain a second illuminant estimation result, and finally, the second generating unit 506 may perform second white balance processing on the image to be processed based on the first illuminant estimation result and the second illuminant estimation result, and generating a second white balance processed image. The embodiment of the disclosure can combine the first illuminant estimation result and the second illuminant estimation result to realize white balance processing, thereby improving the accuracy of the image white balance processing.

Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 6. The electronic device may be either or both of the first device and the second device, or a stand-alone device separate from them, which stand-alone device may communicate with the first device and the second device to receive the acquired input signals therefrom.

FIG. 6 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.

As shown in fig. 6, the electronic device 6 includes one or more processors 601 and memory 602.

The processor 601 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

Memory 602 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer readable storage medium and executed by the processor 601 to implement the white balance processing method of the image of the various embodiments of the present disclosure described above and/or other desired functions. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device may further include: an input device 603 and an output device 604, which are interconnected by a bus system and/or other form of connection mechanism (not shown).

For example, when the electronic device is a first device or a second device, the input device 603 may be the microphone or the microphone array described above for capturing the input signal of the sound source. When the electronic device is a stand-alone device, the input means 603 may be a communication network connector for receiving the acquired input signals from the first device and the second device.

The input device 603 may also include, for example, a keyboard, a mouse, and the like. The output device 604 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 604 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, among others.

Of course, for simplicity, only some of the components of the electronic device relevant to the present disclosure are shown in fig. 6, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device may include any other suitable components, depending on the particular application.

In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the method of white balance processing of images according to various embodiments of the present disclosure described in the "exemplary methods" section of this specification above.

The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.

The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Additionally, embodiments of the present disclosure may also be a computer program, which may include computer readable code. When the above computer readable code is run on a device, a processor in the device performs the steps in the method of white balance processing of an image according to various embodiments of the present disclosure described in the above "exemplary methods" section of this specification.

The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.

In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts in the embodiments are referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.

Those of ordinary skill in the art will understand that: all or part of the steps for implementing the method embodiments may be implemented by hardware related to program instructions, and the program may be stored in a computer readable storage medium, and when executed, the program performs the steps including the method embodiments; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

The methods and apparatus of the present disclosure may be implemented in a number of ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order for the steps of the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless specifically stated otherwise. Further, in some embodiments, the present disclosure may also be embodied as programs recorded in a recording medium, the programs including machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

The description of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to practitioners skilled in this art. The embodiment was chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.

Claims

1. A method of white balance processing of an image, the method comprising:

acquiring an image to be processed in a first color space;

2. The method according to claim 1, wherein performing a second white balance process on the image to be processed based on the first illuminant estimation result and the second illuminant estimation result to generate a second white-balanced processed image comprises:

3. The method according to claim 2, wherein performing a third white balance process on the image to be processed based on the first conversion matrix and the second conversion matrix to generate a second white-balance-processed image comprises:

4. The method according to claim 3, wherein performing a fourth white balance processing on the image to be processed based on the first conversion matrix, the second conversion matrix, and the third conversion matrix to generate a second white-balanced image comprises:

5. The method of any of claims 3-4, wherein the first illuminant estimation result comprises a color value of an illuminant; and

6. The method of claim 5, wherein determining a fourth transformation matrix based on the color temperature comprises:

recalculating the converted light source color value;

7. The method of claim 6, wherein determining a fourth transformation matrix based on the color temperature further comprises at least one of:

8. An apparatus for white balance processing of an image, the apparatus comprising:

9. An electronic device, comprising:

a memory for storing a computer program;

a processor for executing a computer program stored in the memory, and when executed, implementing the method of any of the preceding claims 1-7.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of the preceding claims 1 to 7.