WO2023122860A1 - Image processing method and apparatus, image acquisition device, and storage medium - Google Patents

Abstract

An image processing method and apparatus, an image acquisition device, and a storage medium. The method comprises: obtaining an image to be processed (S102); determining scene type information of pixel blocks in said image (S104); and on the basis of the scene type information of the pixel blocks, determining a target white balance gain, so as to perform white balance correction on said image by using the target white balance gain (S106). According to the method, it is assisted to determine a white balance gain by using the scene type information of the pixel blocks, so that a more accurate white balance gain can be obtained, and a more ideal white balance processing effect can further be obtained.

Description

Image processing method, device, image acquisition device and storage medium technical field

The present application relates to the technical field of image processing, and in particular, to an image processing method, device, image acquisition device, and storage medium.

Background technique

When white balance processing is performed on an image, a white balance gain may be determined based on RGB values of white points in the image, and then white balance processing is performed on the image by using the white balance gain. Therefore, accurately determining the white balance gain is the key to improving the effect of image white balance processing. In the related art, the white point and the white balance gain are usually determined only according to the RGB values of the pixels in the image, and the white balance gain obtained in this way is not accurate enough, resulting in poor image white balance processing effect.

Contents of the invention

In view of this, the present application provides an image processing method, device, image acquisition device and storage medium.

According to a first aspect of the present application, an image processing method is provided, the method comprising:

Get the image to be processed;

determining scene category information of pixel blocks in the image to be processed;

Determine a target white balance gain based on the scene category information of the pixel block, so as to perform white balance correction on the image to be processed by using the target white balance gain.

According to the second aspect of the present application, there is provided an image processing device, the device includes a processor, a memory, and a computer program stored in the memory that can be executed by the processor, and the processor executes the computer program The following steps can be implemented:

Get the image to be processed;

determining scene category information of pixel blocks in the image to be processed;

Determine a target white balance gain based on the scene category information of the pixel block, so as to use the target white balance gain to perform white balance correction on the image to be processed.

According to a third aspect of the present application, an image acquisition device is provided, and the image acquisition device includes the image processing apparatus mentioned in the second aspect above.

According to a fourth aspect of the present application, a computer-readable storage medium is provided, and a computer program is stored on the computer-readable storage medium, and when the computer program is executed, the method mentioned in the above-mentioned first aspect is implemented.

Applying the scheme provided by this application, when determining the white balance gain for white balance correction of the image, considering that the white balance gain determined only based on the RGB value of the image is not accurate enough, it is possible to first determine the pixel block in the image to be processed The scene category information of each pixel block is combined with the scene category information to assist in determining the white balance gain, so that a more accurate white balance gain can be obtained, and a more ideal white balance processing effect can be obtained.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

Fig. 1 is a flowchart of an image processing method according to an embodiment of the present application.

Fig. 2 is a schematic diagram of an image processing method according to an embodiment of the present application.

Fig. 3 is a schematic diagram of a logical structure of an image processing device according to an embodiment of the present application.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

When white balance processing is performed on an image, a white balance gain may be determined based on RGB values of white points in the image, and then white balance processing is performed on the image by using the white balance gain. Accurate determination of white balance gain is the key to improve the effect of image white balance processing. In related technologies, the white point in the image is usually determined only according to the RGB values of the image, and then the white balance gain is determined. The white balance gain obtained in this way is often not accurate enough.

For example, the principle of white balance correction is to find the white objects in the image, and then restore the color of these objects to white. However, it is often impossible to accurately determine the white object in the image only based on the RGB value of the image. For example, for pixel blocks with similar RGB values, they may be different scenes. For example, the RGB values of white objects and yellow objects under yellow light sources are very different. Similar, but one of the two is white and the other is non-white, so it is easy to misjudge when determining the white point. In addition, for pixel blocks with similar RGB values, in indoor scenes and outdoor scenes, the probability of this pixel block being a white object is It is different, or in a rainy day scene or a cloudy day scene, the probability of being a white object is also different. Therefore, if the white balance gain is determined only according to the RGB value of the image block, some pixels that are not white objects may be mistakenly determined as white points, or the pixels corresponding to white objects may be misjudged as non-white points, resulting in a certain white The balance gain is not accurate enough, and the image effect after white balance processing is not ideal.

Based on this, the embodiment of the present application provides an image processing method. When determining the white balance gain used for white balance correction on the image, considering that the white balance gain determined only based on the RGB values of the image is not accurate enough, it can first Determine the scene category information of each pixel block in the image to be processed, and then combine the scene category information of each pixel block to assist in determining the white balance gain, so that a more accurate white balance gain can be obtained, and a more ideal white balance processing effect can be obtained.

The image processing method in the embodiment of the present application can be executed by the image acquisition device, for example, it can be executed by the ISP chip in the image acquisition device, or the image can be collected by the image acquisition device and sent to other devices. Process software execution.

As shown in Figure 1, the flow chart of the image processing method provided by the embodiment of the present application may specifically include the following steps:

S102. Obtain an image to be processed;

In step S102, images to be processed may be acquired, where the images to be processed may be various images that require white balance correction processing.

S104. Determine scene category information of pixel blocks in the image to be processed;

In step S104, the scene category information of the pixel block in the image to be processed can be determined, wherein the scene category information can be various information indicating the scene category to which the pixel block belongs, and the scene category can classify the scene in various ways get. For example, the scene category information may be information indicating which object category the pixel block specifically belongs to, for example, it may be information indicating that the pixel block is of various categories such as green plants, sky, and ground. Alternatively, the scene category information may also be information indicating the scene category to which the entire image to be processed belongs, for example, indicating whether the image to be processed is an indoor scene or an outdoor scene, whether it is a sunny scene or a rainy scene, and the like. It is not difficult to understand that any scene category information that helps to assist in determining the white balance gain of an image falls within the scope of protection of the embodiments of the present application.

Wherein, the pixel block in the image to be processed may be a pixel block obtained by dividing the image to be processed according to a certain size, for example, the pixel block to be processed may be divided into 32×32 pixel blocks, or pixel blocks of other sizes, specifically according to Actual demand setting. Determining the scene category information of the pixel blocks may be to determine the scene category information of all the pixel blocks in the image to be processed, or only determine the scene category information of some pixel blocks. For example, for some pixel blocks, it can be accurately determined based on RGB values whether they are belong to white points, there is no need to determine the scene category information of these pixel blocks.

There are many ways to determine the scene category information of the pixel block. For example, a pre-trained neural network can be used to perform semantic segmentation on the image, or other ways can be used, which are not limited in the embodiment of the present application.

S106. Determine a target white balance gain based on the scene category information of the pixel block, so as to perform white balance correction on the image to be processed by using the target white balance gain.

In step S106, after the scene category information of the pixel block is determined, the scene category information of the pixel block may be used to assist in determining the target white balance gain. For example, it can be combined with the scene category information of the pixel block to determine whether the pixel point of the pixel block is a white point, and then determine the target white balance gain based on the determined white point, or, after determining the white point in the image, further combine the pixel Determine the weight corresponding to each white point based on the scene category information of the block, and determine the target white balance gain by combining the weight corresponding to the white point and the RGB value, or adjust the white balance gain based on the scene category information of the pixel block after determining the white balance gain , to obtain the final target white balance gain, and then use the determined target white balance gain to perform white balance correction on the image to be processed.

In the embodiment of the present application, by determining the scene category information of the pixel blocks in the image, the white balance gain can be determined based on the scene category information, so that the determined white balance gain is more accurate, and a better white balance processing effect can be achieved.

In some embodiments, the scene category information may include the category of the target object to which the pixel block belongs, for example, the specific category of the pixel block is sky, grass, road, vehicle, person, etc., combined with the target object category to which the pixel block belongs, can be It assists in determining whether the pixel block is a white point, or the probability of the pixel block being a white point, and then can accurately determine the white balance gain.

In some embodiments, the scene category information may be information indicating that the image to be processed is an indoor scene or an outdoor scene. When determining the white balance gain, when the image to be processed is an indoor scene or an outdoor scene, it will also have a certain impact on the determination of the white balance gain. For example, if it is the same white point, its brightness will be larger in the outdoor scene , in the indoor scene, its brightness is darker, so for the same RGB pixel, the probability of being a white point is different in the outdoor scene and the indoor scene. Therefore, the white point in the pixel block can be accurately determined based on the information indicating that the image to be processed is an indoor scene or an outdoor scene, or the information can be used to adjust the white balance gain to obtain an accurate white balance gain.

Since there are many object categories, when determining the target object category to which the pixel block belongs, we hope that the determined target object category can help determine the white balance gain, that is, these object categories have an impact on the white balance gain, and can be used to assist in determining the white balance gain. Balance gain. Therefore, in some embodiments, multiple object categories can be preset, and then the pixel block is determined to be a specific category in the preset multiple object categories. For example, the multiple object categories can be the following categories: sky, building, Green plants, soil, roads, floors, others. When setting these multiple object categories, you can set them based on the following principles:

(1) The preset object category can be used to determine whether the image to be processed is an indoor scene or an outdoor scene. For example, the preset object category can be the sky (can be determined to be an outdoor scene), the floor (can be determined to be an indoor scene), or other Other categories that can be used to determine indoor or outdoor scenes. Since indoor and outdoor scenes have an impact on the determination of the white balance gain, determining the target white balance gain can be assisted based on whether the pixel blocks belong to these categories.

(2) The pre-set object category can be used to determine the white point in the image to be processed. For example, accurately determining the white point is a prerequisite for accurately determining the white balance gain. If the white point can be accurately determined in combination with the target object category to which the pixel block belongs, it is bound to Will improve the accuracy of the determined white balance gain. Therefore, the preset object category is a category that can assist in determining whether a pixel is a white point, for example, the sun, if the pixel block is determined to be the sun, then the pixel in the pixel block is a white point.

(3) There is an overlapping area between the color interval of the scene included in the preset object category and at least one color interval of other scenes except the scene. Since there are overlapping color intervals corresponding to two objects, for example, a yellow object and a white object under a yellow light source, a white object under a green plant and a fluorescent light source, and a white object under the sky and a blue light source, etc., these objects correspond to There are overlaps in the color intervals, so it is impossible to determine whether it is a white point based only on the RGB value. If it can be determined whether the pixel block belongs to these categories, it can assist in determining the white point, or assist in determining the white balance gain, so that an accurate white can be obtained. Balance gain. For example, these object categories can be categories such as sky, green plants, snow scenes, etc. that will be confused with other scenes.

In some embodiments, when determining the target object category to which the pixel block belongs, the pixel block can be respectively determined as the target confidence level of each object category in the preset plurality of object categories, and then based on the target confidence level, the A target object category to which the pixel block belongs is determined among the plurality of object categories. For example, the preset multiple object categories are sky, building, green plant, soil, road, floor, and others, so it is possible to determine the target confidence of each pixel block for each of the above categories, and then determine the pixel block based on the target confidence The target object class to which it belongs.

For example, in some embodiments, the object category with the highest target confidence may be directly used as the target object category to which the pixel block belongs. Alternatively, in some embodiments, the target object category to which the pixel block belongs may also be determined based on the target confidence of each pixel block for each of the above categories and the target object categories to which the adjacent pixel blocks of the pixel block belong. For example, if the confidence that the pixel block belongs to the sky is 80%, and the confidence that it belongs to the road is 85%, and its adjacent pixel blocks are all sky, then the pixel block is considered to belong to the sky.

In some embodiments, when determining the target confidence that the pixel block is each object category in multiple object categories, for each object category, the initial confidence that the pixel block is the object category can be determined respectively, for example, Determine the initial confidence that the pixel block is the object category based on the pre-trained neural network, or determine the initial confidence in other ways, and then directly use the initial confidence as the target confidence.

In some embodiments, the initial confidence determined based on the neural network or other methods may not be accurate. Therefore, after the initial confidence is determined, the initial confidence can be further adjusted based on the information related to the pixel value of the pixel block itself, In order to get a more accurate target confidence.

In some embodiments, the pixel value-related information may include one or more of the following: the brightness of the pixel block, the first proportion of the first specified pixel area in the pixel block, wherein the RGB value of the first specified pixel area fall within the color range corresponding to the scene category. For example, for each object category, the brightness of the object category can be pre-calibrated. For example, the brightness of the sky is A. When the brightness of the pixel block is closer to A, the confidence that the pixel block is the sky is greater, so it can be based on The degree of proximity adjusts the initial confidence to obtain a more accurate target confidence.

For another example, for each object category, the color interval corresponding to each object category can be determined, for example, the color interval of each object category can be pre-marked, or can also be obtained based on other methods. Assuming that the color interval of the sky is determined to be B, then, based on the first ratio of the first specified pixel area in the pixel block that falls into the color interval in the entire pixel block, the larger the first ratio, it means that the pixel block is The greater the confidence of the sky, the initial confidence can be adjusted based on the size of the first proportion to obtain a more accurate target confidence.

In some embodiments, the target confidence is directly related to how close the brightness of the pixel block is to the nominal brightness of the object class. When the luminance of the pixel block is closer to the calibrated luminance, it means that the confidence of the pixel block is the object category is greater.

In some embodiments, the target confidence is positively related to the first proportion, and the larger the first proportion, the greater the confidence that the pixel block belongs to the object category.

Of course, in some embodiments, if the brightness of the pixel block is very small, for example, the brightness of the pixel block is less than the preset brightness threshold, then the initial confidence determined based on the neural network is inaccurate, so the target confidence can be adjusted is 0.

In some embodiments, the information indicating that the image to be processed is an indoor scene or an outdoor scene may be one or more of a confidence level that the image to be processed is an indoor scene, and a confidence level that the image to be processed is an outdoor scene.

In some embodiments, the confidence degree of the image to be processed is an indoor scene or the confidence degree of an outdoor scene can be determined in the following manner: First, the object belonging to each target object category can be determined based on the target object category to which each pixel block in the image to be processed belongs. The second ratio of all pixel blocks in the image to be processed is determined based on the second ratio and the weight of each target object category for the indoor scene to determine the confidence that the image to be processed is an indoor scene. Similarly, the confidence that the image to be processed is an outdoor scene may be determined based on the second proportion and the weights of each target object category for the outdoor scene.

For example, suppose the image to be processed includes 100 pixel blocks of the same size, among which, 50 pixel blocks belong to the target object category of the sky, 30 pixel blocks belong to the target object category of green plants, and 10 pixel blocks belong to The target object category of is road, and the target object category to which the 10 pixel blocks belong is other. Therefore, it can be determined that the second proportion of the sky area in the image to be processed is 50%, the second proportion of the green plant area is 30%, the second proportion of the road area is 10%, and the second proportion of other types of areas 10%. Then the weight of each scene relative to the indoor scene and the outdoor scene can be determined. Assume that the weight of the sky is 20% for the indoor scene, 80% for the outdoor scene, 40% for the indoor scene, and 40% for the outdoor scene. 60%, etc., and then based on the second ratio and weight, the confidence level that the image to be processed is an indoor scene, for example, 0.2, and the confidence level for an outdoor scene, such as 0.8, can be determined.

Wherein, the weight of each target object category for the indoor scene or the weight of each target object category for the indoor scene can be preset, or, in some embodiments, in order to obtain a more accurate weight, it can also be based on all objects belonging to each target object category. One or more of the brightness of the pixel block and the type of each target object category are determined.

In some embodiments, when determining the second ratio of all pixel blocks belonging to each target object category in the image to be processed, if it is determined based on only one frame of the image to be processed, due to reasons such as noise in the image, the determined result Might not be very accurate. Therefore, the second proportion may be determined in combination with multiple frames of images collected before and after the image to be processed. For example, multiple frames of target images collected before and/or after the image to be processed can be obtained, the proportion of all pixel blocks belonging to each target object category in the multiple frames of target images, and the proportion of all pixel blocks belonging to each target object category in the image to be processed The proportion of the pixel block is filtered to obtain the second proportion. For example, determine the proportion of the sky in the multi-frame target image and the proportion of the sky in the image to be processed, and then filter the proportion of the sky in these images, for example, calculate the average or weighted average, and obtain the final second proportion Compare. Similar methods can also be used for other categories.

In some embodiments, when determining the target white balance gain based on the scene category information of the pixel block, the target weight of the white point in the pixel block can be determined first based on the target object category to which the pixel block belongs, and then based on each white point in the image to be processed The RGB values of the points and the respective target weights determine the target white balance gain. For example, based on the RGB values of the pixels in the image to be processed, the white point in the image can be initially determined, and then the weight corresponding to the white point can be further combined with the scene category information of each pixel block. For example, if it is determined that the category to which the pixel block belongs is The sky, because the color temperature of the sky is high, when it is regarded as a white point, the color temperature of the final result will be raised, resulting in a yellowish color, so the weight of the white point in this pixel block can be appropriately reduced. When calculating the white balance gain based on the RGB of the white point, weighted average processing may be performed on the white balance gains determined by each white point in combination with the weight, so as to obtain the final target white balance gain.

In some embodiments, the scene category information includes the target object category to which the pixel block belongs. When the target object category to which the pixel block belongs is different, the target weights corresponding to the white points in the pixel block are also different. In some implementations, the scene category information may also include information indicating that the image to be processed is an indoor scene or an outdoor scene. When the image to be processed is an indoor scene or an outdoor scene, the white point of each pixel block in the image to be processed The target weights are also different.

In some embodiments, when determining the target weight of the white point in the pixel block based on the scene category information of the pixel block, the initial weight of the white point in the pixel block can be determined first, and then based on the scene category information of the pixel block, such as , the target object category to which the pixel block belongs, whether the image to be processed is an indoor scene or an outdoor scene etc. adjust the initial weights to obtain the target weight of the white point in the pixel block.

For example, in some embodiments, when determining the initial weight of the white point, the initial weight of the white point in the pixel block may be determined based on one or more of the brightness of the pixel block and the color temperature corresponding to the pixel block, wherein, The corresponding relationship between the RGB value and the color temperature can be calibrated in advance, and then the color temperature corresponding to the pixel block can be determined according to the RGB value of the pixel block.

In order to obtain a better white balance processing effect, when the images to be processed belong to indoor scenes and outdoor scenes respectively, the target weights corresponding to the white points in the same pixel block can be set differently. Therefore, in some embodiments, the scene category information may include the confidence level that the image to be processed is an indoor scene, and/or the confidence level that the image to be processed is an outdoor scene. process to obtain the final target white balance gain. For example, when the image to be processed is an outdoor scene, the first target weight of the white point in the pixel block can be determined, and then the first white balance gain is determined according to the RGB value of the white point in the entire image to be processed and the first target weight, and at the same time When the image to be processed is an indoor scene, the second target weight of the white point in the pixel block can be determined, and then the second white balance gain can be determined according to the RGB value of the white point in the entire image to be processed and the second target weight. Then, based on the confidence that the image to be processed is an indoor scene and the confidence that the image to be processed is an outdoor scene, the first white balance gain and the second white balance gain may be weighted to obtain a target white balance gain. By distinguishing the indoor scene from the outdoor scene, the white balance gains in the two scenes are obtained by two-way processing, and then a comprehensive white balance gain can be obtained based on the confidence that the image to be processed belongs to the above two scenes, so that the final determination The white balance gain is more accurate.

In the related art, when determining the white balance gain, it does not take into account that the color intervals of some scenes are relatively close, and there is a problem of confusion. For example, scenes such as yellow objects and white objects under yellow light, blue objects and white objects under blue light, green plants and white objects under fluorescent light sources, snow scenes and blue sky are easy to confuse due to their close color intervals. As a result, it is impossible to determine the real scene category of the pixel block, which in turn leads to inaccurate white balance gain determination.

In order to solve the above problems, after determining the white balance gain of the image to be processed, the determined white balance gain can be further compensated based on the scene category information of the pixel block, so as to eliminate the influence of the white balance gain on the white balance gain caused by the scene confusion caused by the close color range . For example, when determining the target white balance gain based on the scene category information of the pixel block, the initial white balance gain of the image to be processed can be determined first, and then for each pixel block, it is determined whether the RGB value of the pixel block falls into the color of the confused color Interval, where the color interval of the confusing color can be set in advance, for example, common confusing scenes, and the color interval of these scenes can be predetermined, if the RGB value of the pixel block falls into these color intervals, it means that the pixel block belongs to There is a possibility of confusion in the category of the scene. Then, the initial white balance gain of the pixel block can be compensated based on the scene category information of the pixel block to obtain the target white balance gain of the pixel block, and then the white balance correction process of the pixel block can be performed using the target white balance gain.

In some embodiments, the initial white balance gain is compensated based on the scene category information of the pixel blocks. When the target white balance gain is obtained, the compensation coefficient can be determined based on the scene category information of each pixel block, and then the compensation coefficient can be used to correct the initial white balance gain. The gain is compensated to obtain the target white balance gain corresponding to the pixel block. For example, suppose the RGB value of a certain pixel block falls within the color interval of a white object under a fluorescent lamp. Since the object in this color interval may be a white object or a green plant, it can be combined with the The target scene category determines whether the pixel block is a green plant, and then adjusts the initial white balance gain based on whether it is a green plant to obtain the target white balance gain. Wherein, the compensation coefficient can be set based on the actual situation, so that the final target white balance gain is more accurate, and the image after white balance correction is closer to the real color.

In some embodiments, the compensation coefficient can be determined based on the following method: the third proportion of the second designated area falling into the color interval of the confused color in the pixel block can be determined first, for example, it can be based on the RGB values of each area in the pixel block Determine whether the area falls into the color interval of the confused color, and then calculate the third proportion of all pixel areas falling into the color interval of the confused color in the entire pixel block. Then one of the third ratio, the category of the target object to which the pixel block belongs, the brightness of the pixel block, the color temperature corresponding to the previous frame image of the image to be processed, and the hue corresponding to the previous frame image of the image to be processed or more to determine the compensation coefficient.

Wherein, when determining the initial white balance gain in the image to be processed, it can be determined in a traditional way, for example, it can be determined directly based on the RGB value of the image. Of course, in some embodiments, the initial white balance gain for determination can be more Accurately, it is also possible to determine the target weight of the white point in the pixel block in combination with the scene category of each pixel block in the image to be processed, and then determine the initial white balance gain of the image to be processed based on the RGB value of the white point and the target weight, wherein, determine the pixel For the target weight of the white point in the block, and the specific implementation process of determining the initial white balance gain of the image to be processed based on the RGB value of the white point and the target weight, reference may be made to the description in the foregoing embodiments, and details are not repeated here.

In some embodiments, when the image to be processed is compensated according to whether the pixel block falls into the mixed color range, it can also be processed in two ways for the outdoor scene and the indoor scene. For example, the scene category information may include the confidence that the image to be processed is an indoor scene, and the confidence that the image to be processed is an outdoor scene, and the compensation coefficient includes the first compensation coefficient when the image to be processed is an indoor scene, and the image to be processed is an outdoor scene. The second compensation coefficient in the scene. Then, the first compensation coefficient can be used to compensate the initial white balance gain, and the obtained third white balance gain can be compensated by the second compensation coefficient to obtain the fourth white balance gain, which can then be based on the image to be processed as The confidence of the indoor scene and the confidence that the image to be processed is an outdoor scene are weighted and averaged on the third white balance gain and the fourth white balance gain to obtain the target white balance gain.

For example, after obtaining the initial white balance gain, the compensation coefficient used to compensate the initial white balance can be different for outdoor scenes and indoor scenes, so that it is more in line with the characteristics of indoor scenes or outdoor scenes, so it can be used for indoor scenes. Different compensation coefficients are determined for the scene and the outdoor scene, and then the white balance gains for the indoor scene and the outdoor scene are obtained, so that the final target can be obtained based on the confidence of the image to be processed as the indoor scene and the confidence of the outdoor scene. White balance gain.

In some embodiments, since the white balance processing can eliminate the influence of the light source, the actual scene seen by the user will still be affected by the light source. For example, under a yellow light source, the overall object will be yellowish. In order to make the image closer to the real scene, after performing white balance correction on the image to be processed, the image to be processed can be further processed with ambient color, so that the overall tone of the image is closer to the actual color.

It is not difficult to understand that, if there is no conflict, the solutions in the above embodiments can be combined freely, and due to space reasons, no more examples are given here.

In order to further explain the image processing method in the embodiment of the present application, a specific embodiment is explained below in conjunction with FIG. 2 .

When performing white balance processing on an image, if the white point is determined only based on the RGB value of the image, and then the white balance gain is determined, the obtained white balance gain is often not accurate enough, resulting in a poor image effect after white balance processing. In order to solve the above problems, this embodiment provides an image processing method, which specifically includes the following steps:

1. Determine the target confidence level that each pixel block in the image to be processed belongs to each scene category among a plurality of preset scene categories.

First, multiple scene categories (for example, category A, category B, category C, etc.) can be set in advance, and the principles for setting the multiple category categories are as follows:

(1) It can be used to determine whether the image to be processed is an indoor scene or an outdoor scene, for example, it can be the sky (it can be determined to be an outdoor scene), and the floor (it can be determined to be an indoor scene).

(2) can be used to determine the white point in the image to be processed.

(3) The color range of the scene included in the scene category overlaps with the color range of at least one scene other than this scene, for example, yellow objects and white objects under yellow light sources, green plants and white objects under fluorescent light wait

Then, the initial confidence that each pixel block in the image belongs to each scene category among the plurality of scene categories can be determined through a pre-trained neural network. Since the initial confidence determined directly through the neural network is not accurate enough, the initial confidence can be adjusted by combining the brightness information of the pixel block and the proportion Q1 of the area in the pixel block falling into the color range corresponding to each scene category, and obtaining each The object confidence that the pixel block belongs to each scene category. Wherein, the closer the luminance information in the pixel block is to the pre-marked luminance of the scene category, the greater the target confidence. The larger the ratio Q1 is, the greater the possibility is that the pixel block belongs to the scene category, and the greater the target confidence is.

2. Determine the proportion of the area belonging to each scene category in the plurality of scene categories in the image to be processed.

For each pixel block, the scene category with the highest target confidence can be used as the scene category corresponding to the pixel block, and then the ratio of the area of all pixel blocks belonging to each scene category to the area of the image to be processed can be calculated, and each The proportion Q2 of a scene category in the image to be processed, wherein, since the proportion Q2 determined by a single frame image may not be accurate, it can be combined with the proportion Q2 of the multi-frame images before and after the image to be processed, and the proportion Q2 can be calculated. Filtering processing (for example, averaging or weighted averaging) to obtain the final proportion Q3.

3. Determine the confidence that the image to be processed is an indoor scene or an outdoor scene.

According to the proportion Q3 of each scene category in the image to be processed, and the weight of the scene category corresponding to the indoor scene, the confidence that the image to be processed is an indoor scene is obtained. According to the proportion Q3 of each scene category in the image to be processed, and the weight of the scene category corresponding to the outdoor scene, the confidence that the image to be processed is an outdoor scene is obtained.

Wherein, each scene category corresponds to the weight of the outdoor scene, and each scene category corresponds to the weight of the indoor scene can be based on the brightness of the pixel area belonging to each scene category in the image to be processed, and the type of the scene category (for example, sky The weight corresponding to the outdoor scene will be higher, and the weight of the floor corresponding to the indoor scene will be higher) to determine.

4. Determination of white balance gain

The white point in the image to be processed can be determined based on the RGB values of the pixels of the image to be processed, and for the white point in each pixel block, the weight of the white point in the pixel block can be adjusted according to the scene category to which the pixel block belongs, Get the target weight, for example, for the white point of the sky, green plants, soil and other categories, you can reduce the weight of the white point, and for the scene category that may be a white point, such as outdoor roads, you can increase the weight of the white point, In order to achieve more accurate white finding. After determining the target weight of each white point, the white balance gain can be determined based on the target weight and the RGB value of each white point.

In addition, for the two cases where the image to be processed is an indoor scene and an outdoor scene, the target weight corresponding to the white point may also be different. Therefore, it can be processed in two ways to determine the white balance gain of 1 in the indoor scene and the white balance gain in the outdoor scene. The white balance gain is 2.

5. Confused color processing

Because the color intervals of some scenes are relatively close, there is a problem of confusion. For example, scenes such as yellow objects and white objects under yellow light, blue objects and white objects under blue light, green plants and white objects under fluorescent light sources, snow scenes and blue sky are easy to confuse because their color ranges are very close. Determining the true scene category of the pixel area, which in turn results in an inaccurate determination of the white balance gain. Therefore, after determining the white balance, it is possible to further determine whether the RGB values of each pixel block fall into the color interval of the confusion color based on the scene category to which each pixel block belongs, that is, to determine whether each pixel block may be confused, and then to determine the white balance Gain is compensated.

For example, it is possible to calculate the proportion Q4 of the area where the RGB value of the pixel block falls into the color interval of the confused color, according to the size of the proportion Q4, the scene category to which the pixel block belongs, the ambient brightness, and the color corresponding to the previous frame of the image to be processed The compensation coefficient is determined based on the mild tone, etc., and then the white balance gain determined in the previous step is compensated.

Among them, when compensating the white balance gain, it can also be divided into indoor and outdoor two-way processing. For indoor scenes and outdoor scenes, different compensation coefficients b1 and b2 are determined respectively, and then use b1 to determine the white balance in the indoor scene Compensate with gain 1 to obtain white balance gain 3, and then use b2 to compensate white balance gain 2 in the indoor scene determined in the previous step to obtain white balance gain 4.

6. Integrate the two-way processing results of indoor scenes and outdoor scenes.

According to step 3, it is determined that the image to be processed is the confidence degree of the indoor scene, and the confidence degree of the image to be processed is the outdoor scene, and the white balance gain 3 and the white balance gain 4 are weighted to obtain the final target white balance gain, which is used for the image to be processed. The image is corrected.

7. Atmosphere color processing

After the white balance processing is performed on the image, the ambient color of the image can be further adjusted in combination with the color of the light source to obtain the target image, so that the color of the target image is more in line with the actual scene.

Through the method provided in this embodiment, when performing white balance processing, accurate "white finding" can be achieved, and accurate white balance gains can be obtained. At the same time, the influence of some scenes where the color intervals will be confused on the white balance can be eliminated, and a more accurate white balance can be obtained. Accurate white balance processing effect.

Corresponding to the above method, the embodiment of the present application also provides an image processing device. As shown in FIG. Executing computer program, when the processor 31 executes the computer program, the following steps can be realized:

Get the image to be processed;

determining scene category information of pixel blocks in the image to be processed;

Determine a target white balance gain based on the scene category information of the pixel block, so as to perform white balance correction on the image to be processed by using the target white balance gain.

In some embodiments, the scene category information includes: a target object category to which the pixel block belongs and/or information indicating that the image to be processed is an indoor scene or an outdoor scene.

In some embodiments, wherein the target object category is one of a plurality of preset object categories, each object category in the preset plurality of object categories has at least one of the following characteristics:

Each object category can be used to determine that the image to be processed is an indoor scene or an outdoor scene;

Each of the object categories can be used to determine the white point in the image to be processed;

The color interval of the scene included in each object category overlaps with the color interval of at least one scene other than the scene.

In some embodiments, the category of the target object to which the pixel block belongs is determined based on the following manner:

Respectively determining that the pixel block is a target confidence degree of each object category in a plurality of preset object categories;

Determine the target object category to which the pixel block belongs from the plurality of object categories based on the target confidence.

In some embodiments, the determining the target object category to which the pixel block belongs from the plurality of object categories based on the target confidence degree includes:

The object category with the highest target confidence is taken as the target object category to which the pixel block belongs.

In some embodiments, determining that the pixel block is a target confidence degree of each object category in a plurality of object categories includes:

For each object category, determine the initial confidence that the pixel block is the object category based on the pre-trained neural network;

using said initial confidence as said target confidence; or

For each object category, determine the initial confidence that the pixel block is the object category based on a pre-trained neural network;

The initial confidence is adjusted based on the pixel value related information of the pixel block to obtain the target confidence.

In some embodiments, the pixel value-related information includes one or more of the following:

The brightness of the pixel block and the first proportion of the first specified pixel area in the pixel block, wherein the RGB value of the first specified pixel area falls within the color range corresponding to the scene category.

In some embodiments, the target confidence level is positively related to the first proportion, and the target confidence level is positively related to the closeness of the luminance of the pixel block to the nominal luminance of the object category.

In some embodiments, when the brightness of the pixel block is less than a preset brightness threshold, the target confidence level is 0.

In some embodiments, the information indicating that the image to be processed is an indoor scene or an outdoor scene includes:

The confidence level that the image to be processed is an indoor scene, and/or the confidence level that the image to be processed is an outdoor scene.

In some embodiments, the confidence that the image to be processed is an indoor scene or the confidence of an outdoor scene is determined based on the following manner:

determining a second proportion of all pixel blocks belonging to each target object category in the image to be processed based on the target object category to which each pixel block in the image to be processed belongs;

Determine the confidence that the image to be processed is an indoor scene based on the second proportion and the weight of each target object category for the indoor scene; or

Determine the confidence that the image to be processed is an outdoor scene based on the second ratio and the weights of each target object category for the outdoor scene.

In some embodiments, the weight of each target object category for the indoor scene or the weight of each target object category for the indoor scene is determined based on one or more of the following: the brightness of all pixel blocks belonging to each target object category, and the The type of category.

In some embodiments, the second ratio is determined based on the following methods:

Acquiring multiple frames of target images collected before and/or after the image to be processed;

performing filtering processing on the proportions of all pixel blocks belonging to each target object category in the multi-frame target image and the proportions of pixel blocks belonging to each target object category in the image to be processed, to obtain the second proportion .

In some embodiments, when the processor is configured to determine the target white balance gain based on the scene category information of the pixel block, it is specifically configured to:

determining the target weight of the white point in the pixel block based on the category of the target object to which the pixel block belongs;

The target white balance gain is determined based on the RGB value of the white point and the target weight.

In some embodiments, the scene category information includes the target object category to which the pixel block belongs, and when the target object categories are different, the target weights are different; and/or

The scene category information includes information indicating that the image to be processed is an indoor scene or an outdoor scene, and the target weight of the image to be processed is an indoor scene, which is different from the target weight when the image to be processed is an outdoor scene. target weight.

In some embodiments, when the processor is configured to determine the target weight of the white point in the pixel block based on the scene category information of the pixel block, it is specifically configured to:

determining initial weights for white points in the pixel block;

The initial weight is adjusted based on the scene category information of the pixel block to obtain the target weight of the white point in the pixel block.

In some embodiments, the initial weight is determined based on at least one or more of the following: the brightness of the pixel block, and the color temperature corresponding to the pixel block, wherein the color temperature corresponding to the pixel block is determined based on the pixel The RGB value of the block is determined.

In some embodiments, the scene category information includes: the confidence level that the image to be processed is an indoor scene, and/or the confidence level that the image to be processed is an outdoor scene, and the processor is configured to When determining the target white balance gain based on scene category information, it is specifically used for:

When it is determined that the image to be processed is an outdoor scene, the first target weight of the white point in the pixel block is determined based on the RGB value of the white point and the first target weight to determine a first white balance gain;

When it is determined that the image to be processed is an indoor scene, the second target weight of the white point in the pixel block is determined based on the RGB value of the white point and the second target weight to determine a second white balance gain;

Based on the confidence that the image to be processed is an indoor scene and the confidence that the image to be processed is an outdoor scene, perform weighting processing on the first white balance gain and the second white balance gain to obtain the target white balance gain.

In some embodiments, when the processor is configured to determine the target white balance gain based on the scene category information of the pixel block, it is specifically configured to:

determining the initial white balance gain of the image to be processed;

For each pixel block, when it is determined that the RGB value of the pixel block falls into the color interval of the confused color, the initial white balance gain of the pixel block is calculated based on the scene category information of the pixel block. compensation to obtain the target white balance gain of the pixel block.

In some embodiments, the processor is configured to compensate the initial white balance gain based on the scene category information of the pixel block, and when the target white balance gain is obtained, it is specifically used for:

determining a compensation coefficient based on the scene category information of the pixel block;

The compensation coefficient is used to compensate the initial white balance gain to obtain the target white balance gain.

In some embodiments, the scene category information includes the target object category to which the pixel block belongs, and the compensation coefficient is determined based on the following method:

Determine the third proportion of the second specified area in the pixel block, wherein the second specified area is a color interval in which the RGB value of the pixel block falls into the confused color;

Based on the third ratio, the category of the target scene to which the pixel block belongs, the brightness of the pixel block, the color temperature corresponding to the previous frame of the image to be processed, and the corresponding color temperature of the previous frame of the image to be processed One or more of the hues determine the compensation coefficients.

In some embodiments, when the processor is used to determine the initial white balance gain of the image to be processed, it is specifically used for:

determining the target weight of the white point in the pixel block based on the category of the target object to which the pixel block belongs;

An initial white balance gain of the image to be processed is determined based on the RGB value of the white point and the target weight.

In some embodiments, the scene category information includes the confidence that the image to be processed is an indoor scene, and the confidence that the image to be processed is an outdoor scene, and the compensation coefficient includes that the image to be processed is an indoor scene The first compensation coefficient under, and the image to be processed is the second compensation coefficient under an outdoor scene;

Compensating the initial white balance gain by using the first compensation coefficient to obtain a third white balance gain;

Compensating the initial white balance gain by using the second compensation coefficient to obtain a fourth white balance gain;

Based on the confidence that the image to be processed is an indoor scene and the confidence that the image to be processed is an outdoor scene, perform weighted average processing on the third white balance gain and the fourth white balance gain to obtain the target White balance gain.

In some embodiments, the processor is further configured to:

Ambient color processing is performed on the image to be processed after white balance correction processing.

Further, an image acquisition device is also provided in an embodiment of the present application, and the image acquisition includes the image processing apparatus as described in any one of the above embodiments. The image acquisition device can hold various devices such as pan-tilt cameras, mobile phones, and drones.

Correspondingly, the embodiments of this specification further provide a computer storage medium, where a program is stored in the storage medium, and when the program is executed by a processor, the method in any of the foregoing embodiments is implemented.

Embodiments of the present description may take the form of a computer program product embodied on one or more storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer usable storage media includes both volatile and non-permanent, removable and non-removable media, and may be implemented by any method or technology for information storage. Information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for computers include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Flash memory or other memory technology, Compact Disc Read-Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cartridge, tape magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

As for the device embodiment, since it basically corresponds to the method embodiment, for related parts, please refer to the part description of the method embodiment. The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without creative effort.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. The term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements but also other elements not expressly listed elements, or also elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The methods and devices provided by the embodiments of the present invention have been described in detail above. The principles and implementation methods of the present invention have been explained by using specific examples in this paper. The descriptions of the above embodiments are only used to help understand the methods and methods of the present invention. core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and scope of application. In summary, the content of this specification should not be construed as limiting the present invention .

Claims (50)

1. An image processing method, characterized in that the method comprises:

   Get the image to be processed;

   determining scene category information of pixel blocks in the image to be processed;

   Determine a target white balance gain based on the scene category information of the pixel block, so as to perform white balance correction on the image to be processed by using the target white balance gain.
2. The method according to claim 1, wherein the scene category information includes: the target object category to which the pixel block belongs and/or information indicating that the image to be processed is an indoor scene or an outdoor scene.
3. The method according to claim 2, wherein the target object category is one of a plurality of preset object categories, and each object category in the preset plurality of object categories has at least the following A characteristic:

   Each object category can be used to determine that the image to be processed is an indoor scene or an outdoor scene;

   Each of the object categories can be used to determine the white point in the image to be processed;

   The color interval of the scene included in each object category overlaps with the color interval of at least one scene other than the scene.
4. The method according to claim 2 or 3, wherein the category of the target object to which the pixel block belongs is determined based on the following method:

   Respectively determining that the pixel block is a target confidence degree of each object category in a plurality of preset object categories;

   Determine the target object category to which the pixel block belongs from the plurality of object categories based on the target confidence.
5. The method according to claim 4, wherein determining the target object category to which the pixel block belongs from the plurality of object categories based on the target confidence degree comprises:

   The object category with the highest target confidence is taken as the target object category to which the pixel block belongs.
6. The method according to claim 4 or 5, wherein determining the pixel block is respectively the target confidence level of each object category in a plurality of object categories, comprising:

   For each object category, determine the initial confidence that the pixel block is the object category based on a pre-trained neural network;

   using said initial confidence as said target confidence; or

   For each object category, determine the initial confidence that the pixel block is the object category based on a pre-trained neural network;

   The initial confidence is adjusted based on the pixel value related information of the pixel block to obtain the target confidence.
7. The method according to claim 6, wherein the pixel value-related information includes one or more of the following:

   The brightness of the pixel block and the first proportion of the first specified pixel area in the pixel block, wherein the RGB value of the first specified pixel area falls within the color interval corresponding to the object category.
8. The method according to claim 7, wherein the target confidence is positively related to the first proportion, and the target confidence is positively related to the brightness of the pixel block and the calibrated brightness of the object category the degree of proximity.
9. The method according to claim 7, wherein when the brightness of the pixel block is less than a preset brightness threshold, the target confidence level is 0.
10. The method according to any one of claims 2-9, wherein the information indicating that the image to be processed is an indoor scene or an outdoor scene includes:

    The confidence level that the image to be processed is an indoor scene, and/or the confidence level that the image to be processed is an outdoor scene.
11. The method according to claim 10, wherein the confidence that the image to be processed is an indoor scene or the confidence of an outdoor scene is determined based on the following manner:

    determining a second proportion of all pixel blocks belonging to each target object category in the image to be processed based on the target object category to which each pixel block in the image to be processed belongs;

    Determine the confidence that the image to be processed is an indoor scene based on the second proportion and the weight of each target object category for the indoor scene; or

    Determine the confidence that the image to be processed is an outdoor scene based on the second ratio and the weights of each target object category for the outdoor scene.
12. The method according to claim 11, wherein the weight of each target object category for the indoor scene or the weight of each target object category for the indoor scene is determined based on one or more of the following: all pixel blocks belonging to each target object category brightness, and the type of each target object category.
13. The method according to claim 11 or 12, wherein the second proportion is determined based on the following method:

    Acquiring multiple frames of target images collected before and/or after the image to be processed;

    performing filtering processing on the proportions of all pixel blocks belonging to each target object category in the multi-frame target image and the proportions of pixel blocks belonging to each target object category in the image to be processed, to obtain the second proportion .
14. The method according to any one of claims 1-13, wherein determining the target white balance gain based on the scene category information of the pixel block includes:

    determining the target weight of the white point in the pixel block based on the scene category information of the pixel block;

    The target white balance gain is determined based on the RGB value of the white point and the target weight.
15. The method according to claim 14, wherein the scene category information includes the target object category to which the pixel block belongs, and when the target object category is different, the target weight is different; and/or

    The scene category information includes information indicating that the image to be processed is an indoor scene or an outdoor scene, and the target weight of the image to be processed is an indoor scene, which is different from the target weight when the image to be processed is an outdoor scene. target weight.
16. The method according to claim 14 or 15, wherein determining the target weight of the white point in the pixel block based on the scene category information of the pixel block comprises:

    determining initial weights for white points in the pixel block;

    The initial weight is adjusted based on the scene category information of the pixel block to obtain the target weight of the white point in the pixel block.
17. The method according to claim 16, wherein the initial weight is determined based on at least one or more of the following: the brightness of the pixel block, and the color temperature corresponding to the pixel block, wherein the pixel block The corresponding color temperature is determined based on the RGB values of the pixel block.
18. The method according to any one of claims 13-17, wherein the scene category information includes: the confidence that the image to be processed is an indoor scene, and/or the confidence that the image to be processed is an outdoor scene degree, determining the target white balance gain based on the scene category information of the pixel block, including:

    When it is determined that the image to be processed is an outdoor scene, the first target weight of the white point in the pixel block is determined based on the RGB value of the white point and the first target weight to determine a first white balance gain;

    When it is determined that the image to be processed is an indoor scene, the second target weight of the white point in the pixel block is determined based on the RGB value of the white point and the second target weight to determine a second white balance gain;

    Based on the confidence that the image to be processed is an indoor scene and the confidence that the image to be processed is an outdoor scene, perform weighting processing on the first white balance gain and the second white balance gain to obtain the target white balance gain.
19. The method according to any one of claims 1-13, wherein determining the target white balance gain based on the scene category information of the pixel block includes:

    determining the initial white balance gain of the image to be processed;

    For each pixel block, when it is determined that the RGB value of the pixel block falls into the color interval of the confused color, the initial white balance gain of the pixel block is calculated based on the scene category information of the pixel block. compensation to obtain the target white balance gain of the pixel block.
20. The method according to claim 19, wherein the initial white balance gain is compensated based on scene category information of the pixel block to obtain a target white balance gain, comprising:

    determining a compensation coefficient based on the scene category information of the pixel block;

    The compensation coefficient is used to compensate the initial white balance gain to obtain the target white balance gain.
21. The method according to claim 20, wherein the scene category information includes the target object category to which the pixel block belongs, and the compensation coefficient is determined based on the following method:

    Determine the third proportion of the second specified area in the pixel block, wherein the second specified area is a color interval area in the pixel block whose RGB value falls into the confused color;

    Based on the third ratio, the target scene category to which the pixel block belongs, the brightness of the pixel block, the color temperature corresponding to the previous frame of the image to be processed, and the corresponding color temperature of the previous frame of the image to be processed One or more of the hues determine the compensation coefficients.
22. The method according to any one of claims 19-21, wherein determining the initial white balance gain of the image to be processed comprises:

    determining the target weight of the white point in the pixel block based on the scene category information of the pixel block;

    An initial white balance gain of the image to be processed is determined based on the RGB value of the white point and the target weight.
23. The method according to any one of claims 19-22, wherein the scene category information includes a confidence degree that the image to be processed is an indoor scene, and a confidence degree that the image to be processed is an outdoor scene, so The compensation coefficient includes a first compensation coefficient when the image to be processed is an indoor scene, and a second compensation coefficient when the image to be processed is an outdoor scene;

    Compensating the initial white balance gain by using the first compensation coefficient to obtain a third white balance gain;

    Compensating the initial white balance gain by using the second compensation coefficient to obtain a fourth white balance gain;

    Based on the confidence that the image to be processed is an indoor scene and the confidence that the image to be processed is an outdoor scene, perform weighted average processing on the third white balance gain and the fourth white balance gain to obtain the target White balance gain.
24. The method according to any one of claims 1-23, wherein the method further comprises:

    Ambient color processing is performed on the image to be processed after white balance correction processing.
25. An image processing device, characterized in that the device includes a processor, a memory, and a computer program stored in the memory for execution by the processor, when the processor executes the computer program, the following steps can be realized :

    Get the image to be processed;

    determining scene category information of pixel blocks in the image to be processed;

    Determine a target white balance gain based on the scene category information of the pixel block, so as to perform white balance correction on the image to be processed by using the target white balance gain.
26. The device according to claim 25, wherein the scene category information includes: the target object category to which the pixel block belongs and/or information indicating that the image to be processed is an indoor scene or an outdoor scene.
27. The device according to claim 26, wherein the target object category is one of a plurality of preset object categories, and each object category in the preset plurality of object categories has at least the following A characteristic:

    Each object category can be used to determine that the image to be processed is an indoor scene or an outdoor scene;

    Each of the object categories can be used to determine the white point in the image to be processed;

    The color interval of the scene included in each object category overlaps with the color interval of at least one scene other than the scene.
28. The device according to claim 25 or 27, wherein the category of the target object to which the pixel block belongs is determined based on the following method:

    Respectively determining that the pixel block is a target confidence degree of each object category in a plurality of preset object categories;

    Determine the target object category to which the pixel block belongs from the plurality of object categories based on the target confidence.
29. The device according to claim 28, wherein the determining the target object category to which the pixel block belongs from the plurality of object categories based on the target confidence degree comprises:

    The object category with the highest target confidence is taken as the target object category to which the pixel block belongs.
30. The device according to claim 28 or 29, wherein determining the target confidence of each object category in the plurality of object categories for the pixel block comprises:

    For each object category, determine the initial confidence that the pixel block is the object category based on a pre-trained neural network;

    using said initial confidence as said target confidence; or

    For each object category, determine the initial confidence that the pixel block is the object category based on a pre-trained neural network;

    The initial confidence is adjusted based on the pixel value related information of the pixel block to obtain the target confidence.
31. The device according to claim 30, wherein the pixel value-related information includes one or more of the following:

    The brightness of the pixel block and the first proportion of the first specified pixel area in the pixel block, wherein the RGB value of the first specified pixel area falls within the color interval corresponding to the object category.
32. The device according to claim 31, wherein the target confidence level is positively related to the first proportion, and the target confidence level is positively related to the brightness of the pixel block and the nominal brightness of the object category the degree of proximity.
33. The device according to claim 31, wherein when the brightness of the pixel block is less than a preset brightness threshold, the target confidence level is 0.
34. The device according to any one of claims 25-33, wherein the information indicating that the image to be processed is an indoor scene or an outdoor scene includes:

    The confidence level that the image to be processed is an indoor scene, and/or the confidence level that the image to be processed is an outdoor scene.
35. The device according to claim 34, wherein the confidence that the image to be processed is an indoor scene or the confidence of an outdoor scene is determined based on the following method:

    determining a second proportion of all pixel blocks belonging to each target object category in the image to be processed based on the target object category to which each pixel block in the image to be processed belongs;

    Determine the confidence that the image to be processed is an indoor scene based on the second proportion and the weight of each target object category for the indoor scene; or

    Determine the confidence that the image to be processed is an outdoor scene based on the second ratio and the weights of each target object category for the outdoor scene.
36. The device according to claim 35, wherein the weight of each target object category for the indoor scene or the weight of each target object category for the indoor scene is determined based on one or more of the following: all pixel blocks belonging to each target object category brightness, and the type of each target object category.
37. The device according to claim 35 or 36, wherein the second ratio is determined based on the following method:

    Acquiring multiple frames of target images collected before and/or after the image to be processed;

    performing filtering processing on the proportions of all pixel blocks belonging to each target object category in the multi-frame target image and the proportions of pixel blocks belonging to each target object category in the image to be processed, to obtain the second proportion .
38. The device according to any one of claims 25-37, wherein when the processor is used to determine the target white balance gain based on the scene category information of the pixel block, it is specifically used for:

    determining the target weight of the white point in the pixel block based on the scene category information of the pixel block;

    The target white balance gain is determined based on the RGB value of the white point and the target weight.
39. The device according to claim 38, wherein the scene category information includes the target object category to which the pixel block belongs, and when the target object category is different, the target weight is different; and/or

    The scene category information includes information indicating that the image to be processed is an indoor scene or an outdoor scene, and the target weight of the image to be processed is an indoor scene, which is different from the target weight when the image to be processed is an outdoor scene. target weight.
40. The device according to claim 38 or 39, wherein when the processor is used to determine the target weight of the white point in the pixel block based on the scene category information of the pixel block, it is specifically used to:

    determining initial weights for white points in the pixel block;

    The initial weight is adjusted based on the scene category information of the pixel block to obtain the target weight of the white point in the pixel block.
41. The device according to claim 40, wherein the initial weight is determined based on at least one or more of the following: the brightness of the pixel block, and the color temperature corresponding to the pixel block, wherein the pixel block The corresponding color temperature is determined based on the RGB values of the pixel block.
42. The device according to any one of claims 38-41, wherein the scene category information includes: the confidence that the image to be processed is an indoor scene, and/or the confidence that the image to be processed is an outdoor scene degree, when the processor is used to determine the target white balance gain based on the scene category information of the pixel block, it is specifically used for:

    When it is determined that the image to be processed is an outdoor scene, the first target weight of the white point in the pixel block is determined based on the RGB value of the white point and the first target weight to determine a first white balance gain;

    When it is determined that the image to be processed is an indoor scene, the second target weight of the white point in the pixel block is determined based on the RGB value of the white point and the second target weight to determine a second white balance gain;

    Based on the confidence that the image to be processed is an indoor scene and the confidence that the image to be processed is an outdoor scene, perform weighting processing on the first white balance gain and the second white balance gain to obtain the target white balance gain.
43. The device according to any one of claims 25-37, wherein when the processor is used to determine the target white balance gain based on the scene category information of the pixel block, it is specifically used for:

    determining the initial white balance gain of the image to be processed;

    For each pixel block, when it is determined that the RGB value of the pixel block falls into the color interval of the confused color, the initial white balance gain of the pixel block is calculated based on the scene category information of the pixel block. compensation to obtain the target white balance gain of the pixel block.
44. The device according to claim 43, wherein the processor is configured to compensate the initial white balance gain based on the scene category information of the pixel block, and to obtain a target white balance gain, specifically for:

    determining a compensation coefficient based on the scene category information of the pixel block;

    The compensation coefficient is used to compensate the initial white balance gain to obtain the target white balance gain.
45. The device according to claim 44, wherein the scene category information includes the target object category to which the pixel block belongs, and the compensation coefficient is determined based on the following method:

    Determine the third proportion of the second specified area in the pixel block, wherein the second specified area is a color interval in which the RGB value of the pixel block falls into the confused color;

    Based on the third ratio, the target scene category to which the pixel block belongs, the brightness of the pixel block, the color temperature corresponding to the previous frame of the image to be processed, and the corresponding color temperature of the previous frame of the image to be processed One or more of the hues determine the compensation coefficients.
46. The device according to any one of claims 44-45, wherein when the processor is used to determine the initial white balance gain of the image to be processed, it is specifically used for:

    determining the target weight of the white point in the pixel block based on the scene category information of the pixel block;

    An initial white balance gain of the image to be processed is determined based on the RGB value of the white point and the target weight.
47. The device according to any one of claims 43-46, wherein the scene category information includes the confidence level that the image to be processed is an indoor scene, and the confidence level that the image to be processed is an outdoor scene, so The compensation coefficient includes a first compensation coefficient when the image to be processed is an indoor scene, and a second compensation coefficient when the image to be processed is an outdoor scene;

    Compensating the initial white balance gain by using the first compensation coefficient to obtain a third white balance gain;

    Compensating the initial white balance gain by using the second compensation coefficient to obtain a fourth white balance gain;

    Based on the confidence that the image to be processed is an indoor scene and the confidence that the image to be processed is an outdoor scene, perform weighted average processing on the third white balance gain and the fourth white balance gain to obtain the target White balance gain.
48. The device according to any one of claims 25-47, wherein the processor is further configured to:

    Ambient color processing is performed on the image to be processed after white balance correction processing.
49. An image acquisition device, characterized in that the image acquisition device comprises the image processing device according to any one of claims 25-48.
50. A computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed, the method according to any one of claims 1-24 is realized.

Images