WO2022228032A1

WO2022228032A1 - Image color cast detection method, device, and refrigerator

Info

Publication number: WO2022228032A1
Application number: PCT/CN2022/084706
Authority: WO
Inventors: 毕研华
Original assignee: 青岛海尔电冰箱有限公司; 海尔智家股份有限公司
Priority date: 2021-04-27
Filing date: 2022-04-01
Publication date: 2022-11-03
Also published as: CN115249272A

Abstract

Disclosed are an image color cast detection method, a device, and a refrigerator, the method comprising: cropping multiple ROIs of an image to be tested, so as to obtain N ROIs; separately carrying out color cast detection on the N regions to obtain a color cast detection result of each region; and if the number of regions that have color cast in the N regions is greater than or equal to a first threshold, then determining that said image has color cast. Thus, detection efficiency and accuracy are high.

Description

Image color cast detection method, equipment and refrigerator

technical field

The invention relates to the field of home appliances, and in particular, to a detection method, equipment and refrigerator for image color cast.

Background technique

Color is one of the important features of images, and is usually an important basis for image processing and analysis. It is of great significance for research in computer vision fields such as target recognition and detection, image segmentation, and video retrieval. The color of an object is determined by its reflectivity characteristics. Due to the color constancy of the human visual system, the influence of factors such as lighting conditions on the color can be eliminated to a certain extent. However, the imaging device does not have this "adjustment" function and will be affected by The influence of factors such as the external light environment and the characteristics of the imaging photosensitive components themselves cause a certain degree of error between the image color and the real color of the object, that is, "color cast". At the same time, when the external light environment is too dark or too bright, the photosensitive components cannot obtain a normally exposed image, which will also cause the image color to be too dark or too explosive. If the image color cast is not detected and calibrated in a timely and effective manner, the color-based image vision algorithm may not work.

In the field of refrigerators, when the light inside the refrigerator is blocked by a colored plastic bag, causing the color of the light to change, the image captured inside the refrigerator will have a serious color cast, which in turn affects the accuracy of object recognition in the image. In order to reduce the problem of reducing the accuracy of image recognition caused by the above situation, it is necessary to perform color cast detection on pictures taken inside the refrigerator. However, the current existing theoretical methods for color cast detection, aiming at the color cast detection of images captured inside the refrigerator, require a large amount of calculation and are time-consuming, and at the same time, the accuracy rate is low.

Therefore, there is an urgent need to propose an efficient and high-accuracy color cast detection method for interior images of refrigerators.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an image color cast detection method, equipment and refrigerator.

To achieve one of the above purposes of the invention, an embodiment of the present invention provides a method for detecting color cast of an image, the method comprising:

Cropping multiple ROI regions of the image to be detected to obtain N ROI regions, where N is a positive integer greater than 1;

Perform color cast detection on the N regions respectively, and obtain the color cast detection result of each area;

If the number of regions with color cast in the N regions is greater than or equal to the first threshold, it is determined that the image to be detected has color cast.

As a further improvement of an embodiment of the present invention, performing color cast detection on an area image specifically includes:

Read in the area image in RGB space to obtain the RGB color space of the area image;

Convert the RGB color space of the image to the lab chromaticity space;

Calculate the color cast factor K of the lab chromaticity space. If the color cast factor K is lower than the color cast threshold, it is determined that the regional image has a color cast, otherwise, it is determined that the regional image does not have a color cast.

As a further improvement of an embodiment of the present invention, the method for obtaining the color cast threshold includes:

Obtain a plurality of images with different degrees of color cast, test and calculate their recognition rate and color cast factor K respectively, and select the value range by obtaining the value range of the color cast factor K corresponding to the image whose recognition rate exceeds the recognition rate threshold. The maximum value is used as the color cast threshold.

read in the area image in RGB space;

Convert an image with RGB color channels to HSV channels;

Perform channel segmentation on the image, and divide it into three channel data of H, S, and V respectively;

Calculate the average value of all the pixel points of the image in the H, S, V three channel data respectively;

If at least one of the three average values is not within the set value range, it is determined that there is color cast in the area image.

As a further improvement of an embodiment of the present invention, the method further includes:

After it is determined that the image to be detected has color cast, the color cast of the image is automatically corrected by adjusting the brightness of the fill light disposed around the camera.

As a further improvement of an embodiment of the present invention, the "automatic correction of the color cast of the image by adjusting the brightness of the fill light set around the camera" specifically includes:

While gradually increasing the brightness of the fill light, control the camera to periodically capture images and detect whether the captured images have color cast;

After detecting that the image captured by the camera has no color cast, stop increasing the brightness of the fill light.

As a further improvement of an embodiment of the present invention, a photographed image of a yogurt special area inside the refrigerator is obtained as an image to be detected, wherein the yogurt special area includes a bottle holder for placing yogurt.

As a further improvement of an embodiment of the present invention, the N ROI regions are distributed around the bottle holder of the image to be detected.

To achieve one of the above purposes of the invention, an embodiment of the present invention provides an electronic device, including a memory and a processor, where the memory stores a computer program that can run on the processor, and the processor executes the program At the same time, the steps in any of the above-mentioned methods for detecting color cast of an image are implemented.

In order to achieve one of the above objectives of the present invention, an embodiment of the present invention provides a refrigerator, which includes the above-mentioned electronic device.

Compared with the prior art, the method for detecting color cast of an image of the present invention cuts out a plurality of ROI regions, and performs color cast detection on these ROI regions respectively, and finally can be fast and accurate according to the color cast detection results of each ROI region. The color cast detection result of the image to be detected is obtained, and the detection efficiency is high and the accuracy rate is high.

Description of drawings

FIG. 1 is a schematic flowchart of an image color cast detection method of the present invention.

FIG. 2 is a schematic diagram of the distribution of the ROI region in the image to be detected according to the present invention.

FIG. 3 is a partial structural schematic diagram of the interior of the refrigerator of the present invention.

FIG. 4 shows the calculation steps of the color cast factor K of the present invention.

Among them, 1. refrigerator; 2. yogurt area; 3. first bottle holder; 4. second bottle holder; 5. camera and fill light; 6. image to be detected; 7. ROI area.

Detailed ways

The present invention will be described in detail below with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and structural, method, or functional changes made by those skilled in the art according to these embodiments are all included in the protection scope of the present invention.

The yogurt area is set up in the refrigerator, and the yogurt can be monitored centrally. Generally, the yogurt area is provided with a bottle holder, and the yogurt is placed in the bottle holder. In the process of monitoring the yogurt in the bottle holder, it is necessary to photograph and identify the image of the yogurt area. However, sometimes the camera will be blocked by colored plastic bags or the like, causing the color of the light to change, resulting in a color cast in the captured image, which affects the accuracy of image recognition.

Therefore, the present invention provides an image color cast detection method. The method can cut out multiple ROI regions and perform color cast detection on these ROI regions respectively, and finally, according to the color cast detection results of each ROI region, the method can be fast and accurate. The color cast detection result of the image to be detected is obtained, and the detection efficiency is high and the accuracy rate is high.

As shown in Figure 1, the method includes:

Step S100 : crop multiple ROI regions of the image to be detected to obtain N ROI regions, where N is a positive integer greater than 1.

Since color cast detection is performed on the entire image, the detection efficiency is low and the accuracy rate is low (for example, only 1/3 of the area has color cast, and color cast detection based on the entire image cannot detect color cast). Multiple ROI regions (regions of interest) are cropped to obtain N ROI regions greater than 1. Since the size of the cropped ROI region image is much smaller than the size of the entire image, the color cast detection on these ROI region images has high efficiency, high detection speed, and high accuracy.

The present invention mainly detects the color cast of the image of the yogurt special area in the refrigerator. Therefore, the photographed image of the yogurt special area in the refrigerator is obtained as the image to be detected, wherein the yogurt special area includes a bottle holder for placing yogurt.

Further, as shown in FIG. 2 , the N ROI regions are distributed around the bottle holder of the image to be detected. Since the inside of the bottle holder is used to place yogurt, after the yogurt is placed, the bottle holder is partially covered by the yogurt, which cannot provide a stable environment for color cast detection. Therefore, the surrounding area of the bottle holder that will not be blocked by yogurt is selected as the ROI area for detection, which will not be affected by the placed yogurt. And because the N ROI areas are distributed around the bottle holder, even if a color cast appears in part of the image, it can be detected, and the detection accuracy is high.

Step S200: Perform color cast detection on the N regions respectively, and obtain a color cast detection result of each area.

Color cast detection is performed on each of the N areas, respectively, to obtain a color cast detection result of each area. Specifically, a variety of methods can be used to detect color casts in each area. For example, in order to improve the accuracy of color cast detection, color cast detection based on lab space is used to detect color casts in an area, or in order to perform color cast detection more simply and quickly Color cast detection, you can use the color cast detection based on HSV space to perform color cast detection on an area.

In a preferred embodiment, performing color cast detection based on lab space on an area image specifically includes:

Step S211: Read in the area image in RGB space to obtain the RGB color space of the area image.

The RGB color space is based on the three basic colors of R (Red, red), G (Green, green), and B (Blue, blue), which are superimposed to different degrees to produce rich and extensive colors, so it is commonly known as the three-primary color mode. Specifically, RGB consists of three channels: R, G, and B, the brightest red + the brightest green + the brightest blue = white; the darkest red + the darkest green + the darkest blue = black; And between the brightest and the darkest, the same shade of red + the same shade of green + the same shade of blue = gray. In any channel of RGB, white and black represent the lightness and darkness of the color. Therefore, where there is white or gray-white, the three channels of R, G, and B cannot be black, because there must be three channels of R, G, and B to form these colors.

Step S212: Convert the RGB color space of the image into a lab color space.

Lab is an uncommon color space. It was established on the basis of the International Standard for Color Measurement developed by the International Commission on Illumination (CIE) in 1931. In 1976, it was officially named CIELab after modification. It is a device-independent color system and a color system based on physiological characteristics. This also means that it uses a digital method to describe human visual perception. The L component in the Lab color space is used to represent the brightness of the pixel, and the value range is [0,100], which means from pure black to pure white; a means the range from red to green, and the value range is [127,-128]; b represents the range from yellow to blue, and the value range is [127,-128]. The lightness channel (L) in LAB is specially responsible for the lightness and darkness of the whole picture. Simply put, it is the black and white version of the whole picture. The a channel and the b channel are only responsible for the amount of color. The a channel represents the range from magenta to dark green; b represents the range from burnt yellow to blue.

The RGB color space cannot be directly converted to the Lab color space. You need to use the XYZ color space to convert the RGB color space to the XYZ color space, and then convert the XYZ color space to the Lab color space. The specific conversion process is in the prior art, and details are not repeated here.

Step S213: Calculate the color cast factor K of the lab chromaticity space. If the color cast factor K is lower than the color cast threshold, it is determined that the regional image has a color cast, otherwise, it is determined that the regional image does not have a color cast .

The color cast factor K=the average chromaticity D of the image/the chromaticity center distance M. The specific calculation process is shown in Figure 4, where M and N are the width and height of the regional image respectively, in pixels. In the above calculation process, the concept of equivalent circle is introduced, (da, db) is the center coordinate of the equivalent circle, the radius of the equivalent circle is M, and the center coordinate of the equivalent circle is to the origin of the central axis of the a-b chromaticity plane (a = 0, b = 0) is the distance D.

The larger the value of the color cast factor K, the more serious the color cast. Therefore, if the color cast factor K is lower than the color cast threshold, it is determined that the area image has color cast, otherwise, it is determined that the area image does not have color cast.

Further, the method for obtaining the color cast threshold specifically includes:

It should be noted that, for the above-mentioned "obtaining multiple images with different color casts", it is preferable to select multiple images with different color casts for each ROI area for multiple ROI areas of the yogurt area, and calculate the ROI of each ROI area. The recognition rate and color cast factor of the image are obtained, and the color cast threshold of each ROI area is obtained.

In another preferred embodiment, performing color cast detection based on HSV space on an area image specifically includes:

Step S221: Read in the area image in RGB space.

Step S222: Convert the image of the RGB color channel to the HSV channel.

The data conversion of the image between the RGB color channel and the HSV channel is in the prior art, and details are not described here.

Step S223: Perform channel segmentation on the image, and segment the image into three channel data of H, S, and V respectively.

Step S224: Calculate the average value of the H, S, and V channel data of all the pixels of the image respectively.

Calculate the average value of the H channel data, the average value of the S channel data, and the average value of the V channel data of all pixels of the image, and the three average values are used to characterize the area image in the three channels. composite value.

Step S225: If at least one of the three average values is not within the set value range, determine that the area image has color cast.

It is determined whether the three average values are all within the preset value range, and as long as there is one average value that is not within the preset value range, it is determined that there is color cast in the image in the region.

Before the refrigerator leaves the factory, the initial average value of all pixels in each area in each channel can be obtained through the captured pictures, and the value of the average value of all pixels in the three channels of the image area can be set through the initial average value. scope.

In a specific implementation manner, when no color cast occurs in an image area, the initial average values of the H, S, and V channel data of all pixels in the image area are obtained, according to the three initial average values. , and set the value range of the average value of all pixels in the three channels of the image area.

Step S300 : if the number of regions with color casts in the N regions is greater than or equal to a first threshold, it is determined that the to-be-detected image has color casts.

Since even if a color cast occurs in a part of the area, the accuracy of image recognition will be affected, therefore, the first threshold is preferably 1, that is, as long as a color cast occurs in one area, it is determined that the to-be-detected image has a color cast.

After detecting the color cast in the image, the user can be notified to remove the abnormal situation manually (for example, if the colored plastic bag blocks the camera, the user can remove the colored plastic bag). You can also add a fill light around the camera, and automatically correct the color cast by dimming the fill light. Therefore, in a preferred embodiment, the method further comprises:

Step S400: After it is determined that the image to be detected has color cast, the color cast of the image is automatically corrected by adjusting the brightness of the fill light disposed around the camera.

The fill light of the present invention preferably adopts an LED light that emits white light. When the LED light is gradually brightened, the gradually increasing white light will weaken other colored light, so as to achieve the purpose of correcting the color cast.

Specifically, the step S400 includes:

Step S410: While gradually increasing the brightness of the fill light, control the camera to periodically capture images and detect whether the captured images have color cast.

First, the brightness of the fill light is gradually increased, and at the same time, images are captured periodically (the interval between two adjacent cycles is short), and the methods of steps S100 to S300 are used to detect whether the newly captured image has a color cast.

Step S420: After detecting that the image captured by the camera has no color cast, stop increasing the brightness of the fill light.

When it is detected that the image captured by the camera has no color cast, it means that the quality of the image captured at present has reached the standard and can be used for image recognition normally. Therefore, stop increasing the brightness of the fill light. When the image recognition is complete, turn off the fill light.

In the present invention, the positions of the camera and the fill light, as shown in FIG. 3 , the yogurt special area includes a first bottle seat, the first bottle seat is used for placing yogurt, and the camera and the fill light are arranged on the second bottle seat. At the bottom, preferably the camera and the fill light are on the same level, and the fill light is a plurality of LED lights distributed around the camera according to a certain rule. Specifically, the plurality of fill lights are distributed in a rectangle or circle, and the camera is arranged in the middle of the rectangle or circle. And the multiple fill lights emit downward parallel beams, which can reduce or prevent the generation of shadows while filling the light.

Further, preferably, the distance between the fill light and the camera is in the range of 20-40 cm.

In a specific embodiment, a camera and a fill light are arranged at the bottom of the second bottle holder of the refrigerator, and the camera is used to capture an image of the yogurt special area located at the bottom of the second bottle holder, and the yogurt special area includes a yogurt area. The first bottle holder. The image captured by the camera is used as the image to be detected, and the area around the bottle holder of the image to be detected is cropped to obtain N ROI regions, and the distribution of the N ROI regions is shown in FIG. 2 . Then, the lab space-based color cast detection is performed on the N regions, and if at least one of the N regions has a color cast, it is determined that the image to be detected has a color cast. After detecting the color cast of the image to be detected, the brightness of the fill light is gradually increased, and at the same time, the image is periodically captured, and the above-mentioned color cast detection method is used to continuously detect whether the newly captured image has a color cast. After detecting that the image captured by the camera has no color cast, stop increasing the brightness of the fill light. Finally, when the image recognition is completed, turn off the fill light.

The present invention provides an image color cast detection method, by cutting out the area around the bottle holder in the image to be detected, to obtain N ROI areas, then performing color cast detection on these ROI areas, and finally according to the color cast detection results of each ROI area The color cast detection result of the image to be detected is obtained, and the detection method can quickly and accurately detect the image to be detected. And after detecting the color cast of the image, the brightness of the fill light can be automatically adjusted, and the color cast of the image can be automatically corrected, so that the accuracy of image recognition is not affected by the color cast.

The present invention also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor implements the above-mentioned detection of color cast of the image when the program is executed Any one of the steps in the method, that is, to implement the steps in any one of the technical solutions in the above-mentioned image color cast detection method.

The present invention also provides a refrigerator, which includes the above-mentioned electronic device.

Preferably, as shown in FIG. 3 , the refrigerator further includes a special area for yogurt, a first bottle seat and a second bottle seat, and a camera and a fill light are arranged at the bottom of the second bottle seat. The camera and the fill light The lights are on the same horizontal plane, and the fill light is a plurality of LED lights distributed in a rectangle around the camera, and the camera is arranged in the middle of the rectangle. And the multiple fill lights emit downward parallel beams, which can reduce or prevent the generation of shadows while filling the light.

It should be understood that although this specification is described in terms of embodiments, not every embodiment only includes an independent technical solution, and this description in the specification is only for the sake of clarity, and those skilled in the art should take the specification as a whole, and each The technical solutions in the embodiments can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

The series of detailed descriptions listed above are only specific descriptions for the feasible embodiments of the present invention, and they are not used to limit the protection scope of the present invention. Changes should all be included within the protection scope of the present invention.

Claims

A method for detecting color cast of an image, characterized in that the method comprises:

Cropping multiple ROI regions of the image to be detected to obtain N ROI regions, where N is a positive integer greater than 1;

Perform color cast detection on the N regions respectively, and obtain the color cast detection result of each area;

If the number of areas with color cast in the N areas is greater than or equal to the first threshold, it is determined that the to-be-detected image has color cast.
The method for detecting color cast of an image according to claim 1, wherein the color cast detection on an area image specifically comprises:

Read in the area image in RGB space to obtain the RGB color space of the area image;

Convert the RGB color space of the image to the lab chromaticity space;

Calculate the color cast factor K of the lab chromaticity space. If the color cast factor K is lower than the color cast threshold, it is determined that the regional image has a color cast, otherwise, it is determined that the regional image does not have a color cast.
The method for detecting color cast of an image according to claim 2, wherein the method for obtaining the color cast threshold comprises:

Obtain a plurality of images with different degrees of color cast, test and calculate their recognition rate and color cast factor K respectively, and select the value range by obtaining the value range of the color cast factor K corresponding to the image whose recognition rate exceeds the recognition rate threshold. The maximum value is used as the color cast threshold.
The method for detecting color cast of an image according to claim 1, wherein the color cast detection on an area image specifically comprises:

read in the area image in RGB space;

Convert an image with RGB color channels to HSV channels;

Perform channel segmentation on the image, and divide it into three channel data of H, S, and V respectively;

Calculate the average value of all the pixel points of the image in the H, S, V three channel data respectively;

If at least one of the three average values is not within the set value range, it is determined that there is color cast in the area image.
The method for detecting color cast of an image according to claim 1, wherein the method further comprises:

After it is determined that the image to be detected has color cast, the color cast of the image is automatically corrected by adjusting the brightness of the fill light disposed around the camera.
The method for detecting color cast of an image according to claim 5, wherein the "automatically correcting the color cast of an image by adjusting the brightness of a fill light set around the camera" specifically includes:

While gradually increasing the brightness of the fill light, control the camera to periodically capture images and detect whether the captured images have color cast;

After detecting that the image captured by the camera has no color cast, stop increasing the brightness of the fill light.
The method for detecting color cast of an image according to claim 1, wherein the method comprises:

A photographed image of a yogurt special area inside the refrigerator is obtained as an image to be detected, wherein the yogurt special area includes a bottle holder for placing yogurt.
The detection method of image color cast according to claim 5, it is characterized in that:

The N ROI regions are distributed around the bottle holder of the image to be detected.
An electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that, when the processor executes the program, any one of claims 1-8 is implemented The steps in the method for detecting color cast of an image.
A refrigerator, characterized in that, the refrigerator includes the electronic device according to claim 9 .