WO2022253014A1

WO2022253014A1 - Underwater image color restoration method and apparatus

Info

Publication number: WO2022253014A1
Application number: PCT/CN2022/094350
Authority: WO
Inventors: 苏坦
Original assignee: 影石创新科技股份有限公司
Priority date: 2021-05-31
Filing date: 2022-05-23
Publication date: 2022-12-08
Also published as: CN113411554A

Abstract

The present invention provides an underwater image color restoration method. The method comprises: S1: acquiring the global intensity of each color channel of an underwater image in an RGB color format; and S2, determining at least one compensation channel according to the relative relationship of the global intensity of each color channel, and performing, on the basis of the pixel value of each pixel of the underwater image in the compensation channel, gain compensation on the pixel in the corresponding compensation channel, wherein the global intensity of each color channel is determined by the pixel value of each color channel. Compared with the prior art, according to the present invention, color restoration is performed on each pixel in the underwater image by means of specific steps according to the relative relationship of the global intensity of each color channel of the underwater image, so that the color cast degree of the underwater image is relatively uniform, and the problems in the prior art that underwater foreground objects and background water cannot be correctly distinguished, and/or a red channel of the foreground objects is likely to overflow are solved.

Description

Underwater image color restoration method and device

technical field

The present application relates to the technical field of image processing, in particular to a color restoration method, device and computer-readable storage medium for underwater images.

Background technique

Underwater vision is not only widely used in scientific research activities such as ocean exploration and underwater engineering monitoring, but also an important source of shooting materials for photographers. Therefore, it is particularly important to obtain real underwater images.

Digital cameras on the market often encounter color cast problems when shooting underwater. Because different wavelengths of light have different attenuation rates in water, red light with longer wavelengths has the most obvious attenuation underwater. Objects in underwater shooting pictures generally lack red information, resulting in the overall picture being blue or green. At this time, the automatic white balance of the camera is prone to failure under this special lighting condition, which further leads to color distortion of the picture.

In the existing post-processing process of underwater video or pictures, manual adjustment of video color is mainly based on manpower. For example, post-editing software can be used to use manual white balance correction, channel mixer, and color search for color cast video or pictures. Correcting the surface can restore the real underwater color to a certain extent, but the manual adjustment operation is cumbersome, and when the distance between the shooting object in the picture and the camera changes, or the water depth of the shooting environment changes greatly, the picture will be blurred. The color cast will also change accordingly. At this time, it is often necessary to re-color, which is time-consuming and laborious.

In view of the above problems, Chinese Patent Publication No. CN112348904A, titled "Underwater Image and Underwater Video Color Restoration Method and Device" discloses an underwater image color restoration method. The 8-bit RGB pixel value is converted into a linear sRGB space for adjustment processing, and then the adjusted pixel value is converted to an 8-bit RGB pixel value, and then the obtained 8-bit RGB pixel value is fused with the adjusted pixel value to obtain Restored underwater image.

technical problem

However, the technical solution of the above-mentioned patent has the following defects: 1. This solution needs to calculate a weight value to represent the degree of attenuation. However, due to the changeable underwater environment, the calculation of the attenuation degree is often inaccurate, and thus the underwater foreground object and the background water cannot be correctly distinguished. 2. This solution mainly calculates the channel gain to compensate the underwater color, but since there is very little red, the calculated red gain is relatively large, which makes the red channel of the foreground object overflow easily.

Therefore, it is necessary to improve the existing underwater image color restoration methods.

technical solution

The object of the present invention is to provide an underwater image color restoration method, device, electronic equipment and computer-readable storage medium, aiming at solving the defects in the existing underwater image restoration method.

In a first aspect, the present invention provides a method for color restoration of an underwater image, the method comprising:

S1: Obtain the global intensity of each color channel of the underwater image in the RGB color format;

S2: Determine at least one compensation channel according to the relative relationship of the global intensity of each color channel, and combine the pixel value of each pixel of the underwater image in the compensation channel to perform gain compensation for the pixel in the corresponding compensation channel;

Wherein, the global intensity of each color channel is determined by the pixel value of each color channel.

In a second aspect, the present invention provides an underwater image color restoration device, which includes:

The global intensity acquisition module is used to acquire the global intensity of each color channel of the underwater image in the RGB color format;

A compensation module, configured to determine at least one compensation channel according to the relative relationship of the global intensity of each color channel, and perform gain compensation on the pixel in the corresponding compensation channel in combination with the pixel value of each pixel of the underwater image in the compensation channel;

In a third aspect, the present invention provides an electronic device, comprising:

a memory storing a computer program;

A processor, the processor is configured to execute the computer program to realize the above-mentioned underwater image color restoration method.

In a fourth aspect, the present invention provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the above is realized to realize the above-mentioned underwater image color restoration method.

technical effect

Compared with the prior art, the present invention color-colors each pixel in the underwater image according to specific steps according to the relative relationship of the global intensity of each color channel of the underwater image and the pixel value of a single pixel in each color channel. The restoration process makes the color cast of the underwater picture relatively uniform, and solves the problems in the prior art that the underwater foreground object and the background water cannot be correctly distinguished and/or the red channel of the foreground object is easy to overflow.

Description of drawings

Fig. 1 is a flow chart of the method for restoring the color of an underwater image in Embodiment 1 of the present invention.

Fig. 2 is a flowchart of step S1 in Embodiment 1 of the present invention.

Fig. 3 is a flowchart of step S2 in Embodiment 1 of the present invention.

Fig. 4 is a structural block diagram of an underwater image color restoration device in Embodiment 2 of the present invention.

Fig. 5 is a structural block diagram of an electronic device in Embodiment 3 of the present invention.

Embodiments of the present invention

In order to make the object, technical solution and beneficial effects of the present invention more clear, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

In order to illustrate the technical solutions of the present invention, specific examples are used below to illustrate.

Example 1

As shown in Figure 1, this embodiment discloses a method for color restoration of an underwater image, comprising the following steps:

S0: Detect whether the underwater image is in RGB color format, if not, convert it to RGB color format.

Underwater images refer to photos or video frames taken by the lens of a shooting device (such as a camera or mobile phone) in water. Underwater images can be in any pixel color format (such as BGR, RGB, YUV, etc.), if the underwater image is not in RGB color format, you need to convert it to RGB color format. The conversion between different color formats of images is a prior art, and will not be described in detail in this embodiment.

S1: Obtain the global intensity of each color channel of the underwater image in the RGB color format.

In a specific solution of this embodiment, step S1 includes the following sub-steps.

S11: Obtain the pixel value of each pixel in each color channel.

In this embodiment, an 8-bit RGB color space is taken as an example for illustration, and the pixel value of the acquired underwater image in each color channel is between 0-255.

S12: Perform normalization processing on each pixel value.

The pixel value of each color channel of the acquired underwater image is divided by 255, so that each pixel value becomes between 0-1, that is, the normalization process of each pixel value is completed.

S13: Determine the global intensity of each color channel according to the normalized pixel values in each color channel.

In a specific solution in this embodiment, the global intensity of each color channel is determined by averaging the pixel values of each pixel of the underwater image in each color channel. Specifically, assuming that the underwater image has n pixels, r _i , g _i , and b _i represent the pixel values of the i-th pixel in the red channel, green channel, and blue channel respectively, then the pixel average value of the red channel

Pixel average for the green channel

Pixel average for the blue channel

In another optional solution in this embodiment, the global intensity of each color channel is determined by an average value of pixel values of each pixel of the underwater image in a certain section of each color channel. Specifically, the pixel values of each pixel of the underwater image in each color channel are sorted from large to small, and then a certain proportion (such as 50%) of the pixel values in the middle is taken, and then the average value of these pixel values is calculated. In this way, the influence of the underwater picture on the global intensity of each color channel in extreme cases (such as underexposure or partial areas of the underwater image being occluded) can be reduced.

In other optional solutions in this embodiment, the global intensity of each color channel may also be a median of pixel values in each color channel for each pixel of the underwater image.

S2: Determine at least one compensation channel according to the relative relationship of the global intensity of each color channel, and combine the pixel value of each pixel of the underwater image in the compensation channel to perform gain compensation for the pixel in the corresponding compensation channel.

Generally, images taken underwater are bluish or greenish. The global intensity of each color channel is determined by comparing the average value of the pixel values or the average value of the median value of the pixel values of the underwater image in each color channel, that is, the average value or pixel value of the pixel values of the color channel The greater the average value of the median of , the greater the global intensity of that color channel. In this embodiment, step S2 includes the following sub-steps.

S21: Determine whether the global intensity of the blue channel is greater than the global intensity of the green channel, if yes, go to step S22, otherwise go to step S23.

In this embodiment, it is judged whether the global intensity of the blue channel is greater than the global intensity of the green channel by directly comparing the average pixel value b _a of the blue channel with the average value g _a of the green channel.

S22: Perform gain compensation on the green channel of each pixel in the underwater image.

In this embodiment, for any pixel in the underwater image, the calculation formula of the pixel value g' of the green channel after gain compensation is as follows:

g'=g+w _gb *(b _a -g _a )*(1-g)*b/*b _a

Among them, g is the pixel value of the pixel in the green channel, g _a is the average value of the pixel values or the median value of the pixel values of each pixel in the underwater image in the green channel, and b is the pixel value in the blue channel of the pixel. The pixel value of the color channel, b _a is the average value of the pixel values of each pixel of the underwater image in the blue channel or the average value of the median value of the pixel value, w _gb is the empirical value parameter, and the general value range is 0.2- 0.8, which means to compensate the green channel intensity according to the blue channel intensity of the pixel.

S23: Perform gain compensation on the blue channel of each pixel in the underwater image.

In this embodiment, for any pixel in the underwater image, the calculation formula of the pixel value b' of the blue channel after gain compensation is as follows:

b'=b+w _bg *(g _a -b _a )*(1-b)*g/*g _a

Among them, b is the pixel value of the pixel in the blue channel, b _a is the average value of the pixel value or the median value of the pixel value of each pixel in the underwater image in the blue channel, and g is the pixel value of the pixel The pixel value in the green channel, g _a is the average value of the pixel values or the median value of the pixel values of each pixel in the underwater image in the green channel, w _bg is the empirical value parameter, and the general value range is 0.2- 0.8, means to compensate the blue channel intensity according to the green channel intensity of the pixel.

In other solutions in this embodiment, in step S22 or step S23, gain compensation may also be performed on the red channel of each pixel in the underwater image. The compensation scheme of the red channel of the underwater image is as follows: For any pixel in the underwater image, the calculation formula of the pixel value r' of the red channel after gain compensation is as follows:

r'=r+w _rb *(b _a -r _a )*(1-r)*b/*b _a +w _rg *(g _a -r _a )*(1-r)*g/*g _a

Among them, r, b, g are the pixel values of the pixel in the red channel, blue channel and green channel respectively, and r _a , b _a , g _a are the underwater image pixels in the red channel, blue channel and green channel respectively. The average value of the pixel value of the channel or the average value of the median value of the pixel value, w _rb and w _rg are empirical value parameters, respectively indicating the intensity of red compensation according to the intensity of the green channel and the intensity of the blue channel of the pixel.

It should be noted that, the above compensation for the red channel is generally performed when the global intensity of the blue channel or the global intensity of the green channel of the underwater image is maximum. Of course, when the global intensity of the red channel of the underwater image is the largest, the red channel can also be compensated. From the above formula, it can be known that in this case, the compensation for the red channel is negative compensation, and the compensated red pixel The value becomes smaller, making red objects in underwater images lighter.

In addition, due to the different color adjustments of underwater shooting equipment, even in the same underwater scene, the pictures captured by different equipment will also be different. Therefore, a person skilled in the art can adjust the four empirical parameters w _gb , w _bg , w _rb , and w _rg to make the color of a picture captured by a certain type of camera more natural.

It can be known from the above that the global intensity of each color channel is determined by the pixel value of each color channel.

After the processing of the above steps S1 and S2, the problem of the greenish or bluish picture of the underwater image has been solved, but the color of the water has changed from the original blue or green to light blue or light green close to gray, some The scene is still not realistic enough. In order to make the color of the water closer to the blue of the seawater seen by the naked eye, in the optimization scheme of this embodiment, it also includes step S3: adding saturation processing and/or dark channel defogging algorithm to the underwater image after gain compensation deal with. Wherein, the specific method for increasing the saturation is not limited, and those skilled in the art can choose a scheme for increasing the saturation by themselves. In addition, using the dark channel dehazing algorithm is also a common technique in image processing, and will not be described in detail here. It should be pointed out that if the dark channel dehazing algorithm is directly used without the compensation operation of the color channel, the seawater area where the red channel is seriously missing will be lost. Foreground regions that are misidentified as non-foggy cannot get ideal results.

In the further optimization solution of this embodiment, step S4 is further included: adjusting the gain-compensated underwater image through a three-dimensional color lookup table. Specifically, it is also possible to apply a fixed three-dimensional color look-up table (3D Look-up Table, 3D LUT) to the picture of the underwater image processed in step S2 or S3, by loading the three-dimensional color look-up table on the image or video processing software Table, making the picture color of underwater images more realistic and natural. It needs to be added that if the color channel compensation operation in steps S1 and S2 is not performed, there will be different color casts in different underwater pictures, and the degree of blue cast or green cast will not be uniform. In this case, loading fixation is not applicable. The 3D color look-up table for underwater image processing.

Example 2

An underwater image color restoration device includes a global intensity acquisition module, a compensation module, an RGB conversion module, a saturation processing module, a dark channel defogging module and a loading module.

Specifically, the global intensity acquisition module is used to acquire the global intensity of each color channel of the underwater image in the RGB color format; the compensation module is used to determine at least one compensation channel according to the relative relationship of the global intensity of each color channel, and combine the underwater The pixel value of each pixel of the image in the compensation channel performs gain compensation on the pixel in the corresponding compensation channel; the RGB conversion module is used to convert the underwater image from other color spaces to RGB color format; the saturation processing module is used to adjust the gain The compensated underwater image is increased in saturation; the dark channel defogging module is used to perform dark channel defogging algorithm processing on the gain-compensated underwater image; the loading module is used to load a three-dimensional color lookup table for gain-compensated underwater images The image is processed.

Wherein, the global intensity of each color channel is determined by the pixel value of each color channel, and the specific implementation manner of the determination may refer to the relevant description in Embodiment 1, and no further description is given here.

Example 3

This embodiment discloses an electronic device, including a memory and a processor, and a computer program is stored on the memory; the processor is used to execute the computer program to realize the underwater image color restoration method in Embodiment 1. The electronic device in this embodiment may specifically be a camera or a mobile phone.

Example 4

This embodiment discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the underwater image color restoration method in Embodiment 1 is implemented.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the storage medium can be a computer-readable storage medium, for example, a ferroelectric memory (FRAM , Ferromagnetic Random Access Memory), Read Only Memory (ROM, Read Only Memory), Programmable Read Only Memory (PROM, Programmable Read Only Memory), Erasable Programmable Read Only Memory (EPROM, Erasable Programmable Read Only Memory), Electrically Erasable Programmable Read Only Memory (EEPROM, Electrically Erasable Programmable Read Only Memory), flash memory, magnetic surface memory, optical disc, or CD-ROM (Compact Disk-Read Only Memory) and other memories; it can also be Various devices including one or any combination of the above memories.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. within range.

Claims

A method for color restoration of an underwater image, characterized in that it comprises:

S1: Obtain the global intensity of each color channel of the underwater image in the RGB color format;

S2: Determine at least one compensation channel according to the relative relationship of the global intensity of each color channel, and combine the pixel value of each pixel of the underwater image in the compensation channel to perform gain compensation for the pixel in the corresponding compensation channel;

Wherein, the global intensity of each color channel is determined by the pixel value of each color channel.
The underwater image color restoration method according to claim 1, wherein the global intensity of each color channel is: (1) the average value of the pixel values of each pixel of the underwater image in each color channel, or ( 2) The average value of the pixel values of each pixel of the underwater image in a certain section of each color channel, or (3) the median of the pixel values of each pixel of the underwater image in each color channel.
The underwater image color restoration method according to claim 1, characterized in that, before the step S1, comprising:

S0: Detect whether the underwater image is in RGB color format, if not, convert it to RGB color format.
The underwater image color restoration method according to claim 1, wherein said step S1 comprises:

S11: Obtain the pixel value of each pixel in each color channel;

S12: Perform normalization processing on each pixel value;

S13: Determine the global intensity of each color channel according to the normalized pixel values in each color channel.
The underwater image color restoration method according to claim 1, wherein said step S2 comprises:

S21: Determine whether the global intensity of the blue channel is greater than the global intensity of the green channel, if yes, proceed to step S22, otherwise proceed to step S23;

S22: performing gain compensation on the green channel of each pixel in the underwater image;

S23: Perform gain compensation on the blue channel of each pixel in the underwater image.
The underwater image color restoration method according to claim 5, wherein the step S22 is specifically:

For any pixel in the underwater image, the calculation formula of the pixel value g' of the green channel after gain compensation is as follows:

g'=g+w gb *(b a -g a )*(1-g)*b/*b a

Among them, g is the pixel value of the pixel in the green channel, and g a is the average value of the pixel values of each pixel in the underwater image in the green channel, the average value of the pixel values in a certain section, or the median of the pixel values b is the pixel value of the pixel in the blue channel, b a is the average value of the pixel values of each pixel in the underwater image in the blue channel, the average value of the pixel values in a certain section, or the pixel value of the pixel value Median, w gb is an empirical value parameter, indicating that the intensity of the green channel is compensated according to the intensity of the blue channel of the pixel.
The underwater image color restoration method according to claim 5, wherein the step S23 is specifically:

For any pixel in the underwater image, the calculation formula of the pixel value b' of the blue channel after gain compensation is as follows:

b'=b+w bg *(g a -b a )*(1-b)*g/*g a

Among them, b is the pixel value of the pixel in the blue channel, b a is the average value of the pixel values of each pixel in the underwater image in the blue channel, the average value of the pixel values in a certain section, or the pixel value of the pixel value The median, g is the pixel value of the pixel in the green channel, g a is the average value of the pixel values of each pixel in the underwater image in the green channel, the average value of the pixel values in a certain section, or the pixel value of the pixel value Median, w bg is an empirical value parameter, indicating that the intensity of the blue channel is compensated according to the intensity of the green channel of the pixel.
The underwater image color restoration method according to claim 5, wherein the step S22 or step S23 further comprises performing gain compensation on the red channel of each pixel in the underwater image.
The underwater image color restoration method according to claim 8, wherein the gain compensation for the red channel of each pixel in the underwater image is specifically:

For any pixel in the underwater image, the calculation formula of the pixel value r′ of the red channel after gain compensation is as follows:

r'=r+w rb *(b a -r a )*(1-r)*b/*b a +w rg *(g a -r a )*(1-r)*g/*g a

Among them, r, b, g are the pixel values of the pixel in the red channel, blue channel and green channel respectively, and r a , b a , g a are the underwater image pixels in the red channel, blue channel and green channel respectively. The average value of the pixel value of the channel, the average value of the pixel value in a certain section or the median value of the pixel value, w rb and w rg are empirical value parameters, which respectively represent the intensity of the green channel and the blue channel according to the pixel Intensity to compensate for red intensity.
The underwater image color restoration method according to claim 1, further comprising step S3: performing saturation increase processing and/or dark channel defogging algorithm processing on the gain-compensated underwater image.
The underwater image color restoration method according to any one of claims 1 to 8, further comprising step S4: adjusting the gain-compensated underwater image through a three-dimensional color lookup table.
An underwater image color restoration device is characterized in that it comprises:

The global intensity acquisition module is used to acquire the global intensity of each color channel of the underwater image in the RGB color format;

A compensation module, configured to determine at least one compensation channel according to the relative relationship of the global intensity of each color channel, and perform gain compensation on the pixel in the corresponding compensation channel in combination with the pixel value of each pixel of the underwater image in the compensation channel;

Wherein, the global intensity of each color channel is determined by the pixel value of each color channel.
The underwater image color restoration device according to claim 12, further comprising:

The RGB conversion module is used to convert the underwater image from other color spaces to RGB color format.
The underwater image color restoration device according to claim 12, further comprising:

A saturation processing module and/or a dark channel defogging module, the saturation processing module is used to increase the saturation of the underwater image after gain compensation, and the dark channel defogging module is used to correct the underwater image after gain compensation The image is processed by dark channel dehazing algorithm.
The underwater image color restoration device according to claim 12, further comprising:

A loading module, the loading module is used to load a three-dimensional color lookup table to process the gain-compensated underwater image.
An electronic device, characterized in that it comprises:

a memory storing a computer program;

A processor, the processor is used to execute the computer program to realize the underwater image color restoration method according to any one of claims 1 to 11.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the underwater image according to any one of claims 1 to 11 is realized Color reproduction method.