CN114708180A - Bit depth quantization and enhancement method for pre-distorted image with dynamic range preservation - Google Patents

Abstract

The invention discloses a bit depth quantization and enhancement method for a pre-distorted image with dynamic range maintenance, and belongs to the technical field of image processing. Firstly, normalizing an input high-bit image, then calculating a predistortion template image and adjusting the resolution of the predistortion template image to obtain a predistortion image, then adding the normalized high-bit image and the predistortion image pixel by pixel, then calculating a quantization function with a dynamic range retention characteristic according to the bit depth of the low-bit image to quantize the predistortion high-bit image, finally taking the low-bit image as an input image of a network, and performing bit depth enhancement on the low-bit image through a convolutional neural network to obtain the high-bit image. The quantization method with dynamic range preservation can inhibit the blurring of over-bright or over-dark areas in the reconstructed image and improve the image reconstruction quality.

Description

Bit-depth quantization and enhancement method for predistorted images with dynamic range preservation

technical field

The invention belongs to the technical field of image processing, and in particular relates to a bit depth enhancement method of predistorted images with dynamic range preservation.

Background technique

As a modern mainstream information carrier, images bring convenience to daily life, but also bring great challenges to storage and transmission equipment. And in some application scenarios with asymmetric system complexity such as wireless sensor networks, that is, the computing resources on the data receiving end are not limited, and the computing resources on the data transmitting end are limited, so it is very important to reduce the amount of data on the image transmitting end. necessary. By converting a high-bit image (that is, a high-bit-depth image, the bit depth of the image is the grayscale range that can be expressed by each channel of the pixel, for example, an image with a bit depth of n can represent 2n grayscales, and the color representation is also more delicate, The visual experience of the image is also better) Quantization to a low-bit image can achieve simple compression of the image, however, reducing the bit-depth of the image will bring about the problems of color distortion and false contours, which will seriously affect the viewing experience. Increasing the quantization step size can improve the compression efficiency, but it will introduce more serious false contours and color distortions, so that even the best performing neural network methods cannot guarantee effective suppression of these two artifacts. For this reason, the joint reconstruction method obtains a low-bit image with less artifacts by applying predistortion when generating a low-bit image, which is conducive to reconstructing a high-quality high-bit image. However, due to the loss of dynamic range in the existing quantization methods, the boundaries of the over-bright and over-dark regions of the high-bit image generated by the joint reconstruction method based on deep learning are obviously blurred, which affects the subjective visual experience.

SUMMARY OF THE INVENTION

The purpose of the present invention is to propose a bit depth quantization method with dynamic range preservation, which can further improve the bit depth enhancement performance, realize the suppression of the obvious blurring of the reconstructed high-bit image in the areas that are too bright and too dark, and make the reconstructed high-bit image To achieve the best subjective and objective quality.

The technical scheme adopted in the present invention is:

A predistorted image bit depth quantization and enhancement method with dynamic range preservation, comprising the following steps:

Step S1: Quantize the high-bit image:

Step S101: normalize the input high-bit image: divide the gray value of each pixel by the maximum gray value of the high-bit image;

Calculated based on the bit depth 1 of the quantization target, the quantization step size Q= ¹ /(21-1), that is, the high-bit image will obtain a low-bit image with a bit depth of 1 after being quantized;

Step S102: Calculate and obtain a predistortion function (Planned Sensor Distortion, PSD) template image according to the quantization step size Q;

Step S102: performing non-overlapping tiling processing on the obtained template image to obtain a predistorted image with an image resolution consistent with the high-bit image;

Step S103: adding the predistorted image and the normalized high-bit image pixel by pixel to obtain a pre-distorted high-bit image;

Step S104: according to the quantization step size Q, a quantization mode with dynamic range retention is set, and the predistorted high-bit image is quantized to obtain a low-bit image;

Step S2: establishing a mapping relationship between low-bit images and high-bit images through a convolutional neural network to obtain an image bit depth enhancement network; using the low-bit image obtained in step S1 as the input of the image bit depth enhancement network, based on its output An enhanced image is obtained, that is, a reconstructed high-bit image.

Further, in the step S104, the quantization method with dynamic range preservation is specifically:

If the gray value of the current pixel is less than or equal to Q/2, the gray value of the current pixel is quantized to 0;

If the gray value of the current pixel is greater than 1-Q/2, the gray value of the current pixel is quantized to 1;

If the gray value of the current pixel is located in [Q/2,1-Q/2], divide the value range [Q/2,1-Q/2] into multiple sub-intervals, and for each sub-interval, based on the midpoint Get the quantized value of the current subinterval.

Further, in the step S104, for grayscale values whose value range is [Q/2, 1-Q/2], the new quantization step size Δ=(1-Q)/(2 ^l -2 during quantization ), the quantization value of each sub-interval is [Q+(2k+1)Δ]/2, where k=0, 1, . . . 2 ^l -3.

Further, in S102, the template image is 3×3, and in any 3×3 neighborhood, it is {Q(1-L)/2L, Q(3-L)/2L,...,Q(L-1) /2L}, where L=(2n+1) ² , n=1.

That is, in the present invention, the adopted quantization method with dynamic range preservation belongs to a non-uniform quantization method, because the gray value range of the high-bit image processed by the predistorted template image may be less than 0 and greater than 1, respectively. Q/2 outside the specified range [0,1], so the new value range is modified to [-Q/2,1+Q/2]. When the image is quantized by ¹ bit, there are 21 quantization intervals. When the predistorted high-bit image is in the range of less than or equal to Q/2, the gray value is quantized to 0. When the predistorted high-bit image is greater than 1 -Q/2 grayscale range is quantized to 1. When the predistorted high-bit image is in the middle range [Q/2, 1-Q/2], the range is first uniformly quantized to obtain a new quantization step size Δ=(1-Q)/(2 ⁿ -2), the midpoint of each interval is taken as the quantized value, that is, [Q+(2k+1)Δ]/2. The predistorted high-bit image is quantized in this way to obtain a low-bit image. This process can be expressed as a function as follows:

Among them, _IL represents the quantized low-bit image, and I _PSD represents the predistorted high-bit image.

The technical scheme provided by the present invention brings at least the following beneficial effects:

The traditional uniform quantization method such as rounding and rounding down on the predistorted high-bit image will bring about the loss of dynamic range. Uniform quantization achieves the purpose of maintaining the dynamic range, thereby suppressing the blurring of the reconstructed high-bit image in over-bright and over-dark areas.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic diagram of a processing process of a predistorted image bit depth quantization and enhancement method with dynamic range preservation provided by an embodiment of the present invention;

2 is a schematic diagram of a predistorted template image in an embodiment of the present invention;

FIG. 3 is a schematic diagram of quantization processing in an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

The embodiment of the present invention provides a predistorted image bit depth quantization and enhancement method with dynamic range preservation. The PSD-based bit depth enhancement algorithm belongs to the joint reconstruction algorithm, and considers the generation process of the low-bit image and the high-bit image at the same time. In the reconstruction process, the image is pre-distorted before quantization to construct effective multi-observation of the signal, which is conducive to the reconstruction of the bit depth by the convolutional neural network in the subsequent stage and improves the quality of image reconstruction. When applied to image transmission processing, the quantization processing provided by the embodiment of the present invention can be used to compress the high-bit image, and then perform image transmission processing. The high-bit image is enhanced to achieve high-quality high-bit image reconstruction at the receiving end, so as to realize the display conversion processing of the received low-bit image in the high-bit display.

As shown in FIG. 1 , the specific implementation process of the predistorted image bit depth quantization and enhancement method with dynamic range preservation provided by the embodiment of the present invention includes: normalization processing of high-bit images, predistortion function template image (predistortion template image) image) calculation, acquisition of predistorted image, calculation of quantization function, quantization of predistorted image, bit depth reconstruction based on neural network.

Step 1: Normalize the high-bit image I _H (that is, the image with the first image data format), according to the low-bit image (the image with the second image data format, wherein the image bit depth of the second image data format) The quantization step size is obtained by calculating the bit depth lower than the first image data format) _IL . In this embodiment, assuming that the bit depth of the high-bit image is 16 and the bit depth of the low-bit image is 2, then the 16-bit bit depth can represent The maximum gray value of 2 ¹⁶ -1, after normalization, is I' _H =I _H /(2 ¹⁶ -1), and the quantization step size is Q=1/2 ² =0.25;

Step 2: Calculate the predistorted template image according to the quantization step size Q=0.25. The predistorted template image is {-4/9Q, -3/9Q, -2/9Q, -1/9Q in any 3×3 neighborhood ,0,1/9Q,2/9Q,3/9Q,4/9Q}, that is, the 9 pixel values are randomly arranged in a 3×3 template. As a preferred way, it is the arrangement shown in Figure 2, that is, the gray value of the center point of the predistorted template image is 0, and the gray value of the adjacent pixel points of the center point is distributed symmetrically about the center point, And the values of the two symmetrical pixels are opposite;

Step 3: Assuming that the resolution of the high-bit image is M×N, before the high-bit image is quantized into a low-bit image, the pre-distorted template image calculated in step 2 is flattened from left to right and from top to bottom without overlap and omission. to obtain a predistorted image with a resolution size of M×N, set it as δ ^psd , and add it pixel-by-pixel with the high-bit image I _H , which is expressed as I _PSD =I _H +δ ^psd ;

As a possible implementation, in this step, when acquiring the predistorted image, the predistorted template image can be tiled from left to right and top to bottom without overlapping and omission, and the size of the tiled image is larger than Or equal to M×N. If the size of the tiled image exceeds M×N, the redundant part is cropped to obtain an M×N predistorted image.

As a possible implementation, a sliding window can also be used to obtain an M×N predistorted image. The size of the sliding window is the same as that of the predistorted template image, and the step size is the side length of the predistorted template image. Within the image range of N, the pixels of the predistorted template image are filled pixel by pixel each time they slide once, and when they slide to the very end each time, the window beyond the image range is skipped, that is, no pixel filling is performed, thus obtaining MxN predistorted image.

Step 4: Design a quantization function with dynamic range preservation according to the quantization step size Q=0.25 as follows:

Among them, _IL represents the quantized image. This is a non-uniform quantization method. Different quantization methods are used for the two ends and the middle area. Since the gray value range of the high-bit image processed by the predistorted template image will be less than 0 and greater than 1, respectively exceeding the specified It is 0.125 outside the range [0,1], so the new value range is modified to [-0.125,1.125]. The 2-bit quantization divides the entire interval into 4 segments, and quantizes the predistorted high-bit image in this way to obtain the low-bit image _IL . The schematic diagram of quantization is shown in FIG. 3 .

Step 5: At the image receiving end, a non-linear mapping relationship between a 2-bit predistorted low-bit image and a 16-bit high-bit image is established through a convolutional neural network.

The convolutional neural network may adopt any conventional network structure in the field, which is not specifically limited in the embodiment of the present invention. For high-bit images with good quality, the evaluation methods for image quality mainly include peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). The calculation formula is as follows.

表示通过网络重构得到的16bit图像。PSNR越大，表示比特深度重建图像质量越好，反之则越差。Among them, I _H represents the original 16bit image,

Represents a 16-bit image reconstructed by the network. The larger the PSNR, the better the bit-depth reconstructed image quality, and vice versa.

的均值，

和

分别代表原始图像和恢复图像之间的方差，σ_xy代表I_H和

的协方差，C₁和C₂为常数。根据公式可知，若比特深度重建算法得到的高比特图像与原始高比特图像间差别越小，输出图像质量越好，证明算法效果也越好。Among them, μ _x represents the mean value of the original high-bit image I _H , μ _y represents the high-bit image after bit depth reconstruction

the mean of ,

and

represent the variance between the original image and the restored image, respectively, σ _xy represent I _H and

The covariance of , C ₁ and C ₂ are constants. According to the formula, if the difference between the high-bit image obtained by the bit depth reconstruction algorithm and the original high-bit image is smaller, the quality of the output image will be better, which proves that the effect of the algorithm is also better.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

The foregoing are merely some of the embodiments of the present invention. For those of ordinary skill in the art, without departing from the inventive concept of the present invention, several modifications and improvements can be made, which all belong to the protection scope of the present invention.

Claims (4)

1. a predistorted image bit depth quantization and enhancement method with dynamic range retention, is characterized in that, comprises the following steps: Step S1: Quantize the high-bit image: Step S101: normalize the input high-bit image: divide the gray value of each pixel by the maximum gray value of the high-bit image; Calculated based on the bit depth l of the quantization target, the quantization step size Q=1/(2 ^l -1); Step S102: Calculate and obtain a predistortion function template image according to the quantization step size Q; Step S102: performing non-overlapping tiling processing on the obtained template image to obtain a predistorted image with an image resolution consistent with the high-bit image; Step S103: adding the predistorted image and the normalized high-bit image pixel by pixel to obtain a pre-distorted high-bit image; Step S104: according to the quantization step size Q, a quantization mode with dynamic range retention is set, and the predistorted high-bit image is quantized to obtain a low-bit image; Step S2: establishing the mapping relationship between the low-bit image and the high-bit image through the convolutional neural network to obtain an image enhancement network; using the low-bit image obtained in step S1 as the input of the image enhancement network, and obtaining the enhanced image based on its output. image. 2. The method of claim 1, wherein, in the step S104, the quantization method with dynamic range preservation is specifically: If the gray value of the current pixel is less than or equal to Q/2, the gray value of the current pixel is quantized to 0; If the gray value of the current pixel is greater than 1-Q/2, the gray value of the current pixel is quantized to 1; If the gray value of the current pixel is located in [Q/2,1-Q/2], divide the value range [Q/2,1-Q/2] into multiple sub-intervals, and for each sub-interval, based on the midpoint Get the quantized value of the current subinterval. 3 . The method according to claim 2 , wherein, in the step S104 , for grayscale values whose value range is [Q/2, 1-Q/2], the new quantization step size Δ during quantization is Δ =(1-Q)/(2 ^l -2), the quantization value of each sub-interval is [Q+(2k+1)Δ]/2, where k=0, 1, . . . 2 ^l -3. 4. The method according to any one of claims 1 to 3, wherein in the step S102, the template image is 3×3, and in any 3×3 neighborhood, it is {Q(1-L) The complete set of /2L,Q(3-L)/2L,...,Q(L-1)/2L}, where L=9.

Images