CN115660992A - Local backlight dimming method, system, device and medium - Google Patents
Local backlight dimming method, system, device and medium Download PDFInfo
- Publication number
- CN115660992A CN115660992A CN202211357097.2A CN202211357097A CN115660992A CN 115660992 A CN115660992 A CN 115660992A CN 202211357097 A CN202211357097 A CN 202211357097A CN 115660992 A CN115660992 A CN 115660992A
- Authority
- CN
- China
- Prior art keywords
- backlight
- dimming
- blu
- image
- initial
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 28
- 239000004973 liquid crystal related substance Substances 0.000 claims abstract description 24
- 238000002834 transmittance Methods 0.000 claims abstract description 16
- 238000012545 processing Methods 0.000 claims abstract description 12
- 238000013528 artificial neural network Methods 0.000 claims abstract description 11
- 238000009499 grossing Methods 0.000 claims description 22
- 230000002146 bilateral effect Effects 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000011176 pooling Methods 0.000 claims description 11
- 238000001914 filtration Methods 0.000 claims description 10
- 230000006870 function Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 8
- 238000010276 construction Methods 0.000 claims description 4
- 238000005070 sampling Methods 0.000 claims description 3
- 230000004913 activation Effects 0.000 claims description 2
- 238000012888 cubic function Methods 0.000 claims description 2
- 238000002372 labelling Methods 0.000 claims description 2
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000002054 transplantation Methods 0.000 abstract description 2
- 230000000694 effects Effects 0.000 description 13
- 238000009792 diffusion process Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 238000002474 experimental method Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 238000005259 measurement Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 2
- 230000003321 amplification Effects 0.000 description 2
- 230000000903 blocking effect Effects 0.000 description 2
- 230000015556 catabolic process Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 125000001475 halogen functional group Chemical group 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000007796 conventional method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000013508 migration Methods 0.000 description 1
- 230000005012 migration Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Landscapes
- Liquid Crystal Display Device Control (AREA)
Abstract
The invention discloses a local backlight dimming method, a local backlight dimming system, local backlight dimming equipment and a local backlight dimming medium, and relates to the fields of image processing, artificial intelligence algorithms and liquid crystal displays. The local backlight dimming method comprises the following steps: dividing the backlight unit into a fixed number of backlight blocks; inputting the original image into a simple convolutional neural network, and outputting normalized backlight intensity BLU initial backlight; carrying out fuzzy processing on the initial BLU backlight; amplifying the BLU initial backlight after the fuzzy processing to obtain an integral backlight; modeling a dimming image according to the overall backlight, the dimming transmittance and the light leakage coefficient; and calculating a loss value between the original image and the dimming image, estimating the luminous intensity of the backlight blocks with fixed quantity, finishing the training of the simple convolution neural network, and obtaining the backlight dimming scheme. The method has the advantages of less parameters needing training, obviously reduced calculated amount and delay, simple network, obviously reduced learning cost and simpler training and hardware transplantation.
Description
Technical Field
The invention relates to the field of image processing, artificial intelligence algorithm and liquid crystal display, in particular to a local backlight dimming method, a system, equipment and a medium.
Background
Currently, a Liquid Crystal Display (LCD) has great advantages in terms of life, durability, price, and the like, and thus it has a large market share. However, the LCD is not a self-luminous device, and needs a backlight to display contents, and according to the characteristics of the liquid crystal, the backlight brightness of the conventional LCD cannot be reduced when displaying a low-brightness image, and a part of light passes through the liquid crystal to be blocked, so that light leakage artifacts are formed, and the display quality of the image is reduced. In addition, since the power consumption of the backlight module is the largest in the overall power consumption of the LCD, the conventional LCD may cause a great energy waste.
In view of the above problems, a backlight dimming technique has been developed, which dims a backlight in a dark area and, in turn, compensates for LC transmittance in inverse proportion to a dimming ratio so that an image displayed is less distorted while reducing the influence of leakage characteristics. Nowadays, light Emitting Diodes (LEDs) have replaced Cold Cathode Fluorescent Lamps (CCFLs) as backlight units (BLUs). The BLU is typically divided into blocks so that we can control the intensity of the LEDs in each block locally and individually depending on the image content. This strategy, known as Local Backlight Dimming (LBD), greatly improves contrast while reducing power consumption. However, existing LBDs also often suffer from some degradation, such as loss of detail and halo artifacts. In addition, most of the existing local backlight dimming methods are manual feature extraction or rely on artificial neural networks, and the methods need a large amount of calculation and are difficult to realize on hardware.
Chinese patent application No. CN201910895951.2, published 2019, 9/21, discloses a dynamic dimming backlight diffusion method for image processing based on deep learning, which adopts a regional backlight extraction algorithm to extract backlight brightness of a sample image; inputting the backlight brightness to a backlight diffusion model based on deep learning for backlight brightness diffusion, outputting a backlight brightness diffusion image, multiplying the backlight brightness diffusion image with a compensation image corresponding to the input image to obtain a development image, determining an error between the development image and a sample image, and updating the backlight diffusion model by using the error to obtain a final backlight diffusion model. The backlight diffusion model is established by utilizing the neural network, and the reliable diffusion model without actual measurement is provided for the theoretical research of the dynamic dimming algorithm. However, the method proposed in the patent adopts a traditional regional backlight extraction algorithm to extract backlight brightness from an original image, and cannot consider the problem of light diffusion at adjacent positions when determining backlight; in addition, in the backlight diffusion model, three convolution blocks and four residual blocks are used, each residual block comprises two convolution blocks, the network scale is large, parameters needing to be trained are numerous, and therefore when a large-size image is processed, storage resources are consumed greatly, and effective implementation on hardware is difficult.
Disclosure of Invention
1. Technical problem to be solved
The invention aims to overcome the defects of the prior art, solve the problems that the traditional local backlight dimming method lacks generalization capability, has huge network, is complex in calculation and is difficult to effectively realize on hardware, learn characteristics by adopting a simple convolutional neural network to find the optimal backlight intensity, and provide a local backlight dimming method, a system, equipment and a medium.
2. Technical scheme
The purpose of the invention is realized by the following technical scheme.
The method is realized by the following technical scheme:
in a first aspect, a local backlight dimming algorithm is proposed, which divides a whole backlight unit (BLU) into a fixed number of backlight blocks for local dimming control; using a simple convolutional neural network for an original image, and finally using a sigmoid function to take output as normalized initial backlight intensity of a BLU (block white) after 1 expansion convolutional layer, 4 maximum pooling layers and 1 full-link layer; the network structure is small in size, hardware implementation is easy to achieve, meanwhile, influences of surrounding pixel points can be considered, and experiments prove that the network structure has a good effect. Then, using bilateral smoothing operation and twice bicubic interpolation operation on the BLU initial backlight to perform fuzzy processing on the BLU initial backlight and perform interpolation amplification until the size of the BLU initial backlight is equal to that of the original image to obtain integral backlight; then, a dimming transmissivity compensator is created to compensate the original image, and when the backlight is too dark, the dimming image and the original image keep the same brightness value, so that the compensated liquid crystal transmissivity after the backlight dimming is obtained; finally, considering the problem of light leakage, and completing modeling of a dimming image according to an original image, BLU integral backlight, and the compensated liquid crystal transmissivity and light leakage coefficient after backlight dimming; and calculating a loss value between the original image and the dimming image, estimating the luminous intensity of the backlight blocks BLU with a fixed number under the constraint condition that the output dimming image is infinitely close to the input image, and training the simple convolutional neural network. And finally obtaining a backlight dimming scheme after the training is finished.
The method specifically comprises the following steps:
dividing the entire BLU into a fixed number of backlight blocks, which are the minimum units to individually control the intensity of the LEDs;
inputting an original image into a simple convolutional neural network, outputting a BLU initial backlight of the BLU normalized backlight intensity, and controlling LED backlight display by the BLU initial backlight;
carrying out fuzzy processing on the initial BLU backlight by using smoothing operation;
carrying out interpolation operation on the BLU initial backlight after the smoothing operation, and amplifying the BLU initial backlight to be equal to the size of the original image to obtain the BLU integral backlight;
establishing a dimming transmissivity compensator, and performing dimming transmissivity compensation on the original image according to the BLU integral backlight to obtain the compensated liquid crystal transmissivity after backlight dimming;
performing pixel compensation on the original image according to the BLU integral backlight, the compensation liquid crystal transmissivity and the light leakage coefficient after backlight dimming, and completing modeling of a dimming image;
calculating a loss value between an original image and a dimming image, estimating the luminous intensity of a fixed number of backlight blocks BLU under the constraint condition that the output dimming image is infinitely close to the input image, training the simple convolutional neural network, and optimizing the parameters of a simple convolutional neural network model;
calculating data of the forward propagation of the input image to the nodes of the output layer in each iteration, wherein the output data is BLU initial backlight, smoothing and interpolation calculation are carried out on the BLU initial backlight, pixel compensation is carried out on the original image, and finally a dimming image is output; calculating gradient and then performing back propagation by calculating loss values of the dimming image and the original image, and updating the network model parameter weight w and the bias value bias in the process of back propagation; obtaining stable network model parameters finally through multiple iterations, storing the final network model parameters, completing model construction after the training process is finished, and obtaining an overall backlight dimming scheme;
the overall scheme of backlight dimming includes a calculation method of a BLU initial backlight output by a simple convolutional neural network and a BLU overall backlight.
In some realizations of the first aspect, the whole backlight unit (BLU) is divided into 10 × 16 backlight blocks for local dimming control, in analog training, RGB images of 960 × 1536 pixels are used, i.e. each backlight block covers 96 × 96 pixels;
in some implementations of the first aspect, the initial backlight of the BLU is obtained by using a simple convolutional neural network for the original image, normalizing the generator output to between [0,1] using a sigmoid function through 1 expansion convolutional layer, 4 maximum pooling layers, and 1 full-link layer, and taking the output as the normalized backlight intensity of the BLU.
The operations in the convolutional layer may be represented as:
h out ＝σ(W*h in +bias)
where W and bias represent weight and bias, respectively, and the output of the convolutional layer is input h in The result of the convolution with the weight W is then added to the bias and then the nonlinear activation function σ is used. By using one layer of CNN, the influence of neighboring images can be taken into account, enabling extremely variable modes; and then, the maximum pooling layers with different step sizes are used, the value with the highest pixel value is continuously extracted, and more parameters do not need to be trained, so that the training and the migration on hardware are simpler.
In some implementations of the first aspect, the blurring process is performed using a bilateral smoothing operation on the BLU initial backlight to simulate the effect of the optical diffuser film.
Since the LED is a discrete light source, a multilayer optical diffuser film is needed to disperse the LED light. This allows the illumination to be spread over the area as a planar light source, thereby reducing blocking artifacts around the backlight block. To simulate the effect of an optical diffuser film, the present invention uses a bilateral smoothing operation. Unlike gaussian smoothing, bilateral filter smoothing can keep the edges of the image unaffected while performing blurring. Bilateral filtering is a nonlinear filtering method, is a compromise treatment combining the spatial proximity and the pixel value similarity of an image, and simultaneously considers the spatial information and the gray level similarity to achieve the purpose of edge-preserving and denoising. In local backlight dimming, gaussian filtering is usually used, which results in a relatively obvious blurred edge, and the protection effect on high-frequency details is poor, and bilateral filtering can solve the problem well. The specific operation is as follows:
wherein,
wherein, b blur And b dim Respectively representing the dimmed backlight intensity and the initial backlight intensity at the (c, r) -th block. c ∈ {1, 2., 10} and r ∈ {1,2, \8230;, 16} respectively represent the column index and row index of the block, d is the pixel neighborhood diameter, k is the row of neighboring pixels, and l is the column index of neighboring pixels. p (x) 0 ,y 0 ) Represents the pixel value of the center point, and p (x, y) represents the pixel value at a distance (x, y) from the center point. A larger value of the labeling variance δ in coordinate space means that farther pixels will interact so that larger regions of sufficiently similar color capture the same color. In this patent, in order to optimize the smoothing effect, a compromise σ and a diameter d of a pixel neighborhood are selected, a bilateral filter with d =7 and δ =1.5 is used, and values of row-column indexes k and l of adjacent pixels are [3,3 ]]。
In some implementations of the first aspect, two bicubic interpolation operations are used to enlarge the BLU initial backlight to be equal in size to the original image, resulting in a BLU overall backlight. Bicubic interpolation is an interpolation that creates smoother image edges than bilinear interpolation. The calculation method comprises the following steps:
wherein alpha is ij In order to calculate the coefficient, the value of which depends on the characteristics of the interpolation data, since the basis function of bicubic interpolation is one-dimensional and the pixel is two-dimensional, the row and column of the pixel are calculated separately, and 4 rows (i =0,1,2, 3) and 4 columns (j =0,1,2, 3) nearest to the BLU whole backlight pixel (x, y) to be calculated are selected for calculation. And p (x, y) is a pixel value of the BLU overall backlight pixel point coordinate (x, y) to be calculated. In this method, the pixel value of point (x, y) can be obtained by a weighted average of the nearest 16 sample points in a rectangular grid, calculated once per sample point direction, each time using two polynomial interpolation cubic functions.
In some implementations of the first aspect, a dimming transmittance compensator is created that models the dimming image by keeping the same luminance value as the original image when the backlight is too dark, while taking into account light leakage effects, introducing a light leakage coefficient ε.
The dynamic backlight technology can be developed in the LCD field and is inseparable from the structure of the dynamic backlight technology, and the picture display brightness of the dynamic backlight technology can be adjusted by the liquid crystal panel and the backlight source which can be independently controlled, which is the hardware basis of the dynamic backlight technology. It is generally considered that the luminance of a liquid crystal display in an ideal state can be approximated by the product of the transmittance of a liquid crystal panel (LC) and the magnitude of the backlight luminance, that is:
backlight luminance × liquid crystal transmittance = image pixel value
Before applying LBD, it is assumed that the initial luminance of the BLU is the peak luminance (luminance = 1), which means that all light sources are turned on. Formally, the raw pixel value of the LCD panel at position (x, y) is modeled as:
wherein,
is the original pixel value of the c-th channel,
initial LC transmissivity of the c-th channel, B init (x, y) represents the initial backlight intensity of the c-th channel. Before dimming, all backlights are fully turned on, which corresponds to B init (x, y) =1, and thus, the initial pixel value before dimming is:
when the backlight brightness is dimmed, in order to keep the dimming image at the same brightness value as the original image, the LC transmittance must be compensated to be inversely proportional to the dimming ratio. For this reason, the compensation coefficient C is defined as:
wherein, B LBD (x, y) is the backlight intensity, which has a value between (0, 1). Since the liquid crystal transmittance is bounded, the saturation constraint must be imposed by the clip function. Finally, the compensated liquid crystal transmittance after backlight dimming is expressed as:
wherein f is clip To limit the input value to a clip function between 0 and 1.
According to the transmittance
And backlight intensity B LBD When there is no light leakage, the brightness on the display must be
But due to the presence of B LBD A proportionately small amount of leakage, so the dimming image is modeled as:
where e represents the light leakage coefficient. Depending on the liquid crystal material, device structure and viewing angle characteristics. In this patent, for simplicity, the entire pixel uses e =0.03.
In some implementations of the first aspect, the loss value between the original image and the dimming image is calculated by:
wherein,
for the pixel values of the original image,
is a dimmed image pixel value. W is the width of the image to be dimmed, H is the height of the image to be dimmed, and C is the number of channels of the image to be dimmed. Under the constraint condition that the output dimming image is infinitely close to the input image, the luminous intensity of the backlight blocks BLU with fixed quantity is estimated, and the scheme of backlight dimming is obtained.
In a second aspect, a local backlight dimming system is provided, which includes an initial backlight module, a fuzzy backlight module, an interpolation backlight module, and a backlight dimming module.
The initial backlight module comprises 1 convolution layer, 4 maximum pooling layers and 1 simple convolution neural network of full connection layers, wherein the input of the simple convolution neural network is an original image, and the output of the simple convolution neural network is BLU initial backlight; the fuzzy backlight module is responsible for performing fuzzy processing on the BLU initial backlight luminous intensity output by the initial backlight module by using bilateral smoothing operation; the interpolation backlight module is used for amplifying the initial BLU backlight to be equal to the size of the original image by using two-time and three-time interpolation operation on the output result of the fuzzy backlight module to obtain the integral BLU backlight; and the backlight dimming module is responsible for creating a dimming transmittance compensator and modeling a dimming image.
In a third aspect, a local backlight dimming training device is proposed, the device comprising at least one processor and a memory; the memory stores computer-executable instructions; the at least one processor executes computer-executable instructions stored by the memory, causing the at least one processor to perform the training method as mentioned in the first aspect and some embodiments of the first aspect.
In a fourth aspect, a readable storage medium is proposed, in which computer executable instructions are stored, which when executed by a processor, implement the training method as mentioned in the first aspect and some embodiments of the first aspect.
3. Advantageous effects
The technical scheme of the invention has the advantages of less parameters needing training, obviously reduced calculated amount and delay, simple network, obviously reduced learning cost and simpler training and hardware transplantation. Unlike the traditional method, the convolutional neural network method estimates intensity using features learned over a larger range of pixel values, naturally taking into account the diffusion of light from neighboring blocks, and experiments show that the algorithm achieves reasonable results in terms of objective metrics with fewer parameters to train, and can be implemented efficiently on hardware.
The method is based on the convolutional neural network, the influence of adjacent pixels can be considered, and the problem that the traditional characteristics lack generalization capability is solved. Meanwhile, the convolutional neural network is small in size, only one convolutional layer and one full-connection layer are used, the maximum pooling layer with four layers does not need to be trained, and compared with the conventional dimming method based on the convolutional neural network, which needs to train 108K parameters, the dimming method can achieve a similar dimming result by only training more than two thousand parameters, reduces the redundancy of the parameters, and can be effectively realized on hardware.
The invention uses the bilateral filtering smoothing method, compares the average filtering, the median filtering and the Gaussian filtering, and the bilateral filtering can blur the image and simultaneously keep the edge of the image unaffected, and is particularly important in backlight dimming.
In the method, the pixel value of one point is obtained by weighted average of 16 nearest sampling points in the rectangular grid, the generated effect is best, the graph can be interpolated more accurately by two times of double interpolation calculation than by one time of double interpolation, and meanwhile, compared with multiple times of interpolation, the calculated amount is less.
The invention designs a hardware-friendly local backlight dimming strategy, after the image is subjected to analog dimming, the peak signal-to-noise ratio and the picture similarity are both at a higher value, and the dimming task of the liquid crystal display with various scales can be well supported.
Drawings
FIG. 1 is a block flow diagram of a local backlight dimming algorithm;
FIG. 2 is a schematic diagram of a network structure of a local backlight dimming algorithm;
FIG. 3 is a schematic diagram of a local backlight dimming algorithm;
FIG. 4 is a flow chart of pixel compensation in a local backlight dimming algorithm;
FIG. 5 is a schematic diagram of the test effect of the present invention on DIV2K data set 1;
FIG. 6 is a schematic diagram of the effect of the DIV2K data set test according to the present invention shown in FIG. 2;
FIG. 7 is a diagram of the effect of the DIV2K data set test of the present invention shown in FIG. 3.
Detailed Description
The invention is described in detail below with reference to the drawings and specific examples.
In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the present invention. It will be apparent, however, to one skilled in the art, that the present invention may be practiced without one or more of these specific details. In other instances, well-known features have not been described in order to avoid obscuring the invention.
At present, in order to solve the problems that the backlight brightness of a liquid crystal display cannot be adjusted and reduced when a low-brightness picture is displayed, a perfect black area cannot be manufactured, partial light rays pass through liquid crystal for blocking, light leakage artifacts are formed, the display quality of an image is reduced, and the power consumption of a backlight module accounts for the largest of the whole power consumption of the LCD, so that the traditional LCD can cause great energy waste and the like, a plurality of researchers provide a local backlight dimming method.
Local backlight dimming requires dimming the backlight in dark areas and in turn compensates for LC transmittance, inversely proportional to dimming ratio, so that the displayed image is less distorted while reducing the effect of leakage characteristics. Nowadays, light Emitting Diodes (LEDs) have replaced Cold Cathode Fluorescent Lamps (CCFLs) as backlight units (BLUs). The BLU is typically divided into blocks so that we can control the intensity of the LEDs in each block locally and individually depending on the image content. However, the existing LBD often brings some degradation, such as detail loss and halo artifacts, the effective dimming method uses multiple convolutional layers, the network size is large, and many parameters need to be trained.
In order to overcome the defects of the prior art and solve the problems that the traditional local backlight dimming method lacks generalization capability, and a novel dimming method is large in network and difficult to realize, the invention adopts a hardware-friendly small network to learn characteristics so as to find the optimal backlight intensity. Unlike conventional methods, the method uses a convolutional layer and a pooling layer for continuous training, and estimates intensity using features learned over a larger range of pixel values. Experiments show that the algorithm obtains reasonable results in the aspect of objective measurement.
Example 1
A local backlight dimming algorithm, as shown in fig. 1, divides a whole backlight unit (BLU) into a fixed number of backlight blocks for local dimming control; using a simple convolutional neural network for an original image, passing through 1 expansion convolutional layer, 4 maximum pooling layers and 1 full-link layer, and finally using a sigmoid function to take output as normalized initial backlight intensity of the BLU; then, using bilateral smoothing operation and twice bicubic interpolation operation on the BLU initial backlight to perform fuzzy processing on the BLU initial backlight and perform interpolation amplification until the size of the BLU initial backlight is equal to that of the original image to obtain integral backlight; then, a dimming transmittance compensator is created, when the backlight is too dark, the same brightness value of the dimming image and the original image is kept, meanwhile, the light leakage problem is considered, and the dimming image is modeled according to the original image, the whole backlight, a dimming transmittance compensation coefficient and a light leakage coefficient; and finally, calculating a loss value between the original image and the dimming image, so that under the constraint condition that the output dimming image is infinitely close to the input image, the luminous intensity of a fixed number of backlight blocks BLU is estimated, training a simple convolutional neural network, and after the training is finished, storing all parameters to obtain the overall solution of backlight dimming.
The method comprises the following specific steps:
the entire BLU is divided into a fixed number of backlight blocks, which are the smallest units that individually control the intensity of the LEDs.
And inputting the original image into a simple convolution neural network, outputting the BLU initial backlight of the BLU normalized backlight intensity, and controlling the LED backlight to display by using the BLU initial backlight.
The blurriness process is performed using a smoothing operation on the BLU initial backlight. And carrying out interpolation operation on the BLU initial backlight after the smoothing operation, and amplifying the BLU initial backlight to be equal to the size of the original image to obtain the BLU overall backlight.
And creating a dimming transmissivity compensator, and performing dimming transmissivity compensation on the original image according to the BLU overall backlight to obtain the compensated liquid crystal transmissivity after backlight dimming.
And performing pixel compensation on the original image according to the BLU integral backlight, the compensation liquid crystal transmissivity after backlight dimming and the light leakage coefficient, and completing modeling of the dimming image.
Calculating the loss value between the original image and the dimming image, estimating the luminous intensity of the backlight blocks BLU with fixed quantity under the constraint condition that the output dimming image is infinitely close to the input image, training the simple convolutional neural network, and optimizing the parameters of the simple convolutional neural network model.
Calculating data of the forward propagation of the input image to the nodes of the output layer in each iteration, wherein the output data is BLU initial backlight, performing pixel compensation on the original image after smoothing and interpolation calculation on the BLU initial backlight, and finally outputting a dimming image; calculating gradient by calculating loss values of the dimming image and the original image, then performing back propagation, and updating the network model parameter weight w and the bias value bias in the process of back propagation; and finally obtaining stable network model parameters through multiple iterations, storing the final network model parameters, finishing the model construction after the training process is finished, and obtaining the overall scheme of backlight dimming.
The overall scheme of the backlight dimming comprises a calculation method of BLU initial backlight and BLU overall backlight which are output by a simple convolution neural network.
In this embodiment, a local backlight dimming algorithm of the flow shown in fig. 1 is applied, and based on a network structure of the local backlight dimming algorithm shown in fig. 2, the pixel compensation method shown in fig. 4 is specifically used according to the dimming principle shown in fig. 3. The example uses a DIV2K dataset, published for single image super resolution. The DIV2K dataset consists of content-diverse high-resolution (2K) RGB images, consisting of 800 training pictures and 100 verification pictures. To improve efficiency, training is performed using randomly cropped sub-images from the original data set, the sub-image size being 480 × 768, a quarter of the target image size. 5 x 8 BLUs are estimated from 480 x 768 sub-images, large enough for the network to consider information in surrounding BLU blocks.
After 10 rounds of training, the peak signal-to-noise ratio is stabilized above 35dB, the image similarity also reaches about 0.98, and the method is greatly improved compared with some manual characteristic value extraction methods. Junho Jo et al propose a local backlight dimming algorithm based on a convolutional neural network, 1082065 parameters need to be trained, the peak signal-to-noise ratio is stable and reaches 37.47dB, and the picture similarity reaches 0.984. The effects in the DIV2K dataset are shown in fig. 5-7, and taking fig. 5 as an example, the meanings of the five pictures from left to right are: the original image is subjected to backlight dimming image, initial backlight, fuzzy backlight and the difference value of the original image and the dimmed image through a trained network, and as can be seen from the figure, after local backlight dimming, the loss of the details of the image is small, and the reliability of the local backlight dimming algorithm is proved.
The technical scheme of the invention provides a hardware-friendly local backlight dimming algorithm, a smaller network is used for training, compared with the 108K parameter required by the existing dimming method based on the convolutional neural network, the dimming method only needs to train more than two thousand parameters to achieve a similar dimming result, the redundancy of the parameters is reduced, and the hardware can be effectively realized. Experiments show that the algorithm obtains reasonable results in the aspect of objective measurement.
Example 2
A local backlight dimming system comprises an initial backlight module, a fuzzy backlight module, an interpolation backlight module and a backlight dimming module.
The initial backlight module comprises a simple convolutional neural network with 1 convolutional layer, 4 maximum pooling layers and 1 full-connection layer, wherein the input of the simple convolutional neural network is an original image, and the output of the simple convolutional neural network is BLU initial backlight; the fuzzy backlight module is responsible for performing fuzzy processing on the BLU initial backlight luminous intensity output by the initial backlight module by using bilateral smoothing operation; the interpolation backlight module is used for performing bicubic interpolation operation twice on the output result of the fuzzy backlight module, amplifying the initial backlight of the BLU to be equal to the size of the original image, and obtaining the integral backlight of the BLU; and the backlight dimming module is responsible for creating a dimming transmissivity compensator and modeling the dimming image.
Example 3
A local backlight dimming system training device, comprising: at least one processor and at least one memory; the memory stores computer-executable instructions; the processor executes computer-executable instructions stored by the memory to cause at least one processor to perform the local backlight dimming method of embodiment 1.
Example 4
A readable storage medium having stored therein computer-executable instructions, which when executed by a processor, implement the backlight dimming method of embodiment 1.
The invention and its embodiments have been described above schematically, without limitation, and the invention can be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The illustration in the drawings is only one embodiment of the invention and the actual construction is not limited to this. Therefore, without departing from the spirit of the present invention, a person of ordinary skill in the art should also understand that the present invention shall not be limited to the embodiments and the similar structural modes of the present invention. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Several of the elements recited in the product claims may also be implemented by one element in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (10)
1. A method of local backlight dimming, the method comprising:
dividing the entire BLU into a fixed number of backlight blocks, which are the minimum units to individually control the intensity of the LEDs;
inputting an original image into a simple convolutional neural network, outputting a BLU initial backlight of the BLU normalized backlight intensity, and controlling LED backlight display by the BLU initial backlight;
carrying out fuzzy processing on the initial backlight of the BLU by using a smoothing operation;
carrying out interpolation operation on the BLU initial backlight after the smoothing operation, and amplifying the BLU initial backlight to be equal to the size of the original image to obtain the BLU integral backlight;
establishing a dimming transmissivity compensator, and performing dimming transmissivity compensation on the original image according to the BLU whole backlight to obtain the compensated liquid crystal transmissivity after backlight dimming;
performing pixel compensation on the original image according to the BLU integral backlight, the compensation liquid crystal transmissivity and the light leakage coefficient after backlight dimming, and completing modeling of a dimming image;
calculating a loss value between an original image and a dimming image, estimating the luminous intensity of a fixed number of backlight blocks BLU under the constraint condition that the output dimming image is infinitely close to the input image, training the simple convolutional neural network, and optimizing the parameters of a simple convolutional neural network model;
calculating data of the forward propagation of the input image to the nodes of the output layer in each iteration, wherein the output data is BLU initial backlight, smoothing and interpolation calculation are carried out on the BLU initial backlight, pixel compensation is carried out on the original image, and finally a dimming image is output; calculating gradient by calculating loss values of the dimming image and the original image, then performing back propagation, and updating the network model parameter weight w and the bias value bias in the process of back propagation; obtaining stable network model parameters finally through multiple iterations, storing the final network model parameters, completing model construction after the training process is finished, and obtaining an overall backlight dimming scheme;
the overall scheme of backlight dimming includes a calculation method of a BLU initial backlight output by a simple convolutional neural network and a BLU overall backlight.
2. The local backlight dimming method of claim 1, wherein the simple convolutional neural network comprises 1 expansion convolutional layer, 4 maximum pooling layers and 1 full-link layer, and finally the output is normalized to [0,1] by using a sigmoid function to obtain the normalized backlight intensity of the BLU, i.e. the BLU initial backlight;
wherein the input h of the convolutional layer in As an original image, a backlight image h is output out ：
h out ＝σ(W*h in +bias)
Wherein W is weight, bias is bias value, and sigma is nonlinear activation function;
and setting different step lengths for the four maximum pooling layers, and extracting the highest value of the pixel value of the backlight image.
3. The local backlight dimming method according to claim 1, wherein the specific operation of blurring the initial backlight of the BLU using the smoothing operation is a bilateral filtering smoothing operation:
wherein,
wherein, b blur Is the blurred backlight intensity at block (c, r), b dim Is the initial backlight intensity at block (c, r); c is the column index of the backlight block, r is that of the backlight blockA row index; d is the pixel neighborhood diameter, k is the row index of the adjacent pixel, and l is the column index of the adjacent pixel; delta is the labeling variance of the coordinate space; p (X) 0 ,y 0 ) Represents the pixel value at the center point of the initial backlight block of the BLU, and p (x, y) represents the pixel value at a distance (x, y) from the center point.
4. The local backlight dimming method of claim 1, wherein the BLU initial backlight is enlarged to the original image size using two bicubic interpolation operations to obtain the BLU overall backlight, and the calculation method is:
wherein, a ij For calculating the coefficient, the value of which depends on the characteristics of the interpolation data, p (x, y) is the pixel value of the pixel point coordinate (x, y) of the BLU overall backlight image to be calculated, and i and j are the row and column numbers of the selected interpolation.
5. The local backlight dimming method according to claim 4, wherein the pixel value of the pixel coordinates (x, y) of the BLU overall backlight image to be calculated is obtained by weighted average of the nearest 16 sampling points in the rectangular grid, and the calculation is performed by using two polynomial interpolation cubic functions each time, and the calculation is performed once in each sampling point direction.
6. The local backlight dimming method according to claim 1, wherein when modeling the dimming image, the transmittance compensation coefficient is:
wherein, B LBD (x, y) is the BLU overall backlight intensity;
the compensated liquid crystal transmittance after backlight dimming is expressed as:
wherein f is clip To limit the input value to a clip function between 0 and 1,
initial LC transmittance for the c-th channel;
the dimming image is modeled as:
where ε represents a light leakage coefficient.
7. The local backlight dimming method according to claim 1, wherein the calculating method for calculating the loss value between the original image and the dimming image is:
wherein W is the width of the image, H is the height of the image, C is the number of channels of the image,
for the pixel values of the original image,
are the dimmed image pixel values.
8. A local backlight dimming system, the system comprising:
the initial backlight module comprises 1 convolution layer, 4 maximum pooling layers and 1 simple convolution neural network of full connection layers, wherein the input of the simple convolution neural network is an original image, and the output of the simple convolution neural network is BLU initial backlight;
the fuzzy backlight module is responsible for performing fuzzy processing on the BLU initial backlight luminous intensity output by the initial backlight module by using bilateral smoothing operation;
the interpolation backlight module is used for performing bicubic interpolation operation twice on the output result of the fuzzy backlight module, amplifying the initial backlight of the BLU to be equal to the size of the original image, and obtaining the integral backlight of the BLU;
and the backlight dimming module is responsible for creating a dimming transmissivity compensator and modeling the dimming image.
9. A local backlight dimming system training device, comprising: at least one processor and at least one memory; the memory stores computer-executable instructions; the processor executes computer-executable instructions stored by the memory to cause the at least one processor to perform the local backlight dimming method of any one of claims 1 to 7.
10. A readable storage medium having stored therein computer-executable instructions, which when executed by a processor, implement the local backlight dimming method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211357097.2A CN115660992A (en) | 2022-11-01 | 2022-11-01 | Local backlight dimming method, system, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211357097.2A CN115660992A (en) | 2022-11-01 | 2022-11-01 | Local backlight dimming method, system, device and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115660992A true CN115660992A (en) | 2023-01-31 |
Family
ID=84995114
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211357097.2A Pending CN115660992A (en) | 2022-11-01 | 2022-11-01 | Local backlight dimming method, system, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115660992A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117524117A (en) * | 2023-12-07 | 2024-02-06 | 深圳市创想数维科技有限公司 | Processing method for solving reflection of LED screen |
| CN117690388A (en) * | 2024-02-04 | 2024-03-12 | 深圳康荣电子有限公司 | Picture optimization method and system based on display module backlight |
-
2022
- 2022-11-01 CN CN202211357097.2A patent/CN115660992A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117524117A (en) * | 2023-12-07 | 2024-02-06 | 深圳市创想数维科技有限公司 | Processing method for solving reflection of LED screen |
| CN117690388A (en) * | 2024-02-04 | 2024-03-12 | 深圳康荣电子有限公司 | Picture optimization method and system based on display module backlight |
| CN117690388B (en) * | 2024-02-04 | 2024-04-19 | 深圳康荣电子有限公司 | Picture optimization method and system based on display module backlight |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108877694B (en) | Double-layer liquid crystal screen, backlight brightness control method and device and electronic equipment | |
| US8610654B2 (en) | Correction of visible mura distortions in displays using filtered mura reduction and backlight control | |
| CN109036245B (en) | Display processing method and device, integrated circuit and computer storage medium | |
| CN109887468B (en) | Liquid crystal display dynamic dimming method based on image local features | |
| EP2409194B1 (en) | Area adaptive backlight display and method with reduced computation and halo artifacts | |
| CN102298894B (en) | Display device and contrast enhancement method thereof | |
| JP5138809B2 (en) | Method and system for managing region adaptive backlights | |
| US8717278B2 (en) | Method and apparatus for adjusting drive values for dual modulation displays | |
| US11114046B2 (en) | Display device and method for driving the same, driving apparatus, and computer-readable medium | |
| CN109243384B (en) | Display device, driving method thereof, driving apparatus thereof, and computer readable medium | |
| JP2011013458A (en) | Liquid crystal display device | |
| CN110838090B (en) | Backlight diffusion method for image processing based on residual error network | |
| US8421741B2 (en) | Method, system and apparatus for power saving backlight | |
| KR20110128853A (en) | Signal generation for led/lcd-based high dynamic range displays | |
| CN113408655B (en) | Color sequence display control method and device based on deep learning | |
| CN113674705B (en) | Backlight extraction method based on radial basis function neural network agent model auxiliary particle swarm algorithm | |
| Liao et al. | Mitigation of image blurring for performance enhancement in transparent displays based on polymer-dispersed liquid crystal | |
| JP2011039451A (en) | Display device, luminance unevenness correction method, and device and method for generating correction data | |
| CN115660992A (en) | Local backlight dimming method, system, device and medium | |
| Zhang et al. | Deep CNN-based local dimming technology | |
| Zhang et al. | High‐performance local‐dimming algorithm based on image characteristic and logarithmic function | |
| Korhonen et al. | Modeling LCD displays with local backlight dimming for image quality assessment | |
| Trentacoste et al. | Photometric image processing for high dynamic range displays | |
| JP5389966B2 (en) | Display device and luminance unevenness correction method for display device | |
| JP4942808B2 (en) | Display device and luminance unevenness correction method for display device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |