CN101082992A - Drawing of real time high dynamic range image and display process - Google Patents

Abstract

The invention discloses a method for real-time drawing and displaying high dynamic range images. The steps are: 1) retain the high dynamic range information when drawing the scene; 2) realize the glow effect: first calculate the average brightness of the image, and then filter the brightness of the image according to the set threshold to generate only the filtered high For the new image in the brightness area, two-dimensional Gaussian blur is performed on the new image to produce a glow effect; 3) Tone mapping is performed on the original image containing high dynamic range information, so that the high dynamic range image can be displayed in the low dynamic range Come out and keep as much detail as possible; 4) Alpha blend the tone mapped image and the blurred image to generate the final image. The invention adopts the method of post-processing the image, has small calculation amount, can meet the requirement of real-time drawing, and has very good practical value.

Description

The drafting of real time high dynamic range image and display packing

Technical field

The present invention relates to a kind of drafting and display packing of real time high dynamic range image.

Background technology

The ratio of the maximal value of piece image intensity level and minimum value is called as dynamic range.By the automatic adjusting of eye pupil, the dynamic range of the brightness that human eyes can be experienced is 1000000000: 1.Even in the image that computing machine generates, the dynamic range of the image that employing global illumination algorithm generates is the highest also to reach 30000: 1.In classic method, each color of pixel is that wherein each passage of red, green, blue three looks takies 8, adds 8 alpha passage sometimes by 24 binary digit storages, 32 altogether.The color that renders at last is between 0.0-1.0, and the bright contrast that can show is 256: 1.So classic method is difficult to the real scene of expression high-contrast and high dynamic range.Recent years is because the growth at full speed of computer computation ability can realize real-time high dynamic range drafting at the graphic hardware of consumer level.

When drawing high dynamic range images, need to preserve high dynamic range information.It is directly to preserve with the buffer memory of high precision floating-point format that one class methods are arranged, and supports the buffer memory of A16B16G16R16F and A32B32G32R32F form such as current up-to-date video card.These class methods need consume the more buffer memory of multicapacity, and can't be applied on the old hardware device.The present invention has adopted the coded format of RGBE8, preserves high dynamic range information by information is encoded.This method need not the extra support of hardware, and it is few to consume video memory, can be with the colouring information of different precision compression different range by parameter adjustment.

The indication range of current display device all belongs to low-dynamic range usually, and high dynamic range images can not directly show on common display device, otherwise the high dynamic range information that can lose image causes under-exposure or overexposure.The image of high dynamic range need be mapped to low-dynamic range with colouring information from high dynamic range through tone map, could correctly show on display device.Tone mapping method is divided into overall operator and local operator usually.The basic thought of overall situation operator is that all pixels on the piece image are shone upon with same mapping function.Overall situation operator framework is simple, and efficient is higher, but for brighter and darker zone distortion easily.In local operator method, each pixel is no longer handled by unified mapping function, but decides the result of mapping according to its neighbor.Local operator has improved the problem of details distortion, can obtain than the better effect of overall operator usually.But the common more complicated of algorithm, efficient is not high, generally is applied in the non real-time field of drawing.The present invention is directed to the field of real-time rendering, therefore adopted overall operator method, designed new global map function, when guaranteeing real-time, obtained the better image quality.

There is a kind of like this phenomenon in the true environment:, be referred to as the aura effect because the scattering of light in atmosphere or human eye has one deck halation around the Ming Liang object especially.Still there is very bright part in high dynamic range images through after the tone map, seems not nature if directly show.Therefore need simulation aura effect to obtain more real image.The aura effect need be carried out Fuzzy Processing with former scene, and this generally finishes by the 2-d gaussian filters function.Adopt the quality of this kind method image to depend critically upon the number of sampled point, but sampled point can make that too much calculated amount increases greatly, can't guarantee real-time.The present invention has adopted the method for secondary convolution to carry out Gaussian Blur, makes to n ²Individual sampled point only needs 2n calculating, and this method is simply quick, also can obtain the better image quality simultaneously.

Summary of the invention

The method that the purpose of this invention is to provide a kind of real-time rendering and demonstration high dynamic range images.

Real-time rendering also shows that the method for high dynamic range images comprises the steps:

1) when drawing image, adopts the RGBE8 form, preserve high dynamic range information; This form is with color vector R, G, B, E four representation in components of image pixel, and wherein E is an exponential component, is shared by R, G, three components of B; Each component size is 1 byte, and four components take 4 byte spaces altogether;

2) geometrical mean of the brightness of calculating high dynamic range information image, the image of having preserved high dynamic range information is carried out brightness to be filtered, generate a width of cloth new images, new images only comprises the pixel of original image brightness greater than certain fault value, new images is carried out two-dimentional Gaussian Blur, generate the image of aura effect;

3) adopt overall compression function that the image that has comprised high dynamic range information is carried out tone map, make high dynamic range information be compressed to low-dynamic range;

4) image of image that has carried out tone map and aura effect is carried out alpha and mix, realize final effect.

The described RGBE8 form that when drawing image, adopts, the step of preserving high dynamic range information is:

1) establishing p is image pixel, and its color vector is represented that by four-tuple r, b, b, a wherein r, g, b, a represent red, green, blue, alpha component respectively, calculates the maximal value m of the red, green, blue color component of p;

2) calculate index $e={\log }_{\mathrm{base}}^m,$ To be stored in the alpha component of pixel p p after the e skew compression _a=clamp ((e+128)/256,0,1);

3) by the color value coding of formula 1 with image, by formula 2 decodings,

$p_{\mathrm{rgb}}=\frac{p_{\mathrm{rgb}}}{{\log }_{\mathrm{base}}^m}$ Formula 1

$P_{\mathrm{rgb}}=P_{\mathrm{rgb}}\text{\×}{\mathrm{base}}^{P_a\×256-128}$ Formula 2

Wherein base is the truth of a matter of logarithm operation, is set by the user.When base increased, the scope of compression increased, and compression accuracy reduces; When base reduced, compression zone reduced, and compression accuracy increases.

The computing method of described brightness of image L are: establishing P is a pixel of image,

L=R * 0.2126+G * 0.7152+B * 0.0722 formula 3

Wherein, R, G, B are respectively the values of the red, green, blue component of pixel P color.

The brightness filter method of described high dynamic range images comprises the steps:

1) according to the mean flow rate of image convergent-divergent is carried out in the brightness of each pixel of image;

2) set fault value t, brightness will be filtered out greater than the pixel of t;

3) with the luminance compression of the pixel that filters out in interval [0,1].

Described new images is carried out two-dimentional Gaussian Blur: at first image is carried out the one dimension Gaussian Blur, generate intermediate result image along X-direction; Then middle result images is carried out the one dimension Gaussian Blur one time again along Y direction, realize the effect of two-dimentional Gaussian Blur by the one dimension Gaussian Blur on orthogonal directions of twice substep.

Described high dynamic range images is carried out tone map: behind the brightness convergent-divergent of mean flow rate according to image with pixel P, according to formula 4 with the luminance compression of pixel P in interval [0,1],

$L_{\mathrm{color}}=\frac{L_{\mathrm{scaled}}}{1+L_{\mathrm{scaled}}}$ Formula 4

Wherein, L _ScaledBe the brightness of the pixel P behind the convergent-divergent, L _ColorBe the brightness of carrying out the pixel P after the tone map.

Described handle has carried out the image of tone map and the image of aura effect carries out the alpha mixing: establish P _RgbBe through the arbitrary color of pixel vector of image after the tone map, B _RgbBe the color vector of the respective pixel of aura effect image, the two carries out the alpha mixing according to formula 5,

P _Rgb=P _Rgb+ α * B _RgbFormula 5

Wherein α is the coefficient of control mixed effect.

Described will being stored in the alpha component of pixel after the e skew compression: p _a=clamp ((e+128)/256,0,1), wherein function y=clamp (x, 0,1) is that value with x is truncated in the interval [0,1], promptly when x≤0, y=0; When 0＜x≤1, y=x; When x＞1, y=1.

Described setting fault value t, brightness will be filtered out greater than the pixel of t: at first carry out convergent-divergent according to the brightness of 6 pairs of each pixels of image of formula;

$L_{\mathrm{scaled}}=a\frac{L_w}{{\stackrel{-}{L}}_w}$ Formula 6

Wherein a is a zoom factor, specify by the user,

Be the mean flow rate of image, L _ScaledBe through the brightness behind the convergent-divergent.Given then fault value t, according to formula 7, brightness will be retained greater than the pixel of t, and brightness will be changed to 0 less than the pixel of t,

L _Bright=max (L _Scaled-t, 0) formula 7

L wherein _BrightBe the brightness of the pixel that filters out.

Described luminance compression with the pixel that filters out is in interval [0,1]: establish L _BrightBe the brightness that filters out, designated parameter o is according to 8 couples of L of formula _BrightCompress the brightness L ' after guaranteeing to compress _BrightBe positioned in [0,1] interval,

$L_{\mathrm{bright}}^{\′}=\frac{L_{\mathrm{bright}}}{o+L_{\mathrm{bright}}}$ Formula 8.

The present invention preserves high dynamic range information by the RGBE8 form, can preserve large-scale information simultaneously at the less video memory capacity of consumption, and this method is low to hardware requirement, can extensively operate on the different hardware platforms, has good portability.When carrying out gaussian filtering, the present invention is divided into two one dimension gaussian filterings on orthogonal directions with 2-d gaussian filters, makes to n ²The Gaussian Blur of individual sampled point only needs 2n calculating just can finish, and has improved efficient greatly.The present invention has adopted the global map function that high dynamic range images is carried out tone map, and the simple efficient height of this method principle can satisfy real-time requirement, obtains the better image quality simultaneously.

Description of drawings

Fig. 1 is real-time rendering and the process flow diagram that shows high dynamic range images;

Fig. 2 a is that the present invention is used for the former figure that brightness is filtered;

Fig. 2 b is that to get the fault value be to carry out the brightness filtering result at 0.5 o'clock in the present invention;

Fig. 2 c is that to get the fault value be to carry out the brightness filtering result at 0.8 o'clock in the present invention;

Fig. 3 is the synoptic diagram that the present invention carries out Gaussian Blur;

Fig. 4 is the effect example 1 that high dynamic range images of the present invention is drawn and shown;

Fig. 5 is the effect example 2 that high dynamic range images of the present invention is drawn and shown;

Embodiment

The principle of real-time rendering and demonstration high dynamic range images method is: in the drawing process of image, keep high dynamic range information, generate high dynamic range images.Realize the aura effect by high dynamic range images being carried out Gaussian Blur, by tone map high dynamic range images is mapped to low-dynamic range and and aura effect image mixing generation final image at last.This method at first when drawing image, adopts the high dynamic range information of high precision floating number memory image, and high dynamic range images is stored in the frame buffering with the RGBE8 form, when taking out view data from the frame buffering, decodes earlier.Filter out brightness from original image then and carry out Gaussian Blur, generate the image of a width of cloth aura effect greater than the pixel of certain fault value.Next high dynamic range image tone is mapped to low-dynamic range, carries out alpha with the aura effect image again and mix, generate the final effect image.The step of this method is as shown in Figure 1:

1) with RGBE8 form storage high dynamic range images, this form is made up of R, G, four passages of B, E, and wherein E is the index passage.Each passage is 8 bit byte data types, and 4 passages come to 32 bits.The steps include:

(1) maximal value of each pixel color component of computed image.The RGBE8 coded format is preserved color of pixel with exponential form, three components of the red, green, blue of color vector are shared same index, therefore need find out the maximal value m of color component, calculates index with this.

(2) according to maximal value m, calculate index $e={\log }_{\mathrm{base}}^m,$ Wherein base is specified by the user, and the value of base has determined compressible range of information.Along with the base value increase can information compressed scope also increase, but the precision of compression also reduces accordingly, Mach band effect can occur.Therefore we need weigh compression zone and precision, in our system, and base=1.02, this value can be compressed information in a big way, guarantees the better image quality simultaneously.Because e will be stored in the alpha component of pixel, and the alpha component is the byte data of big or small 8 bits, its span is [0,1] interval, therefore need be with e through type p _a=clamp ((e+128)/256,0,1) is offset compression, and wherein function y=clamp (x, 0,1) is that value with x is truncated in the interval [0,1], promptly when x≤0, and y=0; When 0＜x≤1, y=x; When x＞1, y=1.

(3), recomputate the value P of the red, green, blue component in its color vector according to formula 1 to each pixel of image _Rgb, it is preserved with exponential form.

$p_{\mathrm{rgb}}=\frac{p_{\mathrm{rgb}}}{{\log }_{\mathrm{base}}^m}$ Formula 1

(4) in the time will using, earlier by 2 pairs of data decodings of formula through RGBE8 form coded data.

$P_{\mathrm{rgb}}=P_{\mathrm{rgb}}\×{\mathrm{base}}^{p_a\×256-128}$ Formula 2

2) realize the aura effect.There is a kind of like this phenomenon in the true environment:, be referred to as the aura effect because the scattering of light in atmosphere or human eye has one deck halation around the Ming Liang object especially.Still there is very bright part in high dynamic range images through after the tone map, seems not nature if directly show.Therefore need simulation aura effect to obtain more real image.The steps include:

(1) geometrical mean of the brightness of calculating high dynamic range images.Before the mean flow rate of computed image, at first according to the brightness of each pixel of formula 3 computed image, we have adopted the YUV colour model that each color of pixel value of image is converted to brightness, the vision that this model can more accurate anthropomorphic dummy,

L=R * 0.2126+G * 0.7152+B * 0.0722 formula 3

Wherein R, G, B are respectively the red, green, blue color components of image pixel.

Obtain the geometric average luminance of image then.Compare with arithmetic mean, what geometrical mean can be avoided a few pixels brightness then calculates the ratio of the brightness of each pixel with respect to mean flow rate according to formula 6, judges the relative bright-dark degree of each pixel with this.

$L_{\mathrm{scaled}}=a\frac{L_w}{{\stackrel{-}{L}}_w}$ Formula 6

Wherein a is a zoom factor, is specified by the user, and contrast is not violent especially image for the brightness light and shade, and a value generally gets 0.18.For the very big zone of brightness of image, the desirable less relatively value of a; For the very little zone of brightness of image, the desirable relatively large value of a.

Be the mean flow rate of image, L _ScaledBe through the brightness behind the convergent-divergent.

(2) high dynamic range images being carried out brightness filters.We can calculate the ratio L of the relative mean picture brightness of each pixel according to formula 6 _Scaled, setting fault value t, can come out brightness greater than the pixel extraction of t according to formula 7, generate the new image of a width of cloth.

L _Bright=max (L _Scaled-t, 0) formula 7

This image has only kept the brightness of brightness greater than the pixel of t, and brightness is changed to 0 less than the brightness of the pixel of t.Filtering the brightness value that extracts through brightness may be greater than 1, and we need compress it.Setup parameter o, according to formula 8, we arrive the luminance compression of each pixel in [0,1] interval.

$L_{\mathrm{bright}}^{\′}=\frac{L_{\mathrm{bright}}}{o+L_{\mathrm{bright}}}$ Formula 8

Wherein parameter o has determined the brightness of the bright areas that is separated.Usually the image o value value that contrast is high more is big more.

(3) new image is carried out Gaussian Blur.Gaussian Blur is used to realize dim halation result.We have adopted the 2-d gaussian filters function to realize Gaussian Blur.When adopting the gaussian filtering function to blur, the quality of the image of generation depends critically upon the number of sampled point, but sampled point too much can cause calculated amount to increase greatly, can't satisfy real-time requirement.The gaussian filtering function has the linear separability characteristic, promptly to the calculating of 2-d gaussian filters function can break two independently the one-dimensional space carry out respectively.Based on this characteristic, we are divided into two one dimension gaussian filterings to 2-d gaussian filters along orthogonal directions.For the first time image is carried out the one dimension gaussian filtering along X-direction, generate an intermediate result image.For the second time middle result images is carried out the one dimension gaussian filtering one time again along Y direction.After this method of sampling, to n ²The calculating of individual sampled point only needs the 2n step just can finish, and greatly reduces computation complexity and has guaranteed picture quality simultaneously.

3) image that has comprised high dynamic range information is carried out tone map.Current display device can only show the information of low-dynamic range usually, and high dynamic range images can not directly show, otherwise can cause under-exposed or over-exposed.Therefore high dynamic range images need be mapped to low-dynamic range with high dynamic range information through tone map, and keeps information completely as far as possible.In order to guarantee real-time, to each pixel of high dynamic range images, we have adopted unified non-linear compression function.Non-linear compression function is as follows:

$L_{\mathrm{color}}=\frac{L_{\mathrm{scaled}}}{1+L_{\mathrm{scaled}}}$ Formula 4

This equation guarantees that the brightness of all pixels all is compressed to [0,1] interval.If L _ScaledWhen big, the zoom factor of this equation is approximately 1/L, works as L _ScaledHour, zoom factor is approximately 1.

4) image of image that has carried out tone map and aura effect is carried out alpha and mix, realize final effect.Mix by two width of cloth images being carried out alpha, the aura effect is dissolved in the original image.If P _RgbBe through the arbitrary color of pixel vector of image after the tone map, B _RgbIt is the color vector of the respective pixel of aura effect image.Set hybrid parameter α, the two carries out blend of colors, and account form is as follows:

P _Rgb=P _Rgb+ α * B _RgbFormula 5

Here provided the example that some high dynamic range images are drawn and shown.From example, as can be seen, no matter be indoor light conditions (Fig. 4), or outdoor light conditions (Fig. 5), the algorithm of this chapter is energy real-time rendering and the high dynamic range images that shows under the various environment all, and the efficiency of algorithm height is effective.

Claims (10)

1. A method for real-time drawing and displaying a high dynamic range image, characterized in that it may further comprise the steps: 1) RGBE8 format is used to save high dynamic range information when drawing an image; this format represents the color vector of an image pixel with four components of R, G, B, and E, where E is an exponential component, composed of three components R, G, and B Component sharing; the size of each component is 1 byte, and the four components occupy a total of 4 bytes of space, and the size of the dynamic range included is approximately from 10 ^-38 cd/m ² to 10 ³⁸ cd/m ² ; 2) Calculate the geometric mean value of the brightness of the image with high dynamic range information, and filter the brightness of the image with high dynamic range information to generate a new image. The new image only contains pixels whose brightness is greater than a certain threshold in the original image. The image is subjected to two-dimensional Gaussian blurring to generate an image with a glow effect; 3) A global compression function is used to perform tone mapping on images containing high dynamic range information, so that high dynamic range information is compressed to low dynamic range; 4) Alpha blend the tone-mapped image and the glow-effect image, draw and display the high dynamic range image in real time. 2. a kind of real-time drawing according to claim 1 and display method for high dynamic range image, it is characterized in that described adopt RGBE8 format when drawing image, the step of preserving high dynamic range information is: 1) Let p be an image pixel, and its color vector is represented by a quadruple r, g, b, a, where r, g, b, a represent red, green, blue, and alpha components respectively, and calculate the red, green, and alpha components of p. The maximum value m of the blue color component; 2) Calculate the index $e={\log }_{\mathrm{base}}^m,$ Compress the e offset and store it in the alpha component of pixel p, P _a = clamp((e+128)/256, 0, 1); 3) Encode the color value of the image by formula 1, and decode it by formula 2, $p_{\mathrm{rgb}}=\frac{p_{\mathrm{rgb}}}{{\log }_{\mathrm{base}}^m}$ Formula 1 $P_{\mathrm{rgb}}=P_{\mathrm{rgb}}\&times;{\mathrm{base}}^{p_a\&times;256-128}$ Formula 2 Among them, base is the base number of logarithm operation, which is set by the user. When the base increases, the compression range increases and the compression accuracy decreases; when the base decreases, the compression range decreases and the compression accuracy increases. 3. A method for drawing and displaying high dynamic range images in real time according to claim 1, characterized in that the calculation method of the image brightness L is as follows: let P be a pixel of the image, L＝R×0.2126+G×0.7152+B×0.0722 Formula 3 Among them, R, G, and B are the values of the red, green, and blue components of the color of the pixel P, respectively. 4. A method for real-time drawing and displaying a high dynamic range image according to claim 1, wherein the brightness filtering method of the high dynamic range image comprises the following steps: 1) scaling the brightness of each pixel of the image according to the average brightness of the image; 2) Set the threshold t, and the pixels whose brightness is greater than t will be filtered out; 3) Compress the brightness of the filtered pixels into the interval [0, 1]. 5. A method of real-time drawing and displaying a high dynamic range image according to claim 1, wherein said two-dimensional Gaussian blurring is performed on the new image: first, one-dimensional Gaussian blurring is performed on the image along the X-axis direction, Generate an intermediate result image; then perform a one-dimensional Gaussian blur on the intermediate result image along the Y-axis direction, and realize the effect of a two-dimensional Gaussian blur through two step-by-step one-dimensional Gaussian blurs in the orthogonal direction. 6. A method for real-time drawing and displaying a high dynamic range image according to claim 1, wherein the tone mapping is performed on the high dynamic range image: after scaling the brightness of the pixel P according to the average brightness of the image, Compress the brightness of the pixel P into the interval [0, 1] according to formula 4, $L_{\mathrm{color}}=\frac{L_{\mathrm{scaled}}}{1+L_{\mathrm{scaled}}}$ Formula 4 Wherein, L _scaled is the brightness of the pixel P after scaling, and L _color is the brightness of the pixel P after tone mapping. 7. A method for drawing and displaying high dynamic range images in real time according to claim 1, characterized in that the image that has been tone-mapped and the image of the glow effect are alpha-blended: Let P _rgb be passed through The color vector of any pixel of the image after tone mapping, B _rgb is the color vector of the corresponding pixel of the glow effect image, and the two are alpha mixed according to formula 5, P _rgb ＝P _rgb +α×P _rgb Formula 5 where α is a coefficient controlling the effect of mixing. 8. a kind of real-time drawing according to claim 2 and display the method for high dynamic range image, it is characterized in that described e offset is stored in the alpha component of pixel after being compressed: P _a =clamp((e+128)/256,0,1), where the function y=clamp(x,0,1) intercepts the value of x to the interval [0,1], that is, when x≤0 When , y=0; when 0<x≤1, y=x; when x>1, y=1. 9. A method of real-time drawing and displaying a high dynamic range image according to claim 4, characterized in that in the set threshold t, pixels whose brightness is greater than t will be filtered out: first, according to formula 6, image The brightness of each pixel is scaled; $L_{\mathrm{scaled}}=a\frac{L_w}{{\stackrel{-}{L}}_w}$ Formula 6 where a is the scaling factor, specified by the user,

is the average brightness of the image, and L _scaled is the scaled brightness. Then given the threshold t, according to formula 7, the pixels whose brightness is greater than t will be retained, and the pixels whose brightness is less than t will be set to 0, L _bright = max(L _scaled -t, 0) Formula 7 where _Lbright is the brightness of the filtered pixel. 10. A method for drawing and displaying high dynamic range images in real time according to claim 4, characterized in that the brightness of the filtered pixels is compressed into the interval [0, 1]: let L _bright be the filtered The output brightness, specify the parameter o, compress L _bright according to formula 8, and ensure that the compressed brightness L' _bright is in the interval [0, 1], $L_{\mathrm{bright}}^{\&prime;}=\frac{L_{\mathrm{bright}}}{o+L_{\mathrm{bright}}}$ Formula 8.

Images