TW201608875A - Method and device for encoding a high-dynamic range frame and/or decoding a bitstream - Google Patents

Abstract

The present disclosure generally relates to a method and device for encoding a frame, comprising a processor configured for: - encoding (12) a backlight frame determined (11) from the frame to be encoded; - obtaining (13) at least one component of a residual frame by dividing each component of the frame by a decoded version of the backlight frame; - encoding (19) the residual frame; the method is characterized in that the processor is further configured in order that the backlight frame is represented by a weighted linear combination of at least one set of 2D shape functions, the 2D shape functions of each set forming a partition of unity of a domain defined by the union of their supports. The disclosure relates also to a method and device for decoding a bitstream representing a residual frame calculated by dividing a frame by a backlight frame.

Description

Frame coding method and device, and frame decoding method and device, computer program product and processor readable medium and non-transitory storage medium

This case is generally about frame/video encoding and decoding. In particular, the technical field of the present invention relates to frame encoding/decoding in which pixel values belong to a high dynamic range.

This section is intended to introduce the technical gist of the reader to the following and/or the claimed aspects of the present invention. This discussion is believed to help provide readers with background information, and it is easier to understand the scope of the case. Therefore, it is to be understood that such statements are read from this point of view rather than the prior art.

The luminance value of a low dynamic range frame (LDR frame), expressed as a finite number of bits (most commonly 8 or 10). This restricted representation does not correctly depict small signal variations, especially in the dark and bright range of brightness. In the high dynamic range frame (HDR frame), the signal notation is extended to maintain the high accuracy of the signal over the full range. In the HDR frame, the pixel values are usually expressed in floating point format (each component can be 32 or 16 bits, ie floating or semi-floating), the most common format is open EXR semi-floating format (16 bits per RGB component, That is, 48 bits per pixel), or long notation, in integers, usually at least 16 bits.

The general strategy for encoding an HDR frame is to reduce the dynamic range of the frame to encode the frame using conventional encoding (which initially constitutes the encoded LDR frame).

The backlight frame is determined from the luminance component of the input HDR frame according to a known strategy. The input HDR frame is divided by the backlight frame to obtain the remaining frame. The encoding of both the backlight frame and the remaining frame uses traditional encoders, such as H.264/AVC (<Advanced video for general audiovisual services). Code Writing>, Series H: Audiovisual and Multimedia Systems, Recommendation ITU-T H.264, ITU Telex Standardization Department, January 2012), or HEVC (High Efficiency Video Writing), Series H: Audiovisual And Multimedia Systems, Recommendation ITU-T H.265, ITU Telex Standardization Sector, April 2013).

The compression capability is strongly dependent on the time frame of the frame and the spatial smoothness of each frame. In fact, video encoders use time and intraframe estimates as a powerful tool to compress video data.

The ability to automatically provide visual LDR frames from the coded residual frames is a key advantage as it allows the assignment of HDR frames (video) to users, the installation of standard LDR TVs, and the non-dedicated post-processing of backlight frames and remaining Frame to decode the receiver of the HDR frame (video).

Unfortunately, if someone does not pay attention to the construction of the backlight frame on the encoder side, the backlight frame will lack uniformity, and this unevenness will create several smooth gradients in the remaining frame (LDR frame). The human visual system is sensitive to these human factors. Therefore, there is no need to have a dedicated backlight frame construction to provide a uniform backlight frame.

The present invention is directed to solving some of the shortcomings of the prior art. The encoding method of the frame includes a processor, which comprises: encoding the backlight frame determined by the frame to be encoded; dividing the components of the frame by the backlight frame Decoding version, obtaining at least one component of the remaining frame; encoding the remaining frame; the method is characterized in that the processor is further configured to represent the backlight frame by a weighted linear combination of at least one 2D shape function set, each set The 2D shape function forms a unitary segmentation using the domain defined by its support union.

According to a specific example, the 2D shape function is defined by the smoothness constraint on its support boundary.

According to a specific example, the set 2D shape function has the shape of a model symmetry function.

According to a specific example, the 2D shape function is defined across a regular grid.

According to a specific example, the model symmetry function is the product of two separable 1D polynomials.

According to a specific example, the backlight frame is represented by a weighted linear combination of a set of 2D shape functions, and the 2D shape function of each set forms a single score of the domain defined by the support combination. Cut.

According to a specific example, the coefficients of the weighted linear combination and/or the 2D shape function are encoded in the bit stream.

The present invention further relates to a method for decoding a frame, which is decoded from at least one bit stream representing a back frame obtained by the frame and dividing the frame by the remaining frame calculated by the backlight frame. The decoding method comprises a processor, which comprises: decoding at least part of the bit stream to decode the backlight frame; decoding at least part of the bit stream to decode the remaining frame; and decoding the remaining frame by multiplying the decoded backlight frame The method is characterized in that the processor is further configured to: in order to represent the decoded backlight frame by a weighted linear combination of at least one 2D shape function set, the 2D shape functions of each set form a domain defined by the support combination. Single segmentation.

According to a specific example, the backlight frame is represented by a weighted linear combination of a set of 2D shape functions, and the 2D shape function of each set forms a unitary segmentation of the domain defined by the support union.

According to a specific example, the processor is further configured to obtain a 2D shape function using at least a partial decoding of the bit stream and/or a 2D shape function of the 2D shape function set to at least partially decode the bit stream, the coefficients of the weighted linear combination.

According to other gist, the present invention relates to encoding and decoding devices, computer programs, processor readable media, and non-transitory storage media.

The particular nature of the present invention, as well as other objects, advantages, features, and uses of the present invention, will be apparent from the description of the preferred embodiments illustrated herein.

10‧‧‧The module IC obtains the photometric component L and the potential at least one color component C(i) of the frame I

11‧‧‧Module BAM determines the backlight frame Bal from the luminance component L of frame I

12‧‧‧Determining the information required for the backlight frame Bal, encoded by the encoder ENC1 and added to the bit stream BF

13‧‧‧ Frame divided by the decoded version of the backlight frame , calculate the remaining frame Res

14‧‧‧ Decode at least part of the bit stream BF using the decoder DEC1 to obtain the decoded version of the backlight frame

15‧‧‧Modular BAG from weighting coefficient And some known non-adaptive shape functions , the decoded version of the backlight frame occurs

16‧‧‧Modular TMO tone mapping residual frame Res to obtain visible residual frame Res

17‧‧‧Module SCA scale visible residual frame Res

18‧‧‧Modular CLI clip visible residual frame Res

19‧‧‧Encoding Residual Frame Res with Encoder ENC2

81‧‧‧ Decode at least part of the bit stream F by the decoder DEC2 to decode the remaining frame

82‧‧‧Modular ISCA for decoding residual frames Applying an inverse scale, the remaining frame will be decoded Divide by parameter

83‧‧‧Modular ITMO utilization parameters Decoding the remaining frame Anti-reverse tone mapping

84‧‧‧Decoded residual frame Multiply by the backlight frame Decode frame

90‧‧‧ device

91‧‧‧ Data and address bus

92‧‧‧Microprocessor

93‧‧‧Read-only memory

94‧‧‧ Random Access Memory

95‧‧‧I/O interface

96‧‧‧Battery Pack

A, B‧‧‧Remote device

NET‧‧‧Communication Network

Figure 1 is a block diagram of the steps of the specific frame I coding method in the present case; Figure 2 is an embodiment of a 2D shape function support; and Figure 3 is a model symmetry function on a triangular mesh. Embodiment 4; FIG. 4 is a block diagram showing a method of using four sets of 2D shape functions defined on a rectangular grid; FIG. 5 is a block diagram of a method step of a specific example of the present embodiment; Block diagram; Figure 7 is a block diagram of the method steps of the specific example of the present invention; Figure 8 is a block diagram of the method steps of the specific example of the present case, and decoding the bit stream of the remaining frame calculated by dividing the frame by the backlight frame; 9 is an embodiment of a device structure of a specific example of the present invention; FIG. 10 is a second remote device for communicating through a communication network in the specific example of the present invention.

The present invention is described below with reference to the accompanying drawings, which show specific examples of the present invention. However, this content may be embodied in many alternative forms and should not be construed as being limited to the specific examples set forth below. Therefore, although the present invention allows various modifications and alternative forms, the specific examples exemplified in the drawings will be described in detail below. However, it is to be understood that the present invention is not intended to be limited to the particular forms disclosed, but rather, all modifications, equivalents and variations are intended to be included within the scope of the scope of the invention. Throughout the description of the figures, the same reference numerals refer to the same elements.

The terminology used herein is intended to be illustrative of specific embodiments and is not intended to limit the invention. The use of the singular "-" is also intended to include the plural unless the context clearly dictates otherwise. In addition, the words "including" and "comprising" are used in the specification to refer to the meaning of the features, integers, steps, operations, components and/or components, but do not exclude one or more other features, integers, steps, operations. The presence or addition of components, components, and/or groups thereof. In addition, when the elements referred to are "responsive" or "connected" to another component, the other components may be directly adapted or connected, and the intervening components may be present. In contrast, when the component referred to is "directly involved" or "directly connected" to other components, no intervening component exists. Where "and/or" is used to include any and all combinations of one or more associated list items, the abbreviation can be "/".

It should be noted that although the terms first, second, etc. may be used to describe the elements, such elements are not limited by the terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and a second element could be termed a first element, and the teachings of the present invention are not limited.

Although some of the diagrams have arrows indicating the communication path, indicating the main communication direction, it should be noted that the communication can also occur in the opposite direction of the arrow shown.

Some specific examples are illustrated in the block diagram and the operational flow, where the blocks represent a circuit component, module, or partial code, including one or more executable instructions to implement a particular logic. Edit the function. It should also be noted that in other embodiments, the function of annotation in the block may occur without the order of annotation. For example, the two blocks shown in succession may in fact be executed substantially simultaneously, and sometimes in reverse order, depending on the functionality involved.

References to "a specific example" or "specific" are intended to mean a particular feature, structure, or feature of the particular example, and may be included in at least one embodiment of the present invention. There are many places in the specification that are "in a specific example" or "in accordance with a specific example", and not necessarily all of the same specific examples, nor are they separate or alternative specific examples that must exclude other specific examples from each other.

The reference numbers appearing within the scope of patent application are for reference only and have no limiting effect on the scope of patent application.

Specific examples and variations of the present invention can be employed in any combination or sub-combination unless otherwise indicated.

This case is described in the encoding/decoding frame, but can be extended to the encoding/decoding of the frame sequence, because each frame contained in the sequence is encoded/decoded in the following order.

Figure 1 is a block diagram showing the steps of the encoding method according to the specific example of the present case.

In step 10, the module IC obtains the luminance component L of the frame I to be encoded and the potential at least one color component C(i).

For example, when the frame I belongs to the color space (X, Y, Z), the luminance component L can be obtained by using the transformation f(.) of the component Y, for example, L = f (Y).

When the frame I belongs to the color space (R, G, B), the linear component of the following formula can be used to obtain the luminance component L, for example, the 709 color gamut: L = 0.2127.R + 0.7152.G + 0.0722.B

In step 11, the module BAM determines the backlight frame Bal from the luminance component L of the frame I.

According to the specific example of step 11 shown in FIG. 5, the module 1B1 determines the backlight frame Ba, which is represented by a weighted linear combination of at least one set of 2D shape functions.

In terms of mathematics, the backlight frame Ba is specified as: among them Weighting coefficient, J set of shape functions as a 2D 2D shape function.

The 2D shape function of each set j forms a unitary segmentation of the domain Do defined by its support union, that is, also defined, 2D shape function With ingenious support, the total Do total is equal to 1:

According to the specific example of step 11, the 2D shape function is defined by a full regular grid.

As a variation of this specific example, a rectangular grid can be considered to be multiplied in two directions using a square grid.

According to the specific example of step 11 shown in Fig. 2, the 2D shape function Obtained as follows.

Assume 2D shape function , , with The centers are (0,0), (0,1), (1,0) and (1,1), and in the regular grid (here is a rectangular grid), the four 2D shape functions have their own Its support is contained in four grid elements around its center. Therefore, the domain Do is defined by the combination of four grid elements around the center of each 2D shape function. The singularity condition of the domain Do is divided: ψ ₁ + ψ ₂ + ψ ₃ + ψ ₄ =1

According to the specific example of step 11, the 2D shape function of the set j is a model symmetry function. Translation function, ie . In this case, the singularity condition of the domain Do is divided: Any of the grid elements E shown in Fig. 2.

According to the specific example of step 11, the model symmetry function Is the product of two separable 1D (single-dimensional) polynomials P(x) and P(y), ie (x, y) = P(x)P(y).

When it is consistent with the singularity condition, that is, P(x)+P(1-x)=1, that is, the singularity conditionalization of the 2D shape function is obtained because P ( x ) P ( y )+ P ( 1- x ) P ( y )+ P ( x ) P (1- y )+ P (1- x ) P (1- y )=( P ( x )+ P (1- x ))( P ( y ) + P (1- y ))=1

According to the specific example of step 11, the 2D shape function It is defined by the smoothness constraint on its support boundary.

This allows the continuity of the 2D shape function in the full frame domain and the smoothness of the global representation of the backlight frame Ba, since the finite total of the smoothing functions is smooth.

According to the specific example of step 11, the constraint on the boundary of the 2D shape function is: ̇ boundary constraint P(0)=1, meaning that only a 2D shape function is bet on its center; 连续 Continuity and derivative continuous constraint with respect to the branch boundary of the 2D shape function: P(1)=P'(1)=0; 导 derivative constraint at the center: P'(0)=0 to ensure the center Smoothness around.

The analysis solution of the 1D polynomial P with the above four boundary constraints can be referred to.

For example, the solution to the 1D function P is from P ( x ) = θx ³ + x ² + kx + λ yields three more than one. The above constraints result in the following conditions for the parameters:

- 1=P(0)=λ

- 0=P(1)=θ+ +κ+λ

- 0=P'(1)=3θ+2 +κ

- 0=P’(0)=κ

This linear equation system is easy to solve, and θ=2, =-3, κ=0, λ=1, and the model symmetry function (x, y) got:

Other polynomial embodiments are obtained by some smoothness relaxation. For example, if only continuity is required, it will be noted that P(x)=1-x is a continuous solution, resulting in a single segmentation with the following model symmetry function: Its derivatives are not continuous.

In addition, higher order paranorms can be sought. For example, the above four conditions apply to the forward-order polynomial P ( x ) = μx ³ + θx ³ + When x ² + κx + λ , the linear system under constraints is constrained by a parameter α, thus resulting in the infinity of the polynomial solution P _α .

One way to decide which one is best is to choose the parameter α to minimize the Laplace operator, namely:

Finally, from the analysis of the P minimization problem as follows, a pure numerical solution of non-analysis can be obtained: Wherein not necessarily seen polynomial P, but any sufficiently smooth to function, for example, stage C ^1. The numerical solution to the minimization problem is described in detail below.

Figure 3 shows the model symmetry function when the normal grid is a triangular grid. An embodiment.

Assume that the centers of the 2D shape functions ψ ₁ , ψ ₂ and ψ ₃ are pt1=(0,0), pt2=(-1/3,sqrt(3)/2), and pt3=(1/3,sqrt(3) ) / 2), on a triangular mesh, the three 2D shape functions each have their own count, contained within a hexagonal triangle around their center (Figure 3). Therefore, the domain Do is defined by the union of six triangles around the center of the 2D shape function. The division of the Do single condition is: ψ ₁ + ψ ₂ + ψ ₃ =1 (2)

According to the specific example of step 11, the 2D shape function has a model symmetry function. The shape has a single triangle.

Model symmetry function When defining a single triangle, a representation of the backlight frame Ba 2D shape function support (here, the six triangles around its center) that are segmented according to the singularity condition is obtained.

According to the specific example of step 11, the model symmetric function of the triangle is supported. Is to define its boundary constraints.

This allows the continuity of the 2D shape function in the full frame domain and the smoothness of the global representation of the backlight frame Ba.

According to the specific example of step 11, the model symmetry function The constraints of the boundary are:

Finally, determine the model symmetry function Is the solution to the minimization problem: minimal ∥ Oh, make the symbol conditions (2) and (3) maximize the smoothness of the solution.

The domain Do is separated, and then minimized under the constraint of boundary and unitary segmentation. The numerical solution can be found in a standard way. See Raviart and Thomas in conjunction with "Introduction à l'analyse numérique des équations aux dérivées partielles" for an introduction to the numerical tools involved.

In short, a mesh occurs on the domain Do, model symmetry function That is, take a discrete value for each grid. These discrete values can be aggregated into an unknown vector x . The Laplacian operator itself is separated under the matrix M, so that the operator △ The discrete form becomes Mx . Conditions (2) and (3) result in a matrix system Cx =0. Therefore, the minimization problem becomes: minimization ∥ Mx ∥ causes Cx =0

This is a linear least squares minimization problem under traditional constraints. There are many solutions to find the solution x , which leads to the answer Separate estimates. See PEGill, W. Murray and MHWright in conjunction with Practical Optimization, London Academic Press, 1981.

According to the specific example of step 11, the backlight frame is represented by a weighted linear combination of a set of 2D shape functions, and the 2D shape function of each set forms a unitary segmentation of the domain defined by its support combination.

Figure 4 shows a representation of four sets of 2D shape functions defined using a full rectangular grid. The black map represents the center of the 2D shape function of the first set, the triangle is the center of the second set, the ellipse is the center of the third set, and the fork is the center of the fourth set.

The total of all 2D shape functions for all sets is constant across the grid, where the value is 4. Therefore, a single segmentation is obtained, which is a multiple thereof. The focus is on the 2D shape function superposition, which is more than a single set of 2D shape functions as described in Figures 2 and 3. Using a number of sets of 2D shape functions, you can get more overlaps and use the 2D shape function closer to the center, that is, the shape function density is higher, to smoothly follow the HDR signal, and more accurate.

This case is not limited to the special set number of 2D shape functions, or the way they are combined on the grid, or the type of mesh used to locate the 2D shape function center.

According to the specific example or the modification of step 11, the backlight frame Bal outputted in step 11 is the backlight frame Ba obtained by the equation (1).

According to the specific example of step 11 shown in FIG. 6, the module BM modulates the backlight frame Ba (from equation (1)) by using the average luminance value L _{mean of the} frame I obtained by the module HL.

According to a specific example, the backlight frame Bal outputted in step 11 modulates the backlight frame.

According to a specific example, the module HL constitutes an average luminance value L _{mean for} calculating the total luminance component L.

According to a specific example, the module HL constitutes a calculated average luminance value L _mean , which is utilized. Where β is a coefficient less than 1, and E(X) is the mathematical expected value (average) of the luminance component L.

This last specific example is advantageous because it is avoided that the average luminance value L _mean is affected by a few extremely high-value pixels that tend to cause troublesome time-averaged luminance instability when the frame I belongs to the frame sequence.

This case is not limited to the specific example of calculating the average luminance value L _mean .

According to the variation of the specific example shown in Fig. 7, the module N normalizes the backlight frame Ba (obtained by the equation (1)) by using the average value E(Ba), which can be a frame (if the frame I belongs to the message) The frame sequence is all the frames. Mid-gray-at-one backlight frame Ba _gray :

Then, the module BM is configured to use the following relationship, the average luminance value L _{mean of the} frame L, and the grayscale backlight frame Ba _{gray in the} modulation: Where cst _mod is the modulation factor and α is another modulation factor, less than 1, usually 1/3.

According to this variation, the backlight frame Bal outputted in step 11 is obtained from the modulated backlight frame Ba _{mod of} equation (4).

It should be noted that the modulation factor cst _mod is tuned to give a good look at the brightness of the remaining frames, depending on the process of obtaining the backlight frame. For example, for a least squares backlit frame, cst _mod 1.7.

In fact, using linearity to modulate all operations of the backlight frame, the backlight factor can be applied. As a correction factor, the coefficient Transform into a new coefficient ,Available:

In step 12 (Fig. 1), the data required for the backlight frame Bal outputted in step 11 is determined, encoded by the encoder ENC1, added to the bit stream BF, and stored in a local or remote memory, and/or via communication. Interface transfer (for example to bus, or via a communication network or broadcast network).

For example, when a known non-adaptive shape function is used, the data to be encoded is limited to weighting coefficients. or , but 2D shape function It can also be previously unknown, and then encoded in the bit stream BF, for example, a little optimal mathematical construction for better coordination. All weighting factors or (and potential shape function ) are all encoded into the bit stream BF.

The advantage is that the weighting factor or Both are quantized before encoding to reduce the bit stream BF scale.

In step 13, the borrowing box is divided by the decoded version of the backlight frame. , calculate the remaining frame Res.

Should use the decoded version of the backlight frame , to ensure that the same backlight frame on the encoder and decoder side, resulting in the final decoding frame Better and more accurate.

More precisely, the frame I luminance component L and the potential color component C(i) obtained by the module IC are divided by the decoded version of the backlight frame. . This division is performed pixel by pixel.

For example, when the components R, G, and B of the frame I are expressed in the color space (R, G, B), the scores R _Res , G _{Res ,} and B _Res are as follows:

For example, when the component X, Y or Z of the frame I is expressed in the color space (X, Y, Z), the scores X _Res , Y _Res and Z _Res are as follows:

According to a specific example, in step 14, the decoder DEC1 is used to decode at least part of the bit stream BF, thereby obtaining a decoded version of the backlight frame. .

As described above, step 11 outputs certain data required for the backlight frame, which has been encoded (step 12), and then decoded at least part of the bit stream BF.

Following the above embodiment, the weighting coefficient is obtained. (and potential shape function ), as step 14 output.

Then, in step 15, the module BAG is weighted by the coefficient And some known non-adaptive shape functions, or shape functions , the decoded version of the backlight frame occurs , as follows:

At step 16, the module TMO tone maps the remaining frame Res to obtain the visible frame Res _v .

It is possible that the remaining frame Res is invisible because its dynamic range is too high, and the decoded version of the remaining frame Res also shows visual artifacts. Tone mapping of the remaining frames can solve at least one of these shortcomings.

This case is not limited to any special tone mapping operator. This single condition is that the tone mapping operator should be reversible.

For example, a tone mapping operator defined by Reinhard can be used, see Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J., "Photographic tone reproduction for digital frames". 〉, ACM Transactions on Graphics 21 (July 2002), or Boitard, R., Bouatouch, K., Cozot, R., Thoreau, D., and Gruson, A. (2012). "Temporal coherency for video tone mapping", published in AMJvan Eijk, CCDavis, SMHammel, and AKMajumdar, Proc. SPIE 8499, Applications of Digital Frame Processing (p.84990D- 84990D-10).

In step 19, the visible residual frame Res _v is encoded in the bit stream F by means of the encoder ENC2, can be stored in a local or remote memory, and/or transmitted via a communication interface (for example, in a bus, or via a communication network or Broadcast network).

According to the specific example of step 16, the tone mapping residual frame includes gamma correction or SLog correction, according to the pixel value of the remaining frame.

For example, the residual frame Res _v : Res _v = A.Res ^γ is used to obtain the A-line constant, and the γ-based gamma curve coefficient is, for example, equal to 1/2.4.

In addition, the visible residual frame Res _v is obtained by, for example, the following equation: Res _v = a .ln( Res + b ) + c where a, b, c are SLog curve coefficients, determining 0 and 1 are unchanged, and the SLog curve is The derivative can be continuous at 1 when the gamma curve is extended by 1 or less. Therefore, a, b, and c are functions of the parameter γ.

According to a specific example, the parameter γ of the gamma-Slog curve is encoded in the bit stream BF.

When applying gamma correction to the remaining frame Res, pull the dark area, but not lower than enough height to avoid burning the bright pixels.

Apply SLog correction to the remaining frame Res to reduce the light level enough, but not Pull the dark area.

Then, according to a preferred embodiment of step 16, the module TMO applies gamma correction or SLog correction according to the pixel value of the remaining frame Res.

For example, when the pixel value of the remaining frame Res is lower than the threshold (equal to 1), the gamma correction is applied, otherwise the SLog correction is applied.

With the construction, it is particularly effective to use the above gamma-Slog combination, depending on how often the average frame Res _v is close to 1, depending on the luminosity of the frame I.

According to the method specific example, in step 17, the module SCA scale is visible to the residual frame Resv, and then the code (step 19) multiplies the components of the visible residual frame Resv by the scaling factor cst _scaling . The residual frame Res _s : Res _s = cst _scaling .Res _v

In a particular embodiment, the _scaling factor cst _{scaling is} defined to map the visible residual frame Res _v value between 0 and the maximum value 2 ^N -1, where N is the number of bits that the input is allowed to be coded by the encoder ENC2 .

This borrowing value 1 (roughly the average of the visible residual frame Res _v ) is mapped to the medium gray value 2 ^N-1 naturally. Therefore, for a visible residual frame Res _v having a standard number of bits N=8, the scale factor equal to 120 is a very consistent value because the near neutral gray level is 2 ⁷ = 128.

According to this specific example of the method, in step 19, the remaining frame Res _s is encoded by the encoder ENC2.

According to a specific example of the method, in step 18, the module CLI clips the visible frame Res _v and then encodes to limit its dynamic range to the target dynamic range TDR, for example according to the encoder ENC2 capability.

According to this last specific example, for example, according to the specific example of the method, the residual frame Res _{c is} obtained by the following formula: Res _c = max(2 ^N , Res _v )

^{_{Res c = max (2 N,}} Res s)

This case is not limited to such a clip (max(.)), but can be extended to any kind of clip.

According to this particular embodiment of the method, at step 19, the remaining information block Res _c are encoded by the encoder ENC2.

Combining the scale and the clip specific example, resulting in the following example, the residual frame Res _sc : Res _sc = max (2 ^N , cst _scaling * Res _v ) or Res _sc = max (2 ^N , cst _scaling * Res _s )

According to this specific example of the method, in step 19, the residual frame Res _sc is encoded by the encoder ENC2.

The tone mapping and scale of the visible residual frame Res _v is a parametric process. The parameter can be fixed or not. If it is not solid, it can be encoded in the bit stream BF by the encoder ENC1.

According to a specific example of the method, the gamma correction constant value γ, the scaling factor cst _scaling can be a parameter, encoded in the bit stream BF.

The parameters α, cst _mod , cst _scaling , γ, β should be selected to make the tone mapping have a choice space, suitable for the best content, and follow the taste of experts in post-production and color grading.

On the other hand, the general parameters can be defined so that most of the frames can be received. However, no parameters are encoded into the bit stream BF.

Figure 8 shows the flow of the method steps. According to the specific example of the present case, the bit stream divided by the frame of the backlight frame is decoded and decoded.

As described above, in steps 14 and 15, the backlight frame For example, the bit stream BF is at least partially decoded by the decoder DEC1.

The bit stream BF may have been stored locally or received from a communication network.

At step 81, the bit stream F is at least partially decoded by the decoder DEC2 to obtain a decoded residual frame. .

The bit stream F has been stored locally or received from a communication network.

Decoding the remaining frames as described later , visible using traditional devices.

At step 84, by decoding the remaining frame Multiply by the backlight frame Decode frame .

According to the specific example of step 14, whether the bit stream BF is at least partially decoded from the local memory or by the decoder DEC1, the parameter may be obtained. and / or .

According to the method, in step 82, the module ISCA decodes the remaining frame. Applying an inverse scale, the remaining frame will be decoded Divide by parameter .

In step 83, the module ITMO utilizes parameters , decoding the remaining frame Apply anti-reverse tone mapping.

For example, parameters Defining the gamma curve, and the inverse tone mapping is just from the gamma curve, finding the value, equivalent to decoding the residual frame The pixel value.

The decoders DEC1 and DEC2 are constructed to decode the encoded data of the encoders ENC1 and ENC2, respectively.

The encoders ENC1 and ENC2 (and decoders DEC1 and DEC2) are not limited to special encoders (decoders), but if an entropy encoder (decoder) is required, an entropy coder such as a Huffmann coder, an arithmetic coder or a context adaptation Encoders, such as those used by h264/AVC or HEVC, are preferred.

The encoders ENC1 and ENC2 (and decoders DEC1 and DEC2) are not limited to special encoders and may for example be lost frame/video encoders like JPEG, JPEG2000, MPEG2, h264/AVC or HEVC.

In Figures 1-8, the modules are functional units that are related to the identifiable physical units. For example, such modules or parts may be placed together in a unique component or circuit, or may be functional in software. Conversely, some modules may potentially consist of separate entities. Devices compatible with this case can be implemented in pure hardware, such as using professional hardware such as ASIC, FPGA or VLSI, representing application-specific integrated circuits, external field programmable gateway arrays, very large integrated circuits, or A plurality of integrated electronic components are embedded in the device or are mixed by hardware and software components.

Fig. 9 shows an example of the configuration of the device 90, which can constitute a method described in the first to eighth embodiments.

Apparatus 90 includes the following elements linked together by data and address bus 91: ̇ a microprocessor (or CPU) 92, such as a DSP (digital signal processor); ̇ROM (or read only memory) 93; RAM (random access memory) 94; ̇ I/O interface 95, receiving data to be transmitted from the application; ̇ battery pack 96.

According to a variant, the battery pack 96 is external to the device. The elements of Figure 9 are well known to those skilled in the art and need not be described. In each of the memories, the "scratchpad" used in the specification is equivalent to a small-capacity area (several bits) or a large area (for example, the entire program, or a large amount of received or decoded data). The ROM 93 includes at least one program and a number of parameters. The calculation method of the method of this case is stored in ROM 93. When turned on, the CPU 92 uploads the program in the RAM and executes the corresponding command.

The RAM 94 includes a program executed by the CPU 92 in the register, and uploads the input data in the temporary register, the intermediate data in different states of the method in the temporary register, and other methods used in executing the method in the temporary memory. variable.

The embodiments described herein are implemented, for example, in a method or process, apparatus, software program, data stream, or signal. Even if only a single implementation is discussed in the context (for example, only methods or devices are discussed), the implementation of the features discussed may be implemented in other forms (eg, programs). The device can be implemented, for example, as a suitable hardware, software and firmware. The method can be implemented, for example, in a device, such as a processor, generally referring to a processing device, such as a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes communication devices such as computers, cell phones, portable/personal digital aids (PDAs), and other devices that facilitate communication of information between end users.

According to a specific example of the encoding or encoder, the frame I is taken from the source, and the source belongs to a set, including: ̇ local memory (93 or 94), such as video memory or RAM (random access memory), Flash memory, ROM (read only memory), hard disk; ̇ storage interface (95), such as interface with a large number of memory, RAM, flash memory, ROM, optical disk or magnetic pedestal; ̇ communication interface ( 95), such as a wired interface (such as a bus interface, a wide area network interface, a local area network interface), or a wireless interface (such as an IEEE 802.11 interface or a Bluetooth® interface); a frame capture circuit (such as a sensor such as a CCD) (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor).

Decoding frame according to different specific examples of decoding or decoder Send to the destination; specifically, the destination belongs to the following set: ̇ local memory (93 or 94), such as video memory or RAM, flash memory, hard disk; ̇ storage interface (95), for example Interface with a large number of storage devices, RAM, flash memory, ROM, optical disk or magnetic base; ̇ communication interface (95), such as wired interface (such as bus interface (like USB (universal serial bus)), wide area network Road interface, local area network interface, HDMI (High Definition Multimedia Interface) interface, or wireless interface (such as IEEE 802.11 interface, WiFi® or Bluetooth® interface); ̇ display.

The bitstreams BF and/or F are sent to the destination according to different specific examples of the code or encoder. For example, both bitstreams F and BF are stored in a local or remote memory, such as video memory (94) or RAM (94), hard disk (93). In a variant, one or two bit streams are sent to the storage interface (65), for example to a mass storage interface, a flash memory, a ROM, a compact disc or a magnetic pedestal, and/or to transmit across a communication interface ( 95), for example, an interface with a point-to-point link, a communication bus, a point-to-multipoint link, or a broadcast network.

Depending on the specific example of the decoder or decoder, the bitstreams BF and/or F are obtained from the source. For example, the bit stream is read by a local memory, such as video memory (94) or RAM (94), ROM (93), flash memory (93), or hard disk (93). In a variant, the bit stream is received from the storage interface (95), for example with a large number of memory interfaces, RAM, ROM, flash memory, optical or magnetic base, and/or from the communication interface (95) Receive, for example, an interface to a point-to-point link, a bus, a point-to-multipoint link, or a broadcast network.

According to different specific examples, the device 90 constitutes an implementation of the encoding method described in relation to Figures 1-7, and includes the following collections: ̇ mobile device; ̇ communication device; ̇ game device; ̇ tablet (or tablet); ̇ lap Computer; ̇ still frame camera; ̇ video camera; ̇ coded chip; ̇ stationary frame server; ̇ video server (such as broadcast server, video server, or web server).

According to different specific examples, the device 90 is configured to implement the decoding method described in FIG. 8 and belongs to the following set: ̇ mobile device; ̇ communication device; ̇ game device; 上 set-top box; ̇ TV; ̇ tablet (or tablet); ̇ laptop; ̇ display;

According to the specific example shown in FIG. 10, when the communication between the two remote devices A and B via the communication network NET, the device A includes a mechanism for implementing the coding method for the frame described in FIG. 1, and the device B includes The mechanism constitutes a decoding method described in the eighth diagram.

According to a variant of the present invention, the network is a broadcast network adapted to broadcast a stationary frame or a video frame from device A to a decoding device comprising device B.

Embodiments of the various processes and features described herein may be exemplified by various devices or applications. The device example includes an encoder, a decoder, a processor after processing the decoder output, a preprocessor providing input to the encoder, a video encoder, a video decoder, a video write decoder, a web server, a set-top box, and a laptop. Computers, personal computers, mobile phones, PDAs, and other communication devices. It can be seen that the device can be mobile or even mounted on a mobile vehicle.

In addition, the methods may be implemented by instructions executed by a processor, and such instructions (and/or data values generated by an embodiment) may be stored in a processor readable medium, such as an integrated circuit, a software carrier, or other storage device, such as A hard disk, a compact disc (CD), a compact disc (such as a DVD, often referred to as a digitally diverse optical disc or a digital video disc), a random access memory (RAM), or a read-only memory (ROM). The instructions form an application that is tangibly embodied on the processor readable medium. The instructions may be, for example, in hardware, software or firmware, or a combination. Instructions can be found, for example, in an operating system, separately applied, or a combination of both. Therefore, the processor can be arbitrarily configured to constitute a device for performing the process, or a device (such as a storage device) containing the processor-readable medium, having instructions for performing the process. Moreover, the readable media can store the data values generated by the implementation in addition to or in place of the instructions.

All technical experts know that the implementation can generate various signals, and can be stored or transmitted, for example, by carrying information. The information may include, for example, instructions for performing the method, or the above Information produced by one of the methods. For example, the signal may be formatted, carrying the grammar rules of the above specific example as data, or carrying the actual grammar value written in the above specific example. These signals may be formatted to exhibit, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding the data stream, and modulating the carrier with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. It is well known that signals can be transmitted across a variety of different wired or wireless links. The signal can be stored in a processor readable medium.

Many embodiments have been described above. However, the instructions are subject to various modifications. For example, elements of different embodiments may be combined, supplemented, modified, or removed to yield other embodiments. In addition, the skilled artisan will appreciate that other configurations or processes may be utilized as described above, and that the resulting embodiments may perform at least substantially the same function as the above-described embodiments, achieving at least substantially the same results in at least substantially the same manner. Therefore, the above and other embodiments are contemplated in the present case.

10‧‧‧The module IC obtains the photometric component L and the potential at least one color component C(i) of the frame I

11‧‧‧Module BAM determines the backlight frame Bal from the luminance component L of frame I

12‧‧‧Determining the information required for the backlight frame Bal, encoded by the encoder ENC1 and added to the bit stream BF

13‧‧‧ Frame divided by the decoded version of the backlight frame , calculate the remaining frame Res

14‧‧‧ Decode at least part of the bit stream BF using the decoder DEC1 to obtain the decoded version of the backlight frame

15‧‧‧Modular BAG from weighting coefficient And some known non-adaptive shape functions , the decoded version of the backlight frame occurs

16‧‧‧Modular TMO tone mapping residual frame Res to obtain visible residual frame Res

17‧‧‧Module SCA scale visible residual frame Res

18‧‧‧Modular CLI clip visible residual frame Res

19‧‧‧Encoding Residual Frame Res with Encoder ENC2

Claims (9)

A method for encoding a frame, comprising: encoding (12) a backlight frame determined by the frame to be encoded (11); and dividing the components of the frame by the decoded version of the backlight frame to obtain 13) at least one component of the remaining frame; 编码 encoding the remaining frame (19); the backlight frame is represented by a weighted linear combination of at least one set of 2D shape functions, and the 2D shape function of each set is formed by using the joint combination A unitary segmentation of a domain; a set of 2D shape functions having the shape of a model symmetry function; wherein the weighted linear combination coefficients and/or 2D shape functions are encoded within the bitstream. A frame encoding device, comprising a processor, is configured to: encode a backlight frame determined by a frame to be encoded, and encode the components of the borrowing frame by a decoded version of the backlight frame to obtain a residual frame. At least one component; 编码 encoding the remaining frame; the backlight frame is represented by a weighted linear combination of at least one set of 2D shape functions, and the 2D shape function of each set forms a singular segmentation using the domain defined by its support combination; The set 2D shape function has the shape of a model symmetry function; wherein the coefficients of the weighted linear combination and/or the 2D shape function are encoded within the bit stream. A method for decoding a frame, from at least one meta-stream, representing a backlight frame obtained by the frame, and dividing the frame by a residual frame calculated by the backlight frame, the method comprising the processor, comprising: ̇ utilizing a bit stream at least Partially decoding, obtaining (14) decoding the backlight frame; ̇ using at least partial decoding of the bit stream to obtain a decoded residual frame; 乘 multiplying the decoded residual frame by the decoded back frame to obtain (51) the decoded frame; The decoding backlight frame is represented by a weighted linear combination of at least one set of 2D shape functions, and the 2D shape function of each set forms a unitary segmentation using the domain defined by its support combination; wherein a set of 2D shape functions has a model symmetry function a shape; wherein the 2D shape function weights the linear combination of coefficients, which are at least partially decoded by the bit stream The 2D shape function of the set of and/or 2D shape functions is obtained by at least partial decoding of the bit stream. A frame decoding device, which comprises at least one bit stream representing a backlight frame obtained by a frame, and a frame divided by a remaining frame calculated by a backlight frame, the device comprising a processor, comprising: ̇ utilizing a bit stream At least partial decoding, obtaining a decoded backlight frame; ̇ using at least partial decoding of the bit stream to obtain a decoded residual frame; 乘 multiplying the decoded residual frame by the decoded backlight frame to obtain a decoded frame; decoding the backlight frame Expressed by a weighted linear combination of at least one set of 2D shape functions, the 2D shape function of each set forms a unitary segmentation using the domain defined by its support union; wherein a set of 2D shape functions have the shape of a model symmetry function; Further, the block stream is at least partially decoded, and the 2D shape function is used to weight the linear combination of coefficients, and/or the bit stream is at least partially decoded to obtain a 2D shape function of the 2D shape function set. A computer program product, including a code code instruction, which, when executed on a computer, performs a coding method step of claim 1 of the scope of the patent application. A computer program product, including a code code instruction, which, when executed on a computer, performs a decoding method step of claim 3 of the scope of the patent application. A processor readable medium having instructions for storing therein to cause a processor to at least perform one of the encoding method steps of claim 1 of the patent scope. A processor readable medium having instructions for storing therein to cause a processor to perform at least one of the decoding method steps of claim 3 of the patent application. A non-transitory storage medium with program code instructions for performing a method step of claim 1 or 3 when the program is executed on a computing device.