TW201540052A - Method and device for encoding a high-dynamic range image into a bitstream and/or decoding a bitstream representing a high-dynamic range image - Google Patents

Abstract

The present invention generally relates to a method for encoding an image into a bitstream. The method is characterized in that it comprises: encoding (1100) into the bitstream an illumination map determined (1000) from the image; and encoding (1200) into the bitstream a signalization data indicating that the bitstream comprises the illumination map. The invention relates also a method and device for decoding a bitstream and also the bitstream itself.

Description

Encode high dynamic range images into bitstreams and/or will represent high dynamic range shadows Method and device for decoding bit stream

The present invention generally relates to image/video encoding and decoding, and more particularly to encoding of an image related to the technical field of the present invention, the pixel value of the image belongs to a high dynamic range, and the associated bit stream is decoded, and the bit stream represents a high Dynamic range image.

This paragraph is intended to introduce the reader to various aspects of the subject matter, which are various aspects of the invention described and/or claimed below. The discussion of this letter helps to provide readers with background information and a better understanding of the different aspects of the invention. Therefore, it should be understood that such statements will be read in this light and are not recognized as prior art.

A low dynamic range image (LDR image) is an image whose brightness value is represented by a finite number of bits (usually 8 or 10). This finite representation does not allow correct small signal variations, especially in dim and bright brightness ranges. . In high dynamic range images (HDR images), the extended signal is represented to maintain the signal with high accuracy throughout its range. In HDR images, pixel values are usually expressed in floating point format (for each component is 32 or 16 bits, called floating point or half floating point), and the most common format is open EXR (openEXR) half floating point format. (16 bits per RGB component, ie 48 bits per pixel), or expressed as an integer with a long representation, usually at least 16 bits.

A typical measure of encoding an HDR image is to reduce the dynamic range of the image, which is to be encoded by a conventional encoding scheme that is initially configured for LDR image encoding.

According to the first measure, the input HDR image is applied with a tone mapping operator, and then the tone mapped image is imaged by a conventional 8-10 bit depth coding scheme such as JPEG/JPEG200 or MPEG-2, H.264/AVC. Coding for video ("Advanced video coding for generic audiovisual services"), Series H: Audiovisual and multimedia systems, ITU-T Recommendation H.264, ITU Telecommunications Standardized Sector, January 2012). Next, an inverse tone mapping operator is applied to the decoded image, and a residual image is calculated between the input image and the decoded and inverse tone mapped image. Finally, the residual image is encoded by a second conventional 8-10 bit depth encoder scheme.

This first measure is backward compatible in the sense that a low dynamic range image can be decoded and displayed by a conventional device.

However, this first measure uses a two-coding scheme and limits the dynamic range of the input image to twice the dynamic range of the traditional encoding scheme (16-20 bits). In addition, such measures sometimes result in low dynamic range images and inputs. HDR images have a weak correlation, which results in low coding performance of the image.

According to the second measure, an illumination image is determined from the input HDR image, and then a residual image is obtained from the image and the illumination image, and the illumination image and the residual image are directly encoded.

This particular measure for an input HDR image encoding is not backward compatible with conventional devices that are unable to decode and/or display high dynamic range images.

Moreover, this measure cannot be used in a general communication infrastructure in which an HDR image is transmitted between two remote devices because the conventional transfer member of the infrastructure is not suitable for carrying the illumination map.

To remedy some of the shortcomings of the prior art, the present invention provides a method of encoding an image into a bit stream, comprising the steps of: - encoding a lighting pattern from the image into the bit stream; and - placing a signal The data is encoded into the bit stream, the signalized data indicating that the bit stream includes the illumination map.

According to an embodiment of the method, the method further comprises the step of: - encoding a residual image from the image and the illumination map into the bitstream.

According to an embodiment, the illumination pattern is encoded as an auxiliary image whose syntax conforms to the H264/AVC or HEVC standard.

In addition to the so-called "primary coded image", the auxiliary image has also been defined in the H264/AVC or HEVC standard, and the primary encoded image actually corresponds to the main stream of the content (main video). Auxiliary images can often be used to transmit additional image information such as alpha(alpha) synthesis, chroma enhancement Information or in-depth information for 3D applications.

According to an embodiment, the residual image is encoded as a primary image whose syntax conforms to the H264/AVC or HEVC standard.

This allows a one-bit stream representing an HDR image to be obtained, which is fully applicable to the H264/AVC or HEVC standard: auxiliary material (i.e., illumination map) is transmitted in the same order of the main image encoding order. The auxiliary data decoding method that occurs before the display conforms to the HEVC specification and is therefore used in its prescribed format.

According to an embodiment, the illumination image is a backlit image, and the residual image is obtained by dividing the image by a decoded version of the backlight image.

According to an embodiment, the residual image is tone mapped prior to encoding.

This step provides a visible residual image, i.e., a residual image of the resulting tonal mapping scene that is reasonably coordinated with the original scene in the image, as a result of the residual image. Since the visible residual image can be decoded and/or displayed by conventional devices that cannot handle high dynamic range, the method is backward compatible, and the old (non-HDR) H264/AVC or HEVC decoder can simply drop the illumination map. (It cannot be recognized by the old decoder), and only the residual image is decoded.

In addition, a high dynamic range image coding is achieved by such a method to achieve an efficient coding scheme, because the residual image after the tone mapping (low dynamic range image) and the backlight image are separately coded, and the residual image after tone mapping is used. Spatially highly correlated (and temporally related to other imagery of the same sequence of images). A coding gain is achieved due to the high compression ratio of the residual image after tone mapping and the high compression ratio of the small amount of data used for backlight image coding.

According to an embodiment, the tone mapping of the residual image includes a gamma correction or a SLog (S-logarithm) correction based on pixel values of the residual image.

Gamma correction and SLog correction result in no loss of dark and bright information, resulting in high accuracy of reconstructing an HDR image from residual images and backlit images. In addition, gamma correction and S-log correction avoid flattening the clipping region in both the reconstructed HDR image and the visible residual image.

According to an embodiment, the method further comprises scaling the residual image prior to encoding.

This step places the intermediate grayscale of an image from the residual image at an appropriate value for viewing and encoding both.

According to an embodiment, the method further comprises clipping the residual image prior to encoding.

Clipping the residual image ensures a finite number of bits and allows the residual image to be encoded using a conventional encoding/decoding scheme, and editing the residual image ensures that it is inversely related to the existing infrastructure (codec, display, distribution channel, etc.) Capacity, because only residual images (which have a low dynamic range, typically 8-10 bits) can be transmitted through such infrastructure to display a low dynamic range version of the image. A small bit stream containing backlight data can be carried in a side container through a dedicated infrastructure for distributing the original version of the image (ie, an HDR image).

According to another aspect of the invention, the invention relates to a method of decoding a bit stream, the bit stream representing an image, the method comprising the steps of: - detecting whether a signalized data in the bit stream indicates the bit The metadata stream includes a lighting pattern from which the data is determined; - by decoding the bit stream at least partially, to obtain a decoded illumination image; and - deriving a decoding from a decoded residual image and the decoded illumination image image.

According to an embodiment, the decoded residual image is obtained by at least partially decoding the bit stream.

According to an embodiment, the signalling data is detected from a higher-order syntax element and its use is accomplished by an SEI message.

According to an embodiment, the bit stream comprises a primary image and an auxiliary image, the syntax of which conforms to the standard H264/AVC or HEVC, and wherein the primary image represents a residual image and the auxiliary image represents an illumination map.

According to an embodiment, the decoded illumination image is a backlit image, and wherein the decoded residual image is multiplied by the backlight image to obtain a decoded image.

According to an embodiment, the decoded residual image is inversely mapped before the decoded residual image is multiplied by the backlit image.

According to an embodiment of the method, the illumination map is a low spatial frequency version of the luminance component of the image to be encoded, and by calculating a difference between the luminance component of the image and a decoded version of the encoded low spatial frequency version, Residual image.

According to another aspect of the invention, the invention relates to a bit stream representing an image, characterized by comprising a signalling material indicating that it represents a lighting pattern determined from the image.

According to another aspect of the present invention, the present invention relates to an apparatus for encoding an image, and an apparatus for decoding a bit stream, the apparatus implementing the above method.

The specific nature of the invention, as well as other objects, advantages, features and uses of the invention will be apparent from the accompanying drawings.

1000‧‧‧Lighting diagram determination steps (Fig. 1, 3, 8); device (Fig. 10)

1001‧‧‧ Data and address bus

1002‧‧‧Microprocessor (or CPU)

1003‧‧‧ROM (or read-only memory)

1004‧‧‧RAM (or random access memory)

1005‧‧‧I/O (input and output) interface

1006‧‧‧Battery

1007‧‧‧ display

1010, 1080‧‧‧ components get steps

1020‧‧‧Backlight image determination step

1030‧‧‧Division procedure (calculating residual image)

1040‧‧‧ Residual image tone mapping steps

1050‧‧‧ visible residual image calibration steps

1060‧‧‧ visible residual image editing steps

1070‧‧‧ Backlighting image getting steps

1090‧‧‧Steps for obtaining a low spatial frequency version of the luminance component

1100‧‧‧ Lighting pattern coding steps

1110‧‧‧ Low spatial frequency version coding step

1120‧‧‧Association steps

1130‧‧‧Perceptual threshold determination step

1140‧‧‧ Applying threshold steps

1200‧‧‧ Signaling data steps

1300‧‧‧Residual image coding steps

2000‧‧‧Test steps

2100‧‧‧ Lighting map getting steps

2200‧‧‧Steps to decode residual images

2300‧‧‧Steps to decode images

2310‧‧‧Reverse calibration steps

2320‧‧‧Anti-tone mapping steps

2340‧‧‧Multiplication steps

2350‧‧‧Addition steps

2360‧‧‧Anti-perception modulation steps

A, B‧‧‧ device

a _i , , ‧‧‧weighting factor

Ba, Bal‧‧‧ Backlit image

‧‧‧Backlight image decoding version

Ba gray _gray ‧‧‧ in a backlit images (mid-gray-at-one ) of

Ba _modulation, ‧ ‧ modulated backlight image

BAG‧‧‧Backlight Image Decoding Version Generation Module

BAM‧‧‧Backlight Image Bal Determination Module

BI‧‧‧Backlight Image Ba Judging Module

BM‧‧ ̄ modulation module

C(i) ‧ ‧ color component

CLI‧‧‧editing module

C ^N ‧‧‧ normalized component

Cst _calibration ‧ ‧ calibration factor

Cst _modulation ‧ ‧ modulation coefficient

DEC1, DEC2‧‧‧ decoder

Diff ‧‧‧differential image

‧‧‧Decoded version of the differential image

DIM, IM‧‧‧ illumination map

Residual images decoded by DRI‧‧‧

ENC1, ENC2‧‧‧ encoder

△ E , △ E ₀ , △ E _enc , TH‧‧‧ perception threshold

F, BF‧‧‧ bit flow

I‧‧‧ images

‧‧‧Decode image

IC‧‧‧ component acquisition module

IIC‧‧‧Anti-perception module

ISCA‧‧‧Reverse Calibration Module

ITMO‧‧‧Anti-tone mapping module

HL‧‧‧Brightness average calculation module

L‧‧‧luminance component

L _{average price} ‧ ‧ brightness average

L _lf ‧‧‧Low space frequency version

‧‧‧Decoded version of low spatial frequency version

L _r ‧‧‧differential brightness component

N‧‧‧ normalized module (Figure 4); number of available bits (step 14)

NET‧‧‧Communication Network

POP‧‧‧ decoding image acquisition module

PRP‧‧‧Lighting Chart Judgment Module

PT‧‧‧critical value determination module

QP, γ , ‧‧‧parameter

Res, RI‧‧‧ residual image

Res _c ‧‧‧calibrated tone map image

Res _s ‧‧‧ residual image as a result

Res _v ‧‧‧ visible residual image

‧‧‧Remaining images decoded

SCA‧‧‧ calibration module

SD‧‧‧Signalization data

SMD‧‧‧ test module

SME‧‧‧ Signaling Data Encoding Module

TDR‧‧‧ target dynamic range

TMO‧‧‧tone mapping image acquisition module

‧‧‧brightness

Y _n ‧‧‧Maximum ambient brightness value

ψ _i , ‧‧‧Shape function

Embodiments of the present invention will be described with reference to the accompanying drawings in which: FIG. 1 is a block diagram showing a step of a method of encoding an image into a bit stream in accordance with an embodiment of the present invention; FIG. 2 is a diagram of a method according to the present invention. The embodiment depicts, in a block diagram, a method of decoding a bit stream F, the bit stream representing an image; FIG. 3 is a block diagram showing a sub-step of step 1000 in accordance with an embodiment of the invention; FIG. 4 is in accordance with the present invention. An embodiment shows a sub-step of step 1020 in a block diagram; FIG. 5 shows a sub-step of step 1020 in a block diagram according to an embodiment of the invention; FIG. 6 shows step 1020 in a block diagram according to an embodiment of the invention. Next step; FIG. 7 shows a second step of step 2300 in a block diagram according to an embodiment of the invention; FIG. 8 shows a sub-step of step 1000 in a block diagram according to an embodiment of the invention; FIG. 9 is in accordance with the present invention. An embodiment shows a sub-step of step 2300 in a block diagram; FIG. 10 shows an architecture of a device according to an embodiment of the invention; and FIG. 11 shows a second remote device through a communication network according to an embodiment of the invention. through .

The invention will be described in more detail below with reference to the accompanying drawings, in which FIG. Accordingly, the present invention may be embodied in various embodiments and various modifications and However, it is to be understood that the invention is not intended to be limited to the specific forms disclosed, and the invention is intended to cover all modifications, equivalents and alternatives . Throughout the drawings, the same numerals indicate the same elements.

The terminology used herein is for the purpose of describing particular embodiments only and The singular forms "a", "the", "the" and "the" and "the" are intended to include the plural, unless the context clearly indicates otherwise. It should be understood that "including", "including", " The inclusion of the words "and/or" includes "the terms, integers, steps, operations, components, and / or components, but does not exclude one or more other features, integers, steps, operations, The presence or addition of elements, components, and/or groups. In addition, when a component is "responsive" or "connected" to another component, the component can directly respond to or be connected to the other component, or can be inserted. In contrast, when an element is referred to as being "directly responsive" or "directly connected" to another element, no intervening element is present. The term "and/or" as used herein includes one or more of the listed. Any and all combinations of associated items, and may be abbreviated as "/".

It should be understood that the terms first, second, etc. used herein are used to describe different elements, and such elements are not limited to such terms, and such terms are used only to distinguish one element from another. For example, a first element could be termed a second element, and a second element could be termed a first element, without departing from the teachings of the invention.

While some of the figures include arrows on the communication path to indicate a primary direction of communication, it should be understood that communication can occur in the opposite direction of the arrows shown.

Some embodiments are illustrated in block diagrams and operational flow diagrams, wherein each block represents a circuit component, module or portion code that includes one or more executable instructions for implementing the specified (several) logic functions. Moreover, it should be noted that in other implementations, the (equal) functions shown in the blocks may not occur in the order shown. For example, depending on the functionality involved, the two blocks shown as contiguous are in fact executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order.

References to "an embodiment" or "an embodiment" are intended to mean that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one implementation of the invention. The appearances of the "a" or "an" or "an"

The reference numerals appearing in the scope of the patent application are not limited by the scope of the patent application.

Although not explicitly stated, embodiments and variations of the invention may be utilized in any combination or subcombination.

The present invention describes encoding/decoding an image but extending to encoding/decoding a sequence of images (video), as the following illustrates the sequential encoding/decoding of the images of the sequence. code.

1 is a block diagram showing the steps of a method of encoding an image into a bit stream in block diagrams in accordance with an embodiment of the present invention.

In step 1000, a module PRP determines an illumination map IM from the image I to be encoded.

An illumination map collects illumination data associated with pixels of the image to be encoded I. For example, the illumination map may include three sets of illumination values for each pixel of the image, and each of the three sets of values is used for an illumination value. A color component value of a pixel. The illumination pattern is described below as a low frequency version of the luminance component of a backlit image or image to be encoded, although the invention is not limited to any particular illumination value representation associated with the image to be encoded.

In step 1100, an encoder ENC1 encodes the illumination map IM into a one-bit stream F.

In step 1200, a module SM encodes a signalized data SD into a bitstream F, the signalized data indicating that the bitstream F includes an illumination map IM.

According to an embodiment, the signalized data SD further includes a parameter related to the illumination map structure, and the parameter correlation is to be applied to the decoded illumination map for reconstructing the image I.

According to an embodiment, the illumination pattern structure parameters include at least one image spatial resolution and an image sample bit depth.

According to an embodiment of the method, in step 1300, an encoder ENC2 encodes a residual image RI from the image I and the illumination map IM into the bit stream F.

Then, according to the embodiment, the signalized data SD is also adapted, for example, to obtain a decoded image, and the residual image is synchronized with the illumination image.

The bit stream F includes a signalized data SD, an illumination map IM, and a residual image RI (according to an embodiment), the bit stream F can be stored in a region memory or a remote memory, and/or through a communication interface Transfer (such as to a bus or through a communication network or a broadcast network).

2 is a block diagram showing a method of decoding a bit stream F according to an embodiment of the present invention. The bit stream represents an image, and the bit stream F can be obtained by an image encoding method according to FIG.

In step 2000, a module SMD detects whether a signalized data SD indicator bit stream F in the bit stream F includes a lighting pattern for determining the image to be decoded from the data correlation.

In this case, at step 2100, a decoder DEC1 obtains an illumination map DIM by at least partially decoding the bitstream F.

Potentially, several parameters are also obtained by decoding the bit stream F.

According to an embodiment, the parameters are derived from an SEI message in the bitstream F.

According to a variation of this embodiment, the signalling data is detected from higher order syntax elements and its usage is accomplished by an SEI message.

In step 2200, a decoder DEC2 obtains a decoded residual image DRI from a memory, or according to an embodiment, by at least partially decoding the bitstream F.

In step 2300, a module POP obtains a decoded image from the decoded residual image DRI and the decoded illumination image DIM. .

According to an embodiment of step 1100, the encoder ENC1 is configured to encode the illumination pattern IM as an auxiliary image, the syntax of which is defined by standard H264/AVC or HEVC (B.Bross, WjHan, GJSullivan, JR) Ohm, T. Wiegand's proposed JCTVC-K1003, "High Efficiency Video Coding (HEVC) text specification draft 9", October 2012), and decoder DEC1 configuration To obtain (step 2100) an illumination map DIM from an auxiliary image, the syntax of the auxiliary image is defined by standard H264/AVC or HEVC.

The bit stream F then includes an auxiliary image whose syntax conforms to the standard H264/AVC or HEVC, and the auxiliary image represents the illumination map IM.

Implementing an auxiliary image can be done by specifying (several) new video coding layer (VCL) NAL unit types, as in H.264/AVC (see ITU-T Rec. H.264, Series H: Audiovisual and Multimedia) System, infrastructure for audiovisual services - coding of video animation, March 2009). In H.264/AVC, an auxiliary image corresponds to NAL unit type 19, as shown in the following table. The syntax and decoding process of an auxiliary image is identical to the syntax and decoding process of a primary (non-assisted) encoded image. In other words, the decoding of the auxiliary image related material uses the same decoding syntax and engine as the decoding of the primary encoded image.

Alternatively, an auxiliary coded picture can be implemented as a specific layer in scalable coding, which is recommended for HEVC, as described in the document JCTVC-O0041: REXT/MV-HEVC/SHVC HLS: auxiliary image Layers, and provisions in the recent specification of HEVC scalable extensions, document JCTVC-O1008, High Efficiency Video Coding (HEVC) scalable extension draft 4, November 2013. In this case, signal is represented in the higher-order syntax of the bitstream (referred to in the Video Parameter Set - VPS), and an enhanced scalable layer is added to include the primary encoded image (corresponding in the present invention) Into the base layer of the residual image). As shown in the following table, using a syntax element scalability_mask_flag (corresponding to the "scalable mask index"), the scalable version of the scalable layer indicates that it corresponds to the auxiliary code. image.

In addition, a parameter "auxiliary Id" (indicating the pattern of the auxiliary image) is derived from the scalability Id parameter, which itself is derived from a syntax element size _id, as described in Section F.7.4.3.1.1: The video parameter set of the document JCTVC-O1008 is extended in the semantics.

According to an embodiment of step 1300, the encoder ENC2 is configured to encode the residual image RI as a primary image, the syntax of which conforms to the H264/AVC or HEVC standard, and the decoder DEC2 is configured to be used from a primary image. A decoded residual image is obtained (step 2200) whose grammar is defined by standard H264/AVC or HEVC. The primary coded picture and the auxiliary coded picture typically use the syntax and decoding process specified in the AVC or HEVC specifications. There is no specific decoding process for each type of image, and the main difference is related to higher-order syntax (such as NAL unit type or scalability_mask_flag, as explained above).

In this configuration, according to an embodiment, a specific value of the parameter assistance Id can be defined to indicate the nature of the auxiliary image, and in particular a specific value can be defined for the illumination map, as shown in the following table. As an example, the auxiliary Id equals 1 to an alpha channel, the auxiliary Id equals 2 to a depth map, and the auxiliary Id equals 3 to an illumination map.

In another embodiment, the value of the auxiliary Id for the illumination map is the same as for the alpha passer. Indeed, the use of the alpha channel is similar to the use of a backlight channel, since it multiplies the main input image by an alpha map and scales it, using a final clipping operation to ensure that the signal remains at the minimum and maximum signal limits. within. Therefore, the alpha channel concept can be easily adapted for use in backlighting, and the differences can come from different scaling values and clip values (signaled in the accompanying SEI message).

The bit stream F also includes a primary image whose syntax conforms to the H264/AVC or HEVC standard, and the primary image represents the residual image RI.

According to an embodiment of step 1200, the signalized material SD is carried by an SEI message indicating that the JCTVC-O 1008 is conformed according to a syntax. Then, in step 2000, the signalized data is detected from the message SEI according to the embodiment. SD.

The SEI message can be used to carry parameters and/or indications for obtaining a decoded image from a decoded version of the auxiliary image and a decoded version of the primary image (decoded residual image).

As an example, the grammar pair signal, a SEI message (sei_payload) is based on the syntax provided by JCTVC-O0041/JCTVC-F0031:

For example, the parameters and/or indications used to obtain a decoded image can be any of the following: ̇ Color format 位 The bit depth of the sample (which can be used for different color components) ̇ Image size ̇ Image sample Topology (such as regular sample topology or quincence sample topology) ̇ scaling factor cst _calibration ̇ gamma-Slog curve parameter γ ̇ a backlight image parameter ( a _i , ψ _i ), Where ψ _i corresponds to a shape function, for example, and a _i corresponds to a weighting parameter, each associated with the ψ _i function 得到 to obtain a reconstructed mode of the decoded image 最小 reconstructed (HDR) image minimum and maximum clipping values.

Using the HEVC standard, for example, the following table provides the syntax for the parameters associated with these parameters in the SEI message:

In Table 2, the parameter hdr_illumination_image_color_format specifies the color format of the illumination map, 4:2:0, 4:2:2 or 4:4:4 color format, or the array, or the array A single sample in which an image is synthesized in a monochromatic format.

In Table 2, the parameter hdr_illumination_image_bit_depth_minus 8 specifies the bit depth hdr_illumination_image_bit_depth=8+hdr_illumination_image_bit of the illumination map _depth_minus 8.

According to an embodiment, one bit depth is signaled for the brightness component and one for the chrominance component.

In Table 2, the parameter hdr_illumination_image_width specifies the horizontal size of the illumination map, the parameter hdr_illumination_image_height specifies the vertical size of the illumination map, and the parameter hdr_illumination_image_calibration_type A certain standard processing of the illumination map is specified to obtain a full resolution image.

In Table 3, the parameter hdr_shape-function_size_x defines a shape function ( ψ _i ) for determining the width of a backlight image, and the parameter hdr_shape_function_size_y defines a shape function for determining one The height of the backlit image, the parameter hdr_shape_function [cy][cx] provides the value of the specified filter coefficient at the position (cy, cx), and the parameter hdr_ldr_gamma_slog parameter specifies the parameter for an inverse tone mapping. And the parameter hdr_ldr_scaling_factor specifies that the residual image is processed to obtain a full-resolution image. This preset value is equal to 120.

According to an embodiment, the width and height of the shape function depend on: its position in the image, the value of the auxiliary image to which it is applied, the size of the block to which it is applied, and the color component to which it is applied.

For example, a block function can be defined for each block size.

In another example, a shape function can be defined for the brightness component and a shape function for the chrominance component.

In another example, each brightness value range may define a shape function, for example, for brightness between 0 and 127, applying a first shape function with a large width and height. For a brightness between 128 and 255, a second shape function is applied with a smaller width and height to limit the spread of large values to the neighbors.

These parameters are discussed in detail below.

It may be noted that the parameters specified in the SEI message maintain their validity until a new SEI message is received, in which case the new parameter value will be overwritten.

Figure 3 shows the sub-step of step 1000 in a block diagram in accordance with an embodiment of the present invention.

In step 1010, a module IC obtains a luminance component L of the image to be encoded I and potentially at least one color component C(i).

For example, when the image I belongs to the color space (X, Y, Z), a luminance component L such as L = f(Y) is obtained by a transformation f(.) of the component Y.

When the image I belongs to the color space (R, G, B), the luminance component L is obtained by a linear combination provided by the following formula, for example, in the 709 color gamut: L = 0.2127.R + 0.7152.G + 0.0722.B

In step 1020, a module BAM determines a backlight image Bal from the luminance component L of the image I.

According to this embodiment of step 1000, the backlight image Bal is an illumination map IM.

According to an embodiment of step 1020, as shown in FIG. 4, a module BI determines a backed image Ba as a weighted linear combination of the shape function ψ _i , which is provided by the following formula: Ba = Σ _i a _i ψ _i (1 ) a _i is a weighting factor.

Therefore, determining a backlight image Ba from a luminance component L is to find an optimal weighting coefficient (if not known beforehand, and potentially also to find an optimal shape function), so that the backlight image Ba is suitable for the luminance component L.

Many well-known methods can be used to find the weighting factor a _i . For example, the least squares method can be used to minimize the mean square error between the backlight image Ba and the luminance component L.

The invention is not limited to any particular method for obtaining a backlight image Ba.

It may be noted that the shape function may be a display of the actual physical response of the backlight (eg, made of LEDs, each shape function corresponding to an LED response), or may be purely mathematically configured to best fit the luminance component.

According to this embodiment, the backlight image Bal (the output from step 1020) is the backlight image Ba provided by the formula (1).

According to one embodiment of step 1020, depicted in FIG. 5, an _{average value of} a luminance average value L BM module by a module using the HL image I obtained, the backlight image Ba (from equation (1) Provide) to make modulation.

According to this embodiment, the backlight image Bal (the output from step 1020) is a modulated backlight image.

According to an embodiment, the HL coefficient module configured to calculate _{an average value} in the luminance average value L L of the entire luminance component.

According to an embodiment, the module HL is configured to calculate a mean _{value of the} brightness average L by the following formula: The β system is a coefficient smaller than 1, and the mathematical expectation value (average value) of the E (X) system luminance component L.

This last embodiment is advantageous in that it avoids the fact that a small number of pixels with extremely high values affect the average _{value of the} luminance mean L. When the image I belongs to a sequence of images, these extremely high-value pixels usually cause a harmonic disturbance. The average brightness of people's time is unstable.

The invention is not limited to the particular embodiment of calculating the average _value of the luminance mean L.

According to a variation of this embodiment, as shown in FIG. 6, a module N normalizes the backlight image Ba (provided by the formula (1)) from its average value E(Ba) to obtain a medium gray scale. In a (mid-gray-at-one) backlight image Ba _grayscale for the image (or for all images, if the image I belongs to a sequence of images):

Next, the module BM is configured to modulate the backlight image Ba _{gray scale of the} middle gray scale by using the low spatial frequency version L _lf of the image brightness component L by using the following relationship: The cst _modulation system is a modulation coefficient, and the other modulation coefficient of the α system is less than 1, usually 1/3.

According to this variation, Bal backlight image (output of step 1020) based backlight modulation Ba _modulated image after transformation, by the formula (2) is provided.

It can be noted that the modulation coefficient cst _{modulation is} adjusted to obtain a good viewing brightness for the residual image, and is highly dependent on the processing of obtaining the backlight image, for example, cst _modulation 1.7 A backlight image obtained for the least mean square.

In fact, by linearity, all operations for modulating the backlight image are applied to the backlight coefficient a _i as a correction factor that transforms the coefficient a _i into a new coefficient. In order to obtain:

According to this embodiment of step 1000, in step 1100, the data required to determine the backlight image Bal (the output from step 1020) is encoded by the encoder ENC1 and applied to the bit stream F.

According to an embodiment, as described above, such data is embedded in an SEI message.

For example, when using a conventional non-adaptive shape function, the data to be encoded is limited to the weighting factor a _i or However, the shape function ψ _i may also be a priori unknown and then encoded into the bit stream F, for example in a slightly better mathematical construction for better application. Therefore, all weighting coefficients a _i or (and potentially the shape function ψ _i ) are encoded into the bit stream F.

Advantageously, to reduce the size of the bit stream F, the weighting factor a _i or Quantify.

In step 1030, the image is divided by a decoded version of the backlit image. , a residual image Res is calculated.

It is advantageous to use a decoded version of the backlit image to ensure that both the encoder and the decoder are identical backlight images, thereby achieving a better accurate final decoded image. .

More precisely, the luminance component L of the image I and the potential color component C(i) (obtained from the module IC) are divided by the decoded version of the backlight image. This division is done one by one per pixel.

For example, when the component R, G or B of the image I is represented in the color space (R, G, B), the components R _Res , G _Res and B _Res are obtained as follows: For example, when the component X, Y or Z of the image I is represented in the color space (X, Y, Z), the components X _Res , Y _Res and Z _Res are obtained as follows:

According to a change of step 1030, the decoded version of the backlight image is processed before the residual image Res is obtained. .

Applied to the decoded version of the backlit image The processing may be used, for example, to generate a processed backlight image having the same resolution as its corresponding residual image. Below, the decoded version of the backlit image , the term will be used indiscriminately to represent a decoded version of a backlit image processed or unprocessed .

According to an embodiment, the decoded version of the backlit image is used from the signaled parameter in the signaled material SD. Decoded version with backlight image processing .

According to an embodiment, in step 2100, the bit stream F is at least partially decoded by the decoder DEC1 to obtain a decoded version of the backlight image. .

As described, some of the material needed to obtain a backlit image (the output of step 1020) is encoded (step 1100), and then the data is obtained by at least partial decoding of the bitstream F.

After the examples provided above, the weighting coefficients are then obtained. (and potentially shape function As an output of step 2100.

Next, in step 1070, a module BAG is: Weighting coefficient And some conventional non-adaptive shape functions or shape functions Produce a decoded version of the backlit image .

In step 1040, a module TMO performs tone mapping on the residual image Res to obtain a visible residual image Res _v .

Since the dynamic range of the residual image Res is too high and since the residual image Res shows excessive visible artifacts, it may appear that the residual image Res is not visible. Performing tone mapping on the residual image remedies at least one of these disadvantages.

The invention is not limited to any particular tone mapping operator, and this single condition is that the tone mapping operator should be reversible.

For example, a tone mapping operator defined by Reinhard (Photographing tone reproduction for digital images , published by E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda et al. ) can be used . ACM will report 21 (July 2002), or R. Boitard, K. Bouatouch, R. Cozot, D. Thoreau and A. Gruson et al. (2012), temporal consistency of video tone mapping. In SPIE Protocol 8499, edited by AMJvan Eijk, CCDavis, SMHammel, and AK Maajumdar, Applications of Digital Im ag e Processing (pp. 84990D to 84990D-10).

According to this embodiment of step 1000, in step 1300, the encoder ENC2 system configured to image the visible residue Res _v F is encoded into the symbol stream.

According to an embodiment of step 1040, performing tone mapping on the residual image includes a gamma correction or a SLog correction according to the pixel value of the residual image.

The visible residual image Res _{v is} then provided, for example by the following formula: Res _v = A. Res ^γ A is a constant value, the coefficient of the γ -gamma curve, for example equal to 1/2.4.

Alternatively, for example, the visible residual image Res _v is provided by the following formula: Res _v = a. ln( Res + b )+ c a,b,c is the coefficient of the determined SLog curve so that 0 and 1 are unchanged, and The derivative of the SLog curve lasts at 1 when extended by a gamma curve below 1 and, therefore, a, b, c is a function of the parameter γ .

According to an embodiment, the parameter γ of the gamma-Slog curve is encoded into the bit stream F.

Applying a gamma correction to the residual image Res pulls the dark areas up but does not lower the high light to avoid burning the bright pixels.

A SLog correction is applied to the residual image Res to reduce the high light but not to pull up the dark area.

Next, according to an embodiment of step 1040, the module TMO applies gamma correction or SLog correction according to the pixel value of the residual image Res.

For example, when the pixel value of the residual image Res is below a critical value (equal to 1), gamma correction is applied, otherwise SLog correction is applied.

By construction, depending on the brightness of the image I, the image residue visible Res _v typically has a mean value close to 1 or less, so that the combination of the use of gamma -Slog particularly efficient.

According to an embodiment of the method, in step 1050, a module will be seen by each component SCA afterimage Res _v multiplied by a scaling factor of _scaling cst, the residual image will be visible before encoding Res _v (step 1300) Calibration is performed first. The resulting residual image Res _s is provided by Res _s = cst _scaling by the following formula. Res _v

In a particular embodiment, the scaling factor cst _{is scaled to} define a value of the visible residual image Res _v between 0 and a maximum value of 2 ^N -1, where N is the number of available bits as an input Encoding for encoder ENC2.

This result is naturally obtained by mapping the value 1 (which is the average of the coarse visible residual image Res _v ) to the medium gray scale value 2 ^N-1 , and thus is used for a visible residual image having the standard number of bits N=8. Res _v , the certain scaling factor is equal to the 120-series one-pole coherence value, which is very close to the intermediate gray scale at 2 ⁷ =128.

According to an embodiment of the method, in step 1300, the encoder ENC2 is configured to encode the residual image Res _s .

According to an embodiment of the method, in step 1060, a visible residue CLI module Res _v for editing image before encoding, to limit the dynamic range to a target dynamic range TDR, based for example according to the capabilities of the encoder ENC2 Defined.

According to this last embodiment, the resulting residual image Res _c is according to an embodiment of the method, for example by the following formula: Res _c = max(2 ^N , Res _v ) Res _c = max(2 ^N , Res _s )

The invention is not limited to such clips (max(.)), but extends to other types of clips.

According to this embodiment of the method, in step 1300, the encoder ENC2 is configured to encode the residual image Res _c .

According to an embodiment of the method, in conjunction with the scaling and clipping embodiments results in a residual image Res _sc , provided by the following formula: Res _sc = max (2 ^N , cst _scaling * Res _v ) or by Res _sc = max (2 ^N , cst _calibration *Res _s )

According to this embodiment of the method, in step 1300, the encoder ENC2 is configured to encode the residual image Res _sc .

The tone mapping and scaling of the residual image Res _v can be seen as a parameterization process, which can be fixed or not fixed. In the latter case, the parameters can be encoded into the bit stream F by the encoder ENC1.

Example gamma] constant value, a gamma correction according to the embodiment of the method, _calibration scaling factor may be based cst parameters which F-based encoded bitstream in.

It may be noted that the selection of the parameters α , cst _modulation , cst _scaling , γ , β provides a space for selecting a tone map that best fits the content to follow the taste of the expert in post-production and color grading.

On the other hand, the definable general parameters are acceptable for all kinds of images, and no parameters are encoded in the bit stream F.

According to an embodiment, as described above, at least one parameter α, cst _scaling, γ, β system embedded in a SEI message.

According to this embodiment of step 1000, the residual image RI is visible residual image Res _v or Res _s or Res _c .

Figure 7 shows the sub-step of step 2300 in a block diagram in accordance with an embodiment of the present invention.

As described above, in steps 2100 and 1070, the bit stream F is at least partially decoded by the decoder DEC1 to obtain a backlit image. (A decoded lighting diagram).

The bit stream F can be stored or received locally from a communication network.

In step 2200, the bit stream F is at least partially decoded by a decoder DEC2 to obtain a decoded residual image. .

As described above, the decoded residual image can be seen by a conventional device. .

In step 2340, by decoding the residual image Multiply by backlit image , get a decoded image .

According to a variation of this embodiment, a decoded image is obtained Before processing the backlit image first .

Apply to backlit image Processing may be used, for example, to generate a processed backlight image, corresponding to the decoded residual image Have the same resolution. Following, backlit image , the word will represent the processed or unprocessed backlit image indiscriminately .

According to an embodiment, the signal represented by the signalized data SD is used to image from the backlight Obtaining the processed backlight image .

According to an embodiment of step 2100, the bit stream BF is also at least partially decoded from a region memory or by the decoder DEC1 to obtain a parameter. And/or .

According to the method, in step 2310, a module ISCA uses the residual image to be decoded. Divide by parameter , applying a reverse scaling to the decoded residual image .

At step 2320, a module ITMO is parameterized by , applying an inverse tone mapping to the decoded residual image .

For example, parameters Defining a gamma curve, and the inverse tone mapping system is only used to find the decoded residual image from the gamma curve The value corresponding to the pixel value.

Figure 8 shows the sub-step of step 1000 in a block diagram in accordance with an embodiment of the present invention.

According to this embodiment, the image I to be encoded is divided into a plurality of image blocks B, and each image block B is considered as follows.

In step 1080, a module IC obtains components of the image block B to be encoded, and the image block B includes a luminance component L and potentially at least one color component C ( i ), i is an index, and the image area is identified. A color component of block B. The component of image block B belongs to a perceptual space, usually a 3D (stereo) space, that is, image block B includes a luminance component L and potentially at least one color component C ( i ), such as hereinafter referred to as C1 and C2. Two color components.

However, the invention is not limited to a grayscale image (no color component) nor to an image having one, two or more color components. When the following description encodes a grayscale image, the description portion of the color component is not considered.

A perceptual space has a metric d (( L, C 1 , C 2) , ( L ' , C 1 ' , C 2 ' )) whose value is a representation of the difference between the visual perceptions of two points in the perceptual space ( It is preferably proportional.

In mathematics, the metric d (( L, C 1 , C 2) , ( L ', C 1 ', C 2 ' )) is defined such that there is a perceptual threshold Δ E ₀ (also known as JND, just The difference can be resolved), below which the human can not perceive the visual difference between the two colors of the perceptual space, ie d (( L, C 1 , C 2) , ( L ', C 1 ', C 2 ' )) < △ E ₀ , (3) and this perceptual threshold are independent of the two points ( L, C 1 , C 2) and ( L ', C 1 ', C 2 ' ) of the perceptual space.

Thus, an image (whose component belongs to a perceptual space) is encoded such that the metric d of equation (3) remains below the perceptual threshold Δ E ₀ , ensuring that the display decoded version of the image is visually lossless.

According to an embodiment, the metric is calculated on a pixel basis.

It may be noted that it is easier to control the following three inequalities individually:

It can be noted that if a perceptual threshold greater than Δ E ₀ satisfies equation (3), that is, the encoded image is visually controlled below, ie, the visual loss of the image in a displayed decoded version is controlled.

For example, when the image I includes several components belonging to an unconscious space such as (R, G, B), a perceptual transformation is applied to the image I to obtain a luminance component L and a potential second color component C1 belonging to the perceptual space. And C2.

This perceptual transformation is defined by the lighting conditions of the display and depends on the initial color space.

For example, assuming the initial space system (R, G, B) color space, first transform the image I into a well-known linear space (X, Y, Z) (which potentially requires an inverse gamma correction), Visuals transform result of reference lighting conditions for self-encoded image a decoded version of the display, which in this system the (X, Y, Z) space of several values _{(X n, Y n, Z} n) is a 3D vector.

Thus, for example, when the perceptual space LabCIE 1976 is selected, this perceptual transformation is defined as follows: L *=116 f ( Y / Y _n )-16 a *=500( f ( X / X _n )- f ( Y / Y _n ) b *=200( f ( Y / Y _n )- f ( Z / Z _n )) where f is a transfer function, for example provided by the following formula: if r >(6/29) ³ then f ( r )= r ^1/3 otherwise

The following metrics can be defined in the perceptual space LabCIE 1976: d (( L *, a *, b *) , ( L * ' , a * ', b * ' ) ² = (Δ L *) ² + (△ a *) ² +(△ b *) ² <(Δ E ₀ ) ² Δ L * is between the two colors ( L * , a * , b *) and the luminance components of ( L * ', a * ' , b * ' ) The difference, and Δ a *(Δ b * respectively) are the differences between the color components of the two colors.

According to another example, when the selected appearance space Lu * v *, a perceptual transform is defined as follows: u * = 13 L (u '- u' _white) and v * = 13 L (v ' - v' _white) wherein and

The following Euclidean metrics can be defined in the perceptual space Lu * v *: d (( L *, u *, v *), ( L * ' , u * ' , v * ' ) ² = (△ L ) ² +(Δ u *) ² +(Δ v *) ² Δ L * is the luminance component of two colors ( L *, u * , v *) and ( L * ' , u * ' , v * ' ) The difference between, and Δ u *(Δ v * respectively) is the difference between the color components of the two colors.

The invention is not limited to the perceptual space LabCIE 1976, but can be extended to any type of perceptual space such as LabCIE 1994, LabCIE 2000, which is the same Lab space but with different metrics for measuring the perceptual distance, or for example any other Euclid perceptual space. . Other examples are LMS space and IPT space, provided that the metric is defined in terms of such perceptual spaces, in order for the metric to be better proportional to the perceived difference; as a result, there is a homogenous maximum perceived threshold Δ E ₀ , lower than At this critical value, humans cannot perceive the visual difference between the two colors of the perceptual space.

In step 1090, a module LF obtains a low spatial frequency version L _lf of the luminance component L of image I.

According to this embodiment of step 1000, the low spatial frequency version L _lf of the luminance component L of image I is the illumination map IM.

According to an embodiment, the module LF is configured to assign an average value calculated by averaging the pixel values of the block to each pixel of a block for calculating a low spatial frequency version L of each block. _Lf .

The invention is not limited to a particular embodiment of calculating a low spatial frequency version of image I, and any low pass filtering or downsampling of the luminance component of image I can be used.

In step 1100, encoder ENC1 is configured to encode the low spatial frequency version L _lf into bit stream F.

In step 1110, a differential image Diff obtained, comprising a differential image Diff _{R & lt} differential luminance component L, which is a decoded version of the system by calculating a luminance component L and a low spatial frequency encoded version of L _LF The difference between the two is obtained.

Potentially, at step 1120, a configuration module for each color component based ASSO for causing the image I L _r and differential luminance component is associated, is to obtain a differential image Diff. According to this example, the image I includes two color components C1 and C2, and then the color components C1 and C2 are associated with the differential luminance component L _r to obtain a differential image including three components ( L _r , C 1 , C 2). Diff .

In step 1300, the encoder ENC2 is configured to encode the differential image Diff into the bitstream F.

Potentially, in steps 1100 and/or 1300, encoder ENC1 and/or ENC2 includes an entropy coding.

The coding accuracy of the encoder ENC2 depends on a perceptual threshold Δ E in the perceptual space, which defines an upper limit of the metric in the perceptual space and enables visual loss control in a display decoded version of the image.

In step 1130, the reference illumination condition and the decoded version of the low spatial frequency version L _lf are displayed according to a decoded version of the encoded image. Determine the threshold value of △ E .

The brightness of the low spatial frequency version L _lf does not remain unchanged in the image but varies regionally, for example, if an average value calculated by averaging pixel values of a block is assigned to each of the blocks The pixel calculates the low spatial frequency version L _{lf of} each block, and the perceptual threshold Δ E remains unchanged among the blocks, but the average value of the two blocks of the image may be different. Therefore, the perceptual threshold value Δ E varies regionally based on the luminance value of the video.

The region variation of the perceptual threshold Δ E is not limited to block-based variations, but may extend to any region defined by the image, for example, based on the luminance value of the image, by any operator such as a segment operator. .

In step 2100, the output from step 1100 is decoded by a decoder DEC1 (ie, the bitstream F is at least partially decoded) to obtain a decoded version of the low spatial frequency version L _lf . . This decoder DEC1 performs a reverse operation compared to the operation of the encoder ENC1 (step 1100).

According to an embodiment of step 1130, it is assumed that illumination is potentially increased all the way to a maximum ambient luminance value Y _n during the image display, from the decoded version of the low spatial frequency version L _lf Brightness value The perceived threshold value Δ E is determined as a percentage of the maximum ambient luminance value Y _n .

According to an embodiment of step 1130, when the maximum allowed coding environment degradation in luminance value, by the following equation perceptual threshold △ E: ( X _n , Y _n , Z _n ) is a reference illumination condition displayed by a decoded version of the encoded image, and Is a value indicating the decoded version of the low spatial frequency version L _lf The brightness, and △ E _enc is a perceptual coding parameter. Δ E _enc is typically selected to be close to Δ E ₀ for visual lossless encoding, and greater than Δ E ₀ for a code to have visual loss control in the decoded version of the encoded image.

Thus, using this threshold value △ E perceptual coding adapted to permit the display of the ambient lighting conditions.

Alternatively, the reference lighting conditions _{(X n, Y n, Z} n) a decoded version of the encoded video display, having a local character, with reference to the overall lighting conditions may be substituted to display a decoded version of the coded image by the Defined.

From a coding point of view (color coding), this substitution is equivalent to the selection of the perceptual threshold Δ E (4) because, in the color space LabCIE 1976, Δ E having a color component a * equal to an accuracy is encoded. It is provided by the following formula: Is equivalent to the encoding with a precision equal to the color component a * ' , Δ E _enc , which is provided by the following formula:

The same remark applies to the other component b *, so that the perceptual space is not changed regionally, as long as the threshold is adjusted from Δ E _enc to Δ E .

According to an embodiment of step 1130, in order to avoid sub-coding of the image having a high luminance value portion, the perceptual threshold value Δ E is provided by the following formula: One of the upper limits is set to Δ E _enc E _max , and usually E _max is set to 1. Finally, this formula means that the decoded version of the low spatial frequency version L _lf is never used. The brightness is greater than the maximum ambient brightness value Y _n .

On the other hand, in order to avoid over-encoding of the portion of the image having a very low luminance value, the perceptual threshold Δ E is provided by the following formula: One of the lower limits is set to Δ E _enc E _min ; usually E _min is set to about 1/5. This is due to the decoded version of the low spatial frequency version L _lf The dark area brightness is caused by the contrast mask caused by the maximum ambient brightness value Y _n .

The combination of the two limits is simply derived by the following formula:

According to a variation of the method, at step 1140, a threshold TH is applied to the (equal) component of the differential image Diff to limit each dynamic range of its component to a target dynamic range TDR.

According to an embodiment of step 1300, the components of a differential image are normalized by a perceptual threshold Δ E , and then the normalized differential image is encoded with constant encoding accuracy.

Therefore, the coding accuracy is therefore a function of the perceptual threshold Δ E , which varies regionally and is optimally accurate (assumed to be an ideal perceptual space). By doing so, the normalized differential image coding accuracy to 1, the differential video encoding to ensure the accuracy of △ E as desired.

According to an embodiment of step 1300, a component of a differential image is divided by a normalization of the threshold Δ E by a value that is a function of the threshold Δ E .

In mathematics, for example, a component C of the differential image (including both the differential luminance component and potentially the respective color components) is normalized as follows to obtain a normalized component C ^N : The alpha system has a value equal to, for example, 0.5 or 1.

According to another embodiment of step 1300, at least one parameter of a differential image encoding is dependent on a perceptual threshold Δ E .

For example, a quantization parameter QP of this code depends on the perceptual threshold value Δ E . In fact, this parameter QP exists in the image/video encoder images h264/AVC and HEVC, and can be defined regionally for each coding. Block. In this example, by selecting a coding region QP performed throughout the differential image with the accuracy of a regional (a block to a block) change, to ensure perceptual threshold △ E precision used to encode each block degree.

Figure 9 shows the sub-step of step 2300 in a block diagram in accordance with an embodiment of the present invention.

It may be noted that the following considers a one-bit stream F representing an image I comprising a luminance component and potentially including at least one color component. The (equal) component of image I belongs to the perceptual color space described above.

In step 2100, as described above, the bit stream F is at least partially decoded by a decoder DEC1 to obtain a low spatial frequency version decoded version of the luminance component of the image I. .

In step 2200, the bitstream F is at least partially decoded by the decoder DEC2 to obtain a decoded version of the differential image. .

Therefore, when the image I represented by the bit stream F is a gray scale image, a decoded version of the differential image A differential brightness component L _{r is included} , which represents a low spatial frequency version decoded version of a luminance component L of image I and a luminance component of image I difference between. When the image I represented by the bit stream F is a color image, that is, an image having a luminance component L and at least one color component, a decoded version of the differential image The differential brightness component L _r and each (the at least one) color component of the image I, the differential brightness component representing a low spatial frequency version decoding version of a luminance component L of the image I and a luminance component of the image I difference between.

In step 2350, a decoded version of the differential image is Low spatial frequency version decoded version with the luminance component of the image Add up to get the decoded image .

The decoding accuracy of the decoder DEC2 depends on a perceptual threshold Δ E which defines a metric upper limit in the perceptual space and enables visual loss control in a display decoded version of the image. Therefore, the decoding accuracy is a function of the perceptual threshold, which varies regionally.

In the above description of the related step 1130, according to an embodiment, the determination threshold threshold Δ E is a reference illumination condition (same as for the coder) and a low spatial frequency of the luminance component of the image I according to a decoded version of the encoded image. Version decoding version .

According to an embodiment of step 2200, when the components have a differential image by normalized perceptual threshold △ E, the differential image based decoding version for each component decoder, and the differential image by a constant accuracy Diff Renormalization is performed by the perceptual threshold Δ E .

According to one embodiment of step 2200, and then multiplied by a normalization based value based perceptual threshold a function of △ E.

In mathematics, for example, the components of the decoded version of the differential image Renormalize as follows: The alpha system has a value equal to, for example, 0.5 or 1.

According to a variation, in step 2360, a module IIC is configured to apply an anti-perceptual transformation to the decoded image. (output from step 2350), for example, will decode the image The estimate is transformed into the well-known space (X, Y, Z).

When the perceptual space LabCIE 1976 is selected, the anti-perceptual transformation is provided by the following formula: Y = Y _n f ^-1 (1/116( L *+16))

When the perceptual space Luv is selected, the anti-perceptual transformation is provided by the following formula:

Potentially, the inverse of the image in space (X, Y, Z) is used to obtain decoded images in the initial space such as (R, G, B) space.

Potentially, during step 2100 and/or 2200, the data of the bitstream F is also at least partially entropy decoded.

The decoder DEC1 (DEC2, respectively) is configured to decode the data that has been encoded by the encoder ENC1 (ENC2 respectively).

The encoders ENC1 and/or ENC2 (and decoders DEC1 and/or DEC2) are not limited to a particular encoder (decoder), but when an entropy coder (decoder) is required, an entropy coder such as Hoffman A (Huffmann) encoder, an arithmetic coder, or a context adaptive encoder is advantageous like the Cabac system used in h264/AVC or HEVC.

Encoders ENC1 and ENC2 (and decoders DEC1 and DEC2) are not limited to a particular encoder, which may, for example, utilize an image/video encoder that loses JPEG, JPEG2000, MPEG2, h264/AVC or HEVC.

In Figures 1 to 9, the modules are functional units that may or may not be related to distinguishable physical units, for example, such modules or some of the modules may be co-located in a single part or circuit, or A software functionality contributes. Conversely, some modules may potentially consist of several separate entities, and devices suitable for the present invention are implemented using pure hardware, such as using proprietary hardware such as ASIC or FPGA or VLSI (application-specific integration, respectively) Circuit », «Site The programmable gate array», «maximum integrated circuit»), can be realized by several integrated electronic parts embedded in a device, or by mixing hardware and software parts.

10 illustrates an exemplary architecture of an apparatus 1000 that is configurable to perform the methods described in relation to FIGS. 1-9.

The device 1000 includes the following elements linked by a data and address bus 1001: a microprocessor 1002 (or CPU), for example, a DSP (or digital signal processor); - a ROM (or only Read memory) 1003; - a RAM (or random access memory) 1004; - an I / O (input / output) interface 1005 for receiving data from an application for transmission; and - a battery 1006 .

According to a variation, the battery 1006 is external to the device, and such components of Figure 10 are well known to those skilled in the art and will not be described again. In each of the memories, the term "scratchpad" used in the description of the present invention may correspond to a small capacity area (some bits) or to a maximum area (such as the entire program, or a large amount of received or decoded data). ). The ROM 1003 includes at least one program and a plurality of parameters, and the calculation formula of the method according to the present invention is stored in the ROM 1003. When the switch is turned on, the CPU 1002 uploads the program in the RAM and executes the corresponding instruction.

The RAM 1004 includes a program uploaded by the CPU 1002 and opened by the switch of the device 1000 in a register, and the data is input in a register, and the method is in a different state in a register. Intermediate data in the middle, and other variables used in the execution of the method in a register.

The implementations described herein may be implemented, for example, in a method or a process, a device, a software program, a data stream, or a signal. That is, if it is discussed only in the context of a single-form implementation (for example, only as a method or a device), the implementation of the features discussed may be implemented in other forms (eg, a program). A device can be implemented, for example, in a suitable hardware, software, and firmware. The methods can be implemented, for example, in a device such as a processor, which is generally referred to as a processing device, for example, including a computer, a microprocessor, an integrated circuit, or a programmable Logic device. The processor also includes communication devices such as computers, cell phones, portable or personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

According to an embodiment, the apparatus further includes means for obtaining a reference illumination condition, such as a maximum ambient brightness value Y _{n ,} displayed by a decoded version of the encoded image.

According to an embodiment, the apparatus includes a display 1007, and a reference illumination condition obtained by the component for obtaining a decoded version of the encoded image, the component being configured to be captured from the display 1007 or from the display 1007 by the device In the lighting conditions that are obtained, such reference lighting conditions displayed by a decoded version of the encoded image are determined.

For example, a sensor member for obtaining the maximum system environment brightness Y _n is measured with the monitor environmental conditions, may be used for this purpose, or the like photo-diode.

According to a particular embodiment of the encoding or encoder, the image I is derived from a source, for example, belonging to a collection of: - a regional memory (1003 or 1004), such as video memory or RAM (or random) Access memory), flash memory, ROM (or read-only memory), hard disk; - a storage interface (1005), such as a large storage interface, RAM, flash memory, ROM, CD or magnetic Pad; - a communication interface (1005), such as a wired interface (such as bus interface, WAN interface, regional network interface), or a wireless interface (such as IEEE 802.11 interface or Bluetooth® interface); and - an image capture circuit ( Such as a sensor such as CCD (or charge coupled device) or CMOS (or complementary metal oxide semiconductor).

Decoding the image according to different embodiments of the decoding or decoder Is transmitted to a destination; specifically, the destination belongs to a collection of: - a regional memory (1003 or 1004), such as video memory or RAM, flash memory, hard disk; Storage interface (1005), such as a large storage interface, RAM, flash memory, ROM, CD or magnetic pad; - a communication interface (1005), such as a wired interface (such as bus interface (such as USB (or universal string) Column bus)), WAN interface, regional network interface, HDMI (high-definition multimedia interface) interface, or wireless interface (such as IEEE 802.11 interface, WiFi® or Bluetooth® interface); and - a display.

According to different embodiments of the encoding or encoder, the bitstream F is transmitted to a destination. As an example, the bitstream F is stored in a region or remote memory such as video memory (1004) or RAM (1004). ), hard disk (1003). In a change, the bit stream is transmitted to a Storage interface (1005), such as with a large number of stored interfaces, flash memory, ROM, CD or magnetic pad, or transmitted through a communication interface (1005), such as to point-to-point links, communication bus, single-point to multi-point links Or the interface of the broadcast network.

According to different embodiments of the decoder or decoder, the bit stream BF and/or F is derived from a source, as an example, the bit stream is read from a region memory, such as video memory (1004), RAM. (1004), ROM (1003), flash memory (1003) or hard disk (1003). In one variation, the bit stream is received from a storage interface (1005), such as a mass storage interface, RAM, ROM, flash memory, optical disk or magnetic pad, and/or received from a communication interface (1005) ), such as to a point-to-point link, bus, single-to-multipoint link or broadcast network interface.

According to various embodiments, the apparatus 1000 is configured to implement the encoding method described in relation to Figures 1,3-6 and 8, which is a collection of the following components: - a mobile device; - a communication device; - a gaming device; Set-top box; - tablet (or tablet); - laptop; - still image camera - video camera; - coded chip; - still image server; and - video server (eg broadcast server, video on demand) Server or a web server).

According to various embodiments, the apparatus 1000 is configured to implement the decoding method described in relation to Figures 2, 7 and 9, which is a collection of the following components: - a mobile device; - a communication device; - a gaming device; - an onboard Box;- TV;- tablet (or tablet); - laptop; - display; and - decoder chip.

According to an embodiment illustrated in FIG. 11, in a transmission scenario between two remote devices A and B through a communication network NET, the device A includes a configuration for implementing an image encoding method as described in relation to FIG. The components, and apparatus B, include components configured to implement the decoding method as described in relation to FIG.

According to a variant of the invention, the network is a broadcast network adapted to play still or video images from device A to a plurality of decoding devices comprising device B.

The various processes and features described herein can be implemented in a variety of different devices or applications, such as in a device or application, for example. Examples of such devices include an encoder, a decoder, a post-processor (to process the output of the decoder), a pre-processor (to provide input to the encoder), a video encoder, a video decoder, and a video codec. , web servers, set-top boxes, laptops, personal computers, cell phones, PDAs, and other communication devices. Obviously, the device can be mobile or even installed in a car.

Moreover, the methods can be implemented by instructions executed by a processor, and such instructions (and/or data values generated by an implementation) can be stored in a processor readable medium such as an integrated circuit, software. Carrier, or other storage device such as a hard disk, a compact disc ("CD"), a compact disc (such as a DVD, often referred to as a variety of digital or digital video discs), random access memory ("RAM"), or read-only memory ("ROM"). The instructions may form an application tangibly embodied on a processor readable medium, such as in hardware, firmware, software, or a combination, the instructions being, for example, in an operating system, separate applications Found in the program or a combination of both. Thus, a processor may be configured, for example, as a device for performing a process, and a device comprising a processor readable medium (such as a storage device) having instructions for performing a process. In addition, a processor readable medium can store (adding or replacing instructions in addition to instructions) a data value generated by the implementation.

As will be appreciated by those skilled in the art, several implementations can generate various signals formatted for carrying information, which can be stored or transmitted, for example, which can include, for example, instructions for performing a method, or the implementation. Information produced by one of them. For example, a signal can be formatted for carrying as a material for writing or reading the grammar rules of the embodiment, or The information used to carry as the data is the actual grammatical value written in the embodiment. For example, the signal can be formatted as an electromagnetic wave (e.g., using a radio frequency portion of the spectrum) or as a frequency band signal. Formatting, for example, can include encoding a data stream and utilizing the encoded data stream to modulate a carrier. The information carried by the signal can be, for example, analog or digital information. As is known, the signals can be transmitted over a variety of different wired or wireless links, and the signals can be stored on a processor readable medium.

Several implementations have been described, however, it should be understood that various modifications may be made, such as combining, adding, modifying, or removing elements of different implementations to produce other implementations. In addition, one of ordinary skill in the art will appreciate that other structures or processes may be substituted for those disclosed, and the resulting implementations will be implemented as a plurality of at least substantially the same. The same function is used to achieve (several) at least substantially the same result, and thus, such and other implementations are conceivable from the present invention.

1000‧‧‧Lighting chart determination steps

1100‧‧‧ Lighting pattern coding steps

1200‧‧‧ Signaling data steps

1300‧‧‧Residual image coding steps

ENC1, ENC2‧‧‧ encoder

F‧‧‧ bit flow

I‧‧‧ images

IM‧‧‧Lighting diagram

PRP‧‧‧Lighting Chart Judgment Module

RI‧‧‧ residual image

SD‧‧‧Signalization data

SME‧‧‧ Signaling Data Encoding Module

Claims (19)

A method of encoding an image into a bit stream, comprising the steps of: - encoding (1000) a decision from the image (1100) into the bit stream; and - signaling a data Encoding (1200) into the bitstream, the signalling material indicates that the bitstream includes the illumination map. The method of claim 1, wherein the method further comprises the step of: encoding (1300) a residual image from the image and the illumination image into the bit stream. The method of claim 1 or 2, wherein the illumination pattern is encoded as an auxiliary image, the syntax of which conforms to the H264/AVC or HEVC standard. The method of claim 2, wherein the residual image is encoded as a primary image, the syntax of which conforms to the H264/AVC or HEVC standard. The method of any one of claims 1 to 4, wherein the illumination image is a backlit image, and the residual image is obtained by dividing the image by a decoded version of the backlight image. The method of claim 5, wherein the residual image is subjected to tone mapping before encoding. The method of claim 4, wherein the mapping of the residual image to the tone comprises, based on the pixel value of the residual image, a gamma correction or a SLog (S-logarithm) correction. The method of any one of claims 1 to 7, wherein the method further comprises the calibration of the residual image prior to encoding. The method of any one of claims 1 to 8, wherein the method further comprises editing the residual image prior to encoding. A method of decoding a bit stream, the bit stream representing an image, the method comprising the steps of: - detecting (2000) whether a signalized data in the bit stream indicates that the bit stream includes data correlation Determining an illumination pattern of the image to be decoded; - by at least partially decoding the bit stream, obtaining (2100) a decoded illumination image; and - deriving from a decoded residual image and the decoded illumination image (2300) A decoded image. The method of claim 10, wherein the bit stream is at least partially Decoding to obtain the decoded residual image. The method of claim 10, wherein the signalling data is detected from a higher-order syntax element and its use is performed by an SEI message. The method of any one of claims 10 to 12, wherein the bit stream comprises a primary image and an auxiliary image, the syntax of which conforms to the H264/AVC or HEVC standard, and wherein the primary image represents residual image And the auxiliary image represents the illumination map. The method of any one of claims 10 to 13, wherein the decoded illumination image is a backlit image, and wherein the decoded residual image is multiplied by the backlight image to obtain a decoded image. The method of any one of claims 10 to 14, wherein the decoded residual image is inversely mapped prior to multiplying the decoded residual image by the backlit image. The method of any one of claims 1 to 4, wherein the illumination map is a low spatial frequency version of one of the luminance components of the image to be encoded, and by calculating a luminance component of the image and the encoded low spatial frequency version A residual image is obtained by decoding the difference between the versions. A device for encoding an image, comprising: means for: - encoding a decision (1000) from the illumination pattern of the image (ENC1) into the bit stream; - encoding a signalized data (ENC2) to the bit In the metaflow, the signalling material indicates that the bitstream includes the illumination map. A device for decoding a bit stream, the bit stream representing an image, the method comprising the steps of: - detecting (SMD) whether a signalized data in the bit stream indicates that the bit stream represents a decision Illumination map from the image; - by decoding the bit stream at least partially, to obtain (DEC1) a decoded illumination map; and - deriving (DEC2) a decoded image from a decoded residual image and the decoded illumination image . A bit stream representing an image, the bit stream being characterized by a signalized data indicating that the bit stream represents a lighting pattern determined from the image.