JP2018191269A - Method and device of reconstructing image data from decoded image data - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem generated in the case where a metadata is not matched to an image to which a graphic or an overlay is applied.SOLUTION: A method or a device of reconstructing image data indicating original image data from a parameter acquired by decoded image data and a bit stream includes: a step (102) of checking the parameter; a step (104) of selecting a decoding mode (RMi) in accordance with information data (ID and 103) indicating whether how the parameter is processed in the case where at least one parameter is lost, damaged, or is not matched to the decoded image data to which the graphic or the overlay is applied; and a step (105) of decoding the parameter by adapting reconstruction of the selected decoding mode (RMi) and image data (I3), and considering the decoded parameter (12).SELECTED DRAWING: Figure 1

Description

  The principles of the present application generally relate to the reconstruction of image / video data from decoded image / video (video) data. Specifically, without limitation, the technical field of the present principles relates to restoring parameters for reconstructing an image from another image.

  This section is intended to introduce the reader to various technical aspects. These technical aspects will relate to various aspects of the present principles as described below and / or as set forth in the following claims. We believe that this description will help provide readers with background information to better understand various aspects of the present principles. Accordingly, it should be understood that each description is to be read in light of this point and is not an admission of prior art.

  In the following, image data is used by a display and / or other device to visualize and / or decode all information related to the pixel values of the image (or video) and, for example, the image (or video) Refers to one or several arrays of samples (pixel values) in a particular image / video format that identifies all the information that may be done. The image is usually in the shape of an array of first samples that represents the brightness (or luma) of the image, and the first component and other arrays of samples that typically represent the color (or chroma) of the image. And second and third components in the form. Alternatively, similarly, the same information can be represented by a set of color sample arrays, such as the conventional three-color RGB representation.

  The pixel value is represented by a vector of C values, where C is the number of components. Each value of the vector is represented by the number of bits that define the maximum dynamic range of the pixel value.

  A standard dynamic range image (SDR image) is an image represented by a limited number (usually 8) of bits of luminance values. This limited representation does not allow small signal changes to be accurately rendered, particularly in the dark and bright luminance ranges. In high dynamic range images (HDR images), the signal representation is extended to maintain high signal accuracy throughout the range. In HDR images, pixel values typically represent a luminance level expressed in a floating-point format (at least 10 bits for each component, ie, single-precision floating point (float) or half-precision floating point (half-float)). The value to represent. The most popular format is the openEXR half-precision floating point format (16 bits per RGB component, ie 48 bits per pixel), or an arrangement of “long” type representations, usually at least 16 bits. It is a numerical value.

  The arrival of the High Efficiency Video Coding (HEVC) standard (ITU-T H.265 Telecommunication Standardization Division (10/2014) of ITU, Series H: Audiovisual and Multimedia Systems, Audiovisual Services Infrastructure-Video Coding, High-Efficiency Video Coding, Recommendation ITU-T H.265) enables the deployment of new video services using enhanced viewing experiences such as Ultra HD broadcast services . Compared to the currently deployed standard dynamic range, Ultra HD can improve wide color gamut (WCG) and high dynamic range (HDR) with increasing spatial resolution. Various solutions have been proposed for the representation and coding of HDR / WCG video (SMPTE 2014, "High Dynamic Range Electro-Optical Transfer of Mastering & Reference Display"). Alternatively, SMPTE ST 2084, 2014, or Diaz, R., Blinstein, S. and Qu, S. “Integrating HEVC Video Compression with High Dynamic Vampine Random Vigne Dynamic Randomness” ) ", SMPTE motion (Image Journal, Vol. 125, No. 1, February 2016, pp. 14-21).

  SDR backward compatibility with a decoding device and a rendering device is an important function depending on a video distribution system such as a broadcasting system or a multicasting system.

  A solution based on a single layer encoding / decoding process can have backward compatibility, eg, SDR compatibility, and can utilize legacy distribution networks and existing services.

  Such a single layer based delivery solution enables high quality HDR rendering on consumer electronics (CE) devices that support HDR, while high quality on CE devices that support SDR. Provides SDR rendering.

  Such a single layer based distribution solution generates an encoded signal, eg, an SDR signal and associated metadata (a few bytes per video frame or scene). The metadata can be used to reconstruct another signal, eg, an HDR signal, from a decoded signal, eg, an SDR signal.

  Parameter values stored with metadata that can be used for signal reconstruction may be static or dynamic. Static metadata means metadata that does not change for video (a set of images) and / or programs.

  Static metadata is valid for the entire video content (scenes, movies, clips ...) and may not depend on the image content. Static metadata may define, for example, an image format, color space, or color gamut. For example, SMPTE ST 2086: 2014, “Mastering Display Color Volume Metadata Supporting High Luminance and Wide Color Gamut Images, used in Mastering Display Color Volume Supporting High Color and Gamut Images Environments” This kind of static metadata. The Mastering Display Color Volume (MDCV) Auxiliary Extended Information (SEI) message is H.264. H.264 / AVC ("Advanced Video Coding for Generic Audiovisual Services"), Series H: Audiovisual and Multimedia Systems, Recommendation ITU-T H.264, ITU Electric The communication standardization department, January 2012) and the HEVC video codec will be delivered in ST2086.

  Dynamic metadata is content dependent. That is, the metadata may change, for example, for each image or for each group of images. For example, the SMPTE ST 2094: 2016 standard family “Dynamic Metadata for Color Volume Transform” is dynamic metadata used in the production environment. Thanks to Color Remapping Information (CRI) SEI messages, SMPTE ST 2094-30 can be delivered with HEVC encoded video streams.

  Another single layer based distribution solution exists on distribution networks where display-adaptive dynamic metadata is distributed along with legacy video signals. These single-layer based solutions provide HDR 10-bit image data (eg, image data “Recommended ITU-R BT.2100 where the signal is represented as an HLG10 or PQ10 signal as specified in ITU-R BT.2100-0). -0, image parameter values for high dynamic range television for use in production and international program exchange (from image parameter values for high dynamic range television for use in production and international program signals) Related metadata (usually 12 or 16 bits), eg HEVC main 10 profile code The HDR 10-bit image is encoded using an encoding scheme and the video signal is reconstructed from the decoded video signal and the associated metadata. The dynamic range of the reconstructed signal is adapted according to associated metadata that may depend on the characteristics of the display of interest.

  Dynamic metadata transmission in real-world production and distribution facilities includes splicing, overlay layer insertion, professional equipment pruning bitstreams, affiliate stream processing, and post production / professional plant metadata Due to the current lack of standards for transport, guarantees are difficult and there are concerns about loss and damage.

  The delivery solution based on a single layer does not work without the existence of different bundles of dynamic metadata, but in order to guarantee the successful reconstruction of the video signal, these dynamic metadata Some are important

  A similar problem occurs when the dynamic metadata does not match the image with graphics or overlays. This occurs, for example, when graphics (overlay, OSD,...) Are inserted (applied) into an image outside the distribution chain. This is because the metadata calculated for this image is also applied when graphics are inserted (applied) to the image. The reason is that the metadata calculated for this image is also applied when graphics are inserted (applied) to the image, and the metadata is given graphics or overlays Is considered not to match the existing image. The reason is that the metadata may not be adapted to the part of this image that contains graphics or overlays.

  Such problems can occur when the decoded image is displayed over time or with inappropriate metadata (eg, high-intensity OSD processed by metadata generated for dark content). The image may flicker at a fixed location in the graphics due to undesired effects (saturation, clipping,...) In the portion of the image containing the processed graphics or overlay.

  In the following, a simplified overview of the present principles is provided so that a basic understanding of some aspects of the present principles may be obtained. This summary is not an exhaustive overview of the principles of the application. It is not intended to identify key elements or key elements of the present principles. The following summary merely presents some simplified aspects of the present principles as a prelude to the more detailed description provided below.

At least one of the disadvantages of the prior art using a method or method of reconstructing image data representing original image data from decoded image data and parameters obtained from a bitstream The principle of the present application to overcome is described. The parameters are processed from the original image data. This method
Checking whether the above parameters are lost, corrupted, or aligned with the decoded image data to which graphics or overlays have been applied; ,
If at least one of the above parameters is missing, corrupted, or does not match the decoded image data with graphics or overlays attached,
Selecting a recovery mode according to information data indicating how the above parameters are being processed;
By applying the selected reconstruction mode, the reconstruction of the image data to the at least one missing, corrupted or inconsistent parameter, and further considering the restored parameter Restoring.

  According to one embodiment, the information data is explicitly signaled by syntax elements in the bitstream.

  According to one embodiment, information data (ID) is implicitly signaled.

  According to one embodiment, the information data identifies the content of the process applied to the original image data to process the parameters.

  According to one embodiment, the parameter is considered lost if this parameter is not obtained from the bitstream.

According to one embodiment,
The parameter is
The value of the above parameter is out of the value range,
The above parameters do not have coherent values according to other parameter values,
It is considered broken if at least one of the following conditions is met.

According to one embodiment,
The restoration mode is a parameter that depends on the restored parameters, even if only some of the parameters are not lost, corrupted, or inconsistent with the decoded image data with graphics or overlays applied. It replaces all of.

According to one embodiment,
The restore mode replaces each lost, corrupted, or mismatched parameter with the restored parameter.

  According to one embodiment, the restoration mode replaces each missing, corrupted or mismatched parameter with a value from a set of predetermined parameter values already stored.

  According to one embodiment, the restoration mode is characterized by at least one characteristic of the original image data or a characteristic or reconstruction of the mastering display used to grade the original image or the image data to be reconstructed. Selected according to at least one characteristic of the captured image data or at least one characteristic of the target display.

  In accordance with these other aspects, the present principles store apparatus including means for performing the above-described method and program code instructions for performing the method steps when the program is executed on a computer. It also relates to non-transitory processor readable media.

  In the drawings, examples of the principles of the present application are shown.

FIG. 4 shows steps of a method for reconstructing an image I3 representing an original image I1 from a decoded image according to an example of the principles of the present application. FIG. 4 is a diagram illustrating an end-to-end workflow that supports content production and delivery to CE displays that support HDR and SDR, in accordance with an example of the principles of the present application. FIG. 3 is a diagram illustrating a variation of the end-to-end workflow of FIG. 2 in accordance with an example of the principles of the present application. FIG. 6 is a diagram illustrating a variation of the end-to-end workflow of FIG. 2 in accordance with another embodiment of the present application. It is a figure which illustrates a perceptual conversion function. FIG. 4 shows a piecewise curve used for mapping. It is a figure which shows the example of the curve used for the conversion which returns a signal to a linear light area | region. FIG. 4 illustrates another example of the use of a method for reconstructing an image from decoded image data and parameters obtained from a bitstream, in accordance with an example of the present principles. FIG. 3 illustrates an example of an architecture of a device according to an example of the principles of the present application.

  Similar or identical elements are denoted using the same reference signs.

  The principles of the present application will be described more fully hereinafter with reference to the accompanying drawings, in which examples of the principles of the present application are shown. However, the principles of the present application may be implemented in many alternative forms and should not be construed as limited to the examples shown in the specification of the present application. Accordingly, the principles of the present application can be implemented in various modifications and alternative forms. Particular embodiments of the present principles are illustrated in the drawings and are described in detail herein. However, it is not intended that the principles of the application be limited to the particular forms disclosed, but rather that the disclosure be subject to all modifications within the spirit and scope of the principles of the application as defined by the claims. It will be understood that examples, equivalents, and variations are encompassed.

  The terminology used herein is for the purpose of describing particular embodiments and is not intended to limit the present disclosure. In this specification, unless stated otherwise in the context, the expressions “a”, “one”, “this” (or “an”) include plural expressions unless the context clearly dictates otherwise. Is also intended. Further, where the terms “consisting of”, “including”, and / or “comprising” are used herein, this means that the described feature, integer value, step, process, element, And / or components (components, components) are present, but one or more other features, integer values, steps, processes, elements, components (components, components), and / or their It will be understood that it does not exclude the presence or addition of groups. Further, when an element is described as “responding” or “connected” to another element, it may respond directly to or be connected to another element There may be elements. In contrast, when an element is described as “directly responding” or “directly connected” to another element, there are no intervening elements present. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items and may be abbreviated as “/”.

  In this specification, terms such as first and second are used to describe various elements, but it should be understood that these elements should not be limited by these terms. Like. These terms are only used to distinguish one term from another. For example, the first element may be defined as the second element without departing from the teachings of the present principles, and similarly, the second element may be defined as the first element.

  Although some drawings include an arrow on a communication path indicating the main direction of communication, it can be understood that communication may be performed in a direction opposite to the depicted arrow.

  Some embodiments are described with reference to block diagrams and operational flowcharts, where each block is a circuit element, module, or one or more executables for performing a particular logical function. Represents a portion of code that contains a simple instruction. Note that in other embodiments, the functions shown in the blocks may be in a different order than shown. For example, even if two blocks are shown in succession, depending on the function involved, the blocks may actually be executed substantially in parallel, or May be executed in reverse order.

  In this specification, references to “one embodiment” or “an embodiment” may mean that a particular feature, structure, or characteristic described in connection with the embodiment is at least a part of the present principles. It means that it may be included in one embodiment. The expressions “in one embodiment” or “in accordance with an embodiment (according to an embodiment)” in various places in the specification are not necessarily all referring to the same embodiment, and Neither embodiments nor alternative embodiments are mutually exclusive with respect to other embodiments.

  Reference numerals appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.

  Although not explicitly described, this embodiment and the modified examples can be used in any combination or in partial combination.

  In the following, uppercase symbols, for example, (C1, C2, C3) indicate components of the first image, and lowercase symbols, for example, (c1, c2, c3) have the first dynamic range of luminance. The components of another image that are smaller than the dynamic range of the luminance of the other image are shown.

The dynamic range of the brightness of the image is the ratio of the maximum value to the minimum value of the brightness of the image. Usually, the dynamic range of the luminance of the SDR image 500 is (100 cd / ^{m 2} for 0.2 cd / ^{m 2),} the dynamic range of the luminance of the HDR image is 1000 cd for 10000 (0.1cd / ^{m 2} / m ² ).

は、第１の画像のガンマ圧縮された成分を指し、これらのプライム記号が小文字の記号であるとき、例えば、（ｙ’，ｕ’，ｖ’）は、第２の画像のガンマ圧縮された成分を指す。 In the following, the prime symbols are, when these prime symbols are uppercase symbols and prime symbols, for example:

Refers to the gamma compressed component of the first image, and when these prime symbols are lowercase symbols, for example, (y ′, u ′, v ′) is a gamma compressed component of the second image Refers to ingredients.

  The principle of the present application will be described with respect to image encoding / decoding / reconstruction. As described below, since each image in a sequence of images is sequentially encoded / decoded / reconstructed, an image (video ( Video)) sequence encoding / decoding / reconstruction.

から原画像Ｉ_１を現す画像Ｉ_３を再構成する方法のステップを示す図である。 FIG. 1 illustrates a decoded image (outside 1) according to an example of the present principles.

FIG. 6 shows the steps of a method for reconstructing an image I _{3 representing an} original image I ₁ from

と一致していない場合に復元されたパラメータＰ_ｒである。 In step 10, the set of parameter SP is obtained in order to reconstruct the image I _3. These parameters are the parameters P obtained from the bitstream B, or the decoded image with at least one parameter P missing, corrupted, or given graphics or overlay (outside 2)

Is the restored parameter _Pr .

を取得し、ステップ１２において、モジュールＭ２は、パラメータＳＰの組を使用して復号された画像
（外４）

から画像Ｉ３を再構成する。 In step 11, the module M1 decodes the decoded image (outside 3).

In step 12, the module M2 determines that the decoded image (outer 4) using the set of parameters SP

To reconstruct the image I3.

は、ビットストリーム（信号）Ｂまたは他の任意のビットストリームから取得され、場合によっては、このビットストリームは、ローカル・メモリまたは他の任意の記憶媒体に記憶することができる。 Decoded image data (outside 5)

Is obtained from the bitstream (signal) B or any other bitstream, and in some cases this bitstream can be stored in local memory or any other storage medium.

  In sub-step 101 of (Step 10), the module M3 acquires a parameter P necessary for reconstructing the image I3.

と一致していないかどうかをチェックする。 In sub-step 102 of (Step 10), the module M4 determines that at least one of the parameters P has been lost, corrupted, or has been given a decoded image (external 6). )

Check whether it matches.

と一致していないものが存在しない場合には、パラメータＳＰの組は、パラメータＰのみを含む。 Decoded image (outside 7) missing, corrupted, or with graphics or overlay in parameter P

If there is no match, the set of parameters SP includes only the parameter P.

と一致していない場合には、（ステップ１０）のサブステップ１０３において、モジュールＭ５は、どのように上記パラメータが処理されているかを示す情報データＩＤを取得し、（ステップ１０）のサブステップ１０４において、モジュールＭ６は、上記情報データＩＤに従って、復元モジュールＲＭ_ｉを選択し、（ステップ１０）のサブステップ１０５において、モジュールＭ７は、選択された復元モジュールＲＭ_ｉを適用することによって、上記少なくとも１つの失われている、破損している、または一致していないパラメータを復元する。少なくとも１つの復元されたパラメータＰｒは、パラメータＳＰの組に追加される。 Decoded image (outside 8) where at least one of the parameters P is missing, corrupted, or provided with graphics or overlay

If not, in sub-step 103 of (Step 10), the module M5 obtains an information data ID indicating how the parameters are processed, and the sub-step 104 of (Step 10). The module M6 selects the restoration module RM _i according to the information data ID, and in the sub-step 105 of (Step 10), the module M7 applies the selected restoration module RM _i to at least 1 Restore one missing, corrupt, or mismatched parameter. At least one restored parameter Pr is added to the set of parameters SP.

In step 12 and taking into account the at least one restored parameter Pr, the image I ₃ is reconstructed.

This method is further used when multiple single-layer based delivery solutions share the same set of syntax elements that carry a common set of parameters. Parameters for single-layer based delivery solutions when you need various restoration modes (processing) to restore parameters that are missing, corrupted or mismatched Can be obtained with the advantage of ensuring the success of the reconstruction of the image I ₃ over the delivery solution based on the single layer.

の上にグラフィックスを挿入する場合に、この方法は、一致していないパラメータを復号された画像Ｉ_２にグラフィックス（またはオーバーレイ）を加えたものに適応したパラメータで置き換えるための特定の復元モードを選択し、グラフィックスまたはオーバーレイが付与された上記復号された画像
（外１０）

から上記復元されたパラメータを使用することによって画像Ｉ_３を再構成するため、これにより、何らかのちらつきのアーティファクトや所望しない効果によって再構成された画像の品質に影響が与えられることを回避するという利点がある。 This method further includes a CE device, usually a set-top box or player decoded image (outside 9)

When inserting graphics on top of this, this method uses a specific restoration mode to replace inconsistent parameters with parameters adapted to the decoded image I ₂ plus graphics (or overlay) And select the above decoded image (outside 10) with graphics or overlay

The advantage of reconstructing the image I ₃ by using the restored parameters from is that it avoids affecting the quality of the reconstructed image due to some flickering artifacts or unwanted effects There is.

  The method described with reference to FIG. 1 can be used in various applications where the image has to be reconstructed from the decoded image.

  FIG. 2 illustrates an end-to-end workflow that supports content production and distribution to CE displays that support HDR and SDR, in accordance with an example of the present principles.

および取得されたパラメータＳＰの組から原画像データＩ_１を表す画像Ｉ_３を再構成する方法を使用した例を示している。 This workflow involves delivery based on a single layer with associated metadata, and in accordance with the example of the present principles illustrated in FIG.

And an example using a method of reconstructing an image I ₃ representing original image data I ₁ from a set of acquired parameters SP.

  Basically, this single layer based delivery solution includes a pre-processing part and a post-processing part.

In the preprocessing part, the preprocessing stage 20 decomposes the original image I ₁ into a set of output images I ₁₂ and parameters SP, and the switching stage 24 encodes the original image I 1 or the output image I ₁₂ into a bitstream B. (Step 23).

In step 23, the image I ₂ can be encoded using a conventional video codec, and the bitstream B is transmitted on a particular channel or associated metadata embedded in the bitstream B. Carried over an existing conventional distribution network with (a set of parameters SP).

  In a variation, the bitstream B with accompanying metadata is stored on a storage medium such as a Blu-ray disc or a memory or register of a set top box, for example.

  In a variant, the associated associated metadata is carried by another specific channel or stored on a separate storage medium.

  Preferably, the video is H.264. 265 / HEVC codec (ITU-T H.265 Telecommunications Standards Division (10/2014) of ITU, Series H: Audiovisual and Multimedia Systems, Audiovisual Services Infrastructure-Video Coding, High Efficiency Video Encoding, recommendation ITU-T H.265) or H.264. H.264 / AVC ("Advanced Video Coding for Generic Audiovisual Services"), Series H: Audiovisual and Multimedia Systems, Recommendation ITU-T H.264, ITU Electric It is encoded using the communication standardization department, January 2012).

Depending on the information data ID, the original image I ₁ (possibly represented by components (C1, U ′, V ′) or Y′CbCr 4: 2: 0 PQ10 or HLG10 video signal) is step 23. If it is determined that the original image I1 is encoded, the original image I1 can be encoded with the HEVC main 10 profile.

If it is determined by the information data ID that the output image I ₁₂ is encoded in step 23, the output image I ₁₂ (Y′CbCr 4: 2: 0) is represented as a gamma transfer characteristic (standard dynamic range) signal. Can be encoded with any HEVC profile, including the Main 10 or Main profile.

が取得され（ステップ１１）、パラメータＳＰの組は、図１に説明されているように取得される（ステップ１０）。そして、前処理段階１０とは機能的に逆である、後処理段階２０は、復号された画像
（外１３）

およびパラメータＳＰの組から画像Ｉ_３を再構成する。 The information data ID can be transmitted as related metadata (step 23). In the post-processing part, the image decoded from the bit stream B (outside 12)

Are obtained (step 11), and the set of parameters SP is obtained as described in FIG. 1 (step 10). The post-processing stage 20, which is functionally opposite to the pre-processing stage 10, is a decoded image (outside 13).

And reconstruct the image I ₃ from the set of parameters SP.

  This single layer based delivery solution can optionally include format adaptation steps 21, 22, 25, 26 (optional).

のフォーマットを後処理段階１２の入力の特定のフォーマットに適応化させることができ、ステップ２６において、画像Ｉ_３を、対象の装置（例えば、セットトップ・ボックス、コネクテッドＴＶ、ＨＤＲ／ＳＤＲ対応ＣＥ装置、ブルーレイ・ディスク・プレイヤ）の少なくとも１つの特性に適応化させることができ、且つ／または、復号された画像
（外１５）

および画像Ｉ_３または原画像Ｉ_１が異なる色空間および／または色域で表される場合に、逆色域マッピングを使用することができる。 For example, in step 21 (optional), the format of the original image I ₁ can be adapted to the particular format (C1, U ′, V ′) of the input of the preprocessing stage 20, and in step 22 (optional) The format (c, u ′, v ′) of the output image I ₁₂ can also be adapted to a specific output format before encoding. In step 25, the decoded image (outside 14)

Can be adapted to the specific format of the input of the post-processing stage 12, and in step 26 the image I ₃ is converted to the target device (eg set-top box, connected TV, HDR / SDR compliant CE device). , Blu-ray Disc player) and / or decoded image (outside 15)

And, if the image I _3, or the original image I ₁ is represented by a different color space and / or color gamut, it is possible to use the inverse color gamut mapping.

  This format adaptation step (21, 22, 25, 26) may include color space conversion and / or gamut mapping. Normal format adaptation processing includes RGB to YUV or YUV to RGB conversion, BT. 709 to BT. 2020, or BT. 2020 to BT. 709, can be used for down-sampling or up-sampling of the saturation component. The known YUV color space also refers to a known YCbCr in the prior art. Annex E to Recommendation ETSI Recommendation ETSI TS 103 433 V1.1.1, Release 2016-8 provides format adaptation processing and inverse gamut mapping (Annex D).

To the input format adaptation step 21 further by applying a transfer function to the original image I _1, for example, to adapt the original image I ₁ of the bit depth for a particular bit-depth such as 10 bit Can be included. For example, PQ or HLG transfer functions can be used (Recommendation ITU-R BT.2100-0).

  More particularly, the preprocessing stage 20 includes steps 200-202.

ここで、ＴＭは、マッピング関数である。マッピング関数ＴＭは、原画像Ｉ_１の輝度のダイナミック・レンジを減少または増加させることができ、その逆の処理は、画像の輝度のダイナミック・レンジを増加または減少させることになるであろう。 In step 200, a first component c1 of the output image _{I 12} are obtained by mapping the first component C1 of the original image _{I 1.}

Here, TM is a mapping function. The mapping function TM can decrease or increase the dynamic range of the luminance of the original image I ₁ , and vice versa, will increase or decrease the dynamic range of the luminance of the image.

In step 201, the second and third components u ′, v ′ of the output image I ₁₂ correct the second and third components U ′, V ′ of the original image I ₁ according to the first component c _1. Is derived by

  The correction of the saturation component can be managed under control by tuning the mapping parameters. Therefore, color saturation (saturation) and hue are controlled.

によって除算される。 According to the embodiment in step 201, the second and third components U ′ and V ′ are
A scaling function (outside 16) having a value dependent on the first component c ₁

Divided by.

によって与えられる。 Mathematically speaking, the second and third components u ′, v ′ are

Given by.

ここで、ａおよびｂは、パラメータＳＰの組の２つのパラメータである。 In some cases, in step 202, the first component c1 can be adjusted to further control the perceived saturation, as follows.

Here, a and b are two parameters of the set of parameters SP.

This step 202 allows the brightness of the output image I ₁₂ to be controlled to ensure a perceived color match between the color of the output image I _{12 and} the color of the original image I ₁ .

に関連するパラメータを含むことができる。これらのパラメータは、ダイナミック・メタデータに関連しており、ビットストリームにおいて、例えば、ビットストリームＢにおいて搬送される。パラメータａおよびｂもまた、ビットストリームにおいて搬送することができる。 The set of parameters SP is the mapping function TM or its inverse function ITM, scaling function (outside 17)

Can include parameters related to. These parameters are related to the dynamic metadata and are carried in the bitstream, eg, bitstream B. Parameters a and b can also be carried in the bitstream.

  More specifically, in the post-processing portion, in step 10, a set of parameters SP is acquired as described in FIG.

  According to the embodiment of step 10, the set of parameters SP is carried by static / dynamic metadata obtained from a bitstream, including a specific channel or bitstream B, and possibly stored in a storage medium. .

を取得し、復号された画像
（外１９）

は、そして、ＳＤＲまたはＨＤＲ対応ＣＥディスプレイのために利用可能となる。 In step 11, the module M1 decodes the bitstream B to decode the image (outside 18).

Obtained and decoded image (outside 19)

Will then be available for SDR or HDR compliant CE displays.

  More particularly, the post-processing stage 12 includes steps 120-122.

の第１の成分ｃ_１が以下のように調節される。

ここで、ａおよびｂは、パラメータＳＰの組のうちの２つのパラメータである。 In optional step 120, the decoded image (outside 20)

The first component c ₁ is adjusted as follows.

Here, a and b are two parameters in the set of parameters SP.

In step 121, the first component C1 of the image _{I 3} is obtained by inverse mapping the first component _{c 1.}

の第２および第３の成分ｕ’、ｖ’を逆補正することによって導出される。 In step 122, an image obtained by decoding the second and third components U ′ and V ′ of the image I ₃ according to the component c ₁ (outside 21)

Of the second and third components u ′ and v ′ of FIG.

によって乗算される。 According to one embodiment, the second and third components u ′ and v ′ have a scaling function (outer 22) having a value that depends on the first component c _1.

Multiplied by

Mathematically speaking, the two first and second components U ′, V ′ are given by:

第２および第３の成分Ｕ’、Ｖ’は、（ＢＴ．７０９ ＯＥＴＦに近い）平方根を使用した擬似ガンマ化を原画像Ｉ_１のＲＧＢ成分に適用することによって導出される。

As illustrated in FIG. 3, according to the first embodiment of the method of FIG. 2, the preprocessing portion, the first component C1 of the original image I ₁ is less by the RGB components of the original image I ₁ The acquired linear light intensity component L,

The second and third components U ′, V ′ are derived by applying pseudogammaization using a square root (close to BT.709 OETF) to the RGB components of the original image I ₁ .

In step 200, the first component y1 of the output image I12 is obtained by mapping the line light component L.
y ₁ = TM (L)

In step 201, the second and third components u ′, v ′ of the output image I ₁₂ are derived by correcting the first and second components U ′, V ′ according to the first component y _1. The

In the post-processing portion, in step 121, the line light luminance component L of the image I ₃ is obtained by inverse mapping the first component c _1.

In step 122, the second and third component U of the image I ₃ ', V' is derived by inverse correction of the second and third components of the output image I ₁₂ according to the first component y _1.

によって乗算される。 According to one embodiment of step 122, the second and third components u ′ and v ′ are scaled functions (outside 23) whose values depend on the _first component y _1.

Multiplied by

によって与えられる。 Mathematically speaking, the two first and second components U ′, V ′ are

Given by.

によって、原画像Ｉ１のガンマ圧縮されたＲＧＢから取得された成分Ｙ’であり、第２および第３の成分Ｕ’、Ｖ’は、原画像Ｉ_１のＲＧＢ成分に対してガンマ処理を適用したものである。

ここでγは、好ましくは、２．４である、ガンマ係数とすることができる。 According to the second embodiment of the method of FIG. 2, as illustrated in FIG. 4, in the preprocessing part, the first component C1 of the original image I ₁ is

Accordingly, 'a, the second and third component U' original image gamma compressed component obtained from RGB Y of I1, V 'applied a gamma processing to the RGB components of the original image I ₁ Is.

Here, γ can be a gamma coefficient, preferably 2.4.

  The component Y ′, which is a non-linear signal, is different from the line light luminance component L.

In step 200, the first component y′1 of the output image I12 is obtained by mapping the component Y ′.

が第１の成分ｙ’１を逆マッピングすることによって取得される。

ここで、ＩＴＭは、マッピング関数ＴＭの逆関数である。 In step 121, the reconstructed component (outside 24)

Is obtained by inverse mapping the first component y′1.

Here, ITM is an inverse function of the mapping function TM.

は、よって、成分Ｙ’の値のダイナミック・レンジに属する。 Reconstituted ingredients (outside 25)

Therefore belongs to the dynamic range of the value of the component Y ′.

に従って、第１および第２の成分Ｕ’、Ｖ’を補正することによって導出される。 In step 201, the second and third components u ′, v ′ of the output image I ₁₂ are the first component y ′ ₁ and the reconstructed component (outer 26).

And is derived by correcting the first and second components U ′, V ′.

This step 201 to control the color of the output image I _12, it becomes possible to ensure a match between the color of the original image I _1.

  The correction of the saturation component can be managed under control by tuning the mapping parameters. Therefore, color saturation (saturation) and hue are controlled. Such control is usually not possible when non-parametric perceptual transfer functions are used.

の比に値が依存するスケーリング関数
（外２８）

によって除算される。

ここで、Ωは、原画像Ｉ_１の原色に依存する定数値である（例えば、ＢＴ．２０２０では、１．３となる）。 According to the embodiment of step 201, the second and third components U ′ are reconstructed components (outside 27) for component y ′ ₁

Scaling function whose value depends on the ratio of (outside 28)

Divided by.

Here, Ω is a constant value depending on the primary color of the original image I ₁ (for example, 1.3 in BT.2020).

が第１の成分ｙ’_１を逆マッピングすることによって取得される。

In the post-processing portion, in step 121, the components (outside 29) of the image I ₃ are displayed.

Is obtained by inverse mapping the _first component y ′ ₁ .

に従って、復号された画像
（外３１）

の第２および第３の成分ｕ’、ｖ’を逆補正することによって導出される。 In step 122, the second and third components U ′, V ′ of the image I ₃ are changed to the first component y ′ ₁ and the component (outside 30).

According to the decoded image (outside 31)

Of the second and third components u ′ and v ′ of FIG.

によって乗算される。 According to the embodiment of step 122, the second and third components u ′ and v ′ are scaled functions (outside 32).

Multiplied by

によって与えられる。 Mathematically speaking, the two first and second components U ′, V ′ are

Given by.

The mapping function TM is based on a perceptual transfer function, the purpose of which is to reduce (or increase) the dynamic range of luminance values by converting the components of the original image I _{1 into} the components of the output image I ₁₂ . There is. Thus, the component values of the output image I ₁₂ have a lower (or higher) dynamic range than the component values of the original image I ₁ .

  This perceptual transfer function uses a limited set of control parameters.

  FIG. 5a illustrates the perceptual transfer function. This perceptual transfer function can be used for luminance component mapping, but a similar perceptual transfer function can be used for luminance component mapping.

The mapping is controlled by a mastering display peak luminance parameter (which is 5000 cd / m ² in FIG. 5a). In order to better control the black and white levels, signal-stretching between content-dependent black and white levels is applied. The transformed signal is then mapped using a piecewise curve composed of three parts, as illustrated in FIG. 5b. The lower and upper parts are linear and the steepness is determined using the shadowGain parameter and the highlightGain parameter, respectively. The middle part is a parabola that provides a smooth bridge between the two linear parts. The width of the crossover is determined by the midToneWidthAdjFactor parameter.

  All of the parameters that control the mapping can be carried using, for example, SEI messages defined by JCTVC-W0133 carrying SMPTE ST 2094-20 metadata.

FIG. 5c shows a perceptual transfer function showing how a perceptually optimized video signal is converted back into the linear region based on the conventional display maximum brightness of the object, eg 100 cd / m ^2. An example of the inverse function of TM (FIG. 5a) is shown.

から画像Ｉ_３を再構成するために取得される。 In step 10 (FIG. 1), an image obtained by decoding a set of parameters SP (outside 33)

It is obtained in order to reconstruct the image I ₃ from.

  These parameters can be obtained from metadata obtained from a bitstream, eg, bitstream B.

  Recommendation ETSI TS 103 433 V1.1.1 Section 6, 2016-08 provides examples of the syntax of the metadata.

から任意の画像Ｉ_３を再構成することに拡張することができる。 The syntax of Recommendation ETSI TS 103 433 v1.1.1 has been described for reconstructing SDR video from HDR video, but this syntax can be used for any decoded image (outside 34).

It can be extended to reconstruct any image I ₃ from.

の処理を行う。これは、これらの関数が第１の成分ｃ_１に依存するためである。 Post-processing (step 12) consists of inverse mapping function ITM and scaling function (outside 35) derived from dynamic metadata.

Perform the process. This is because these functions depend on the _first component c1.

  According to the recommendation ETSI TS 103 433 V1.1.1, the dynamic metadata can be carried according to a so-called parameter-based mode or a table-based mode.

  Parameter-based mode is beneficial for delivery workflows and its main purpose is direct SDR backward compatibility with the use of very low additional payload or bandwidth to carry dynamic metadata To provide services. Table-based mode is beneficial for workflows with low-end terminals or when a high level of adaptation is required to properly represent both HDR and SDR streams.

In the case of parameter-based mode, the dynamic metadata to be conveyed is a luminance mapping parameter representing the inverse function ITM, ie
tmInputSignalBlackLevelOffset,
tmInputSignalWhiteLevelOffset,
shadowGain,
highlightGain,
midToneWidthAdjFactor,
tmOutputFineTuning parameters,
It is.

を定義するために使用される色補正パラメータ（（ｓａｔｕｒａｔｉｏｎＧａｉｎＮｕｍＶａｌ、ｓａｔｕｒａｔｉｏｎＧａｉｎＸ（ｉ）およびｓａｔｕｒａｔｉｏｎＧａｉｎＹ（ｉ））である（ＥＴＳＩ勧告ＥＴＳＩ ＴＳ １０３ ４３３ Ｖ１．１．１節６．３．５および６．３．６）。 Other dynamic metadata to be further transported is a function (outside 36)

Are the color correction parameters ((saturationGainNumVal, saturationGainX (i) and saturationGainY (i)) (ETSI recommendations ETSI TS 103 433 V1.1.1 section 6.3.5 and 6.3. 6).

  Note that the parameters a and b are each carried / hidden in the above-described saturationGain function parameter.

  These dynamic metadata are stored in the HEVC color volume reconfiguration information (CVRI) user data registration SEI message based on the SMPTE ST 2094-20 specification (Recommendation ETSI TS 103 433 V1.1.1 Annex A.3). Can be used for transmission.

  A typical dynamic metadata payload is about 25 bytes per scene.

  In step 101, a CVRI SEI message is parsed to obtain mapping parameters and color correction parameters.

  In step 12, an inverse mapping function ITM (so-called IutMapY) is reconstructed (derived) from the acquired mapping parameters (see Recommendation ETSI TS103 433V1.1.1 section 7.2.3. For further details). 1).

（ＩｕｔＣＣ）もまた、取得された色補正パラメータから再構成される（導出される）（さらなる詳細については、勧告ＥＴＳＩ ＴＳ１０３ ４３３Ｖ１．１．１ 節７．２．３．２を参照）。 In step 12, the scaling function (outside 37)

(IutCC) is also reconstructed (derived) from the acquired color correction parameters (see recommendation ETSI TS103 433V1.1.1 section 7.2.3.2 for further details).

In the table-based mode, the dynamic data to be conveyed is a piecewise linear curve pivot point representing the inverse mapping function ITM. For example, the dynamic metadata is a luminance MappingNumVal indicating the number of pivot points, a LuminanceMappingX indicating the x value of the pivot point, and a Luminance MappingY indicating the y value of the pivot point (for further details, see Recommendation ETSI TS103 433 V1.1). .1 see section 6.2.7 and section 6.3.7).

を表す区分的な線形曲線のピボット・ポイントとすることができる。例えば、ダイナミック・メタデータは、ピボット・ポイントの数を示すｃｏｌｏｒＣｏｒｒｅｃｔｉｏＮｕｍＶａｌ、ピボット・ポイントのｘ値を示すｃｏｌｏｒＣｏｒｒｅｃｔｉｏｎＸ、ピボット・ポイントのｙ値を示すｃｏｌｏｒＣｏｒｒｅｃｔｉｏｎＹである（さらなる詳細については、勧告ＥＴＳＩ ＴＳ１０３ ４３３Ｖ１．１．１ 節６．２．８および節６．３．８を参照）。 In addition, other dynamic metadata to be transported is converted into a scaling function (outside 38).

Can be a pivot point of a piecewise linear curve representing. For example, the dynamic metadata is colorCollectionNumVal indicating the number of pivot points, colorCollectionX indicating the x value of the pivot point, and colorCollectionY indicating the y value of the pivot point (for further details, see Recommendation ETSI TS103 433 V1.1). .1 see section 6.2.8 and section 6.3.8).

  These dynamic metadata can be converted using the HEVC color remapping information (CRI) SEI message, whose syntax is based on the SMPTE ST 2094-30 specification (Recommendation ETSI TS 103 433 V1.1.1 Annex A.4). Can be transmitted.

  A typical payload is about 160 bytes per scene.

を表す区分的な線形曲線のピボット・ポイントおよび彩度から輝度への入射パラメータａおよびｂが取得される。 In step 102, a piecewise linear curve representing the inverse mapping function ITM by parsing a CRI (color remapping information) SEI message (as defined in the HEVC / H.265 version published in December 2016). Pivot point, scaling function (outside 39)

Pivot points of the piecewise linear curve representing and the incident parameters a and b from saturation to luminance are obtained.

  In step 12, the inverse mapping function ITM is derived from that of the pivot point for the piecewise linear curve representing this inverse mapping function ITM (for further details see Recommendation ETSI TS103 433V1.1.1 section 7.2). See 3.3.).

もまた、このスケーリング関数
（外４１）

を表す区分的な線形曲線に関するピボット・ポイントのものより導出される（さらなる詳細については、勧告ＥＴＳＩ ＴＳ１０３ ４３３Ｖ１．１．１ 節７．２．３．４を参照）。 In step 12, the scaling function (outside 40)

Also this scaling function (outside 41)

Is derived from that of the pivot point for piecewise linear curves representing (see recommendation ETSI TS103 433V1.1.1 section 7.2.3.4 for more details).

  Note that the static metadata used by the post-processing stage can also be carried by the SEI message. For example, the choice of either a parameter-based mode or a table-based mode may be selected based on the information (TSI) user user as defined by Recommendation ETSI 103 433 V1.1.1 (section A.2.2). It can be carried by a data registration SEI message (payloadMode). For example, the primary color or maximum display mastering display luminance is carried by a mastering display color volume (MDCV) message as defined by AVC, HEVC.

  According to the embodiment of step 103, in the bitstream, the information data ID is explicitly signaled by a syntax element in the bitstream and is thus obtained by parsing the bitstream.

  For example, the syntax element is part of an SEI message.

According to one embodiment, the information data ID identifies what processing is applied to the original image I ₁ to process the set of parameters SP.

According to this embodiment, and, using the information data ID, it leads to how to use the parameters, it is possible to reconstruct the image I ₃ (step 12).

がＳＤＲ画像であることを示す。 For example, if the information data ID is 1, the information data ID is acquired parameter SP by applying a pre-processing stage (step 20) with respect to the original HDR image I _1, the decoded image ( 42)

Indicates an SDR image.

がＨＤＲ１０画像であり、マッピング関数ＴＭがＰＱ伝達関数であることを示す。 If the information data ID is 2, this information data ID is obtained by applying the preprocessing stage (step 20) to the HDR 10-bit image (input of step 20), and the decoded image (Outside 43)

Indicates an HDR10 image and the mapping function TM is a PQ transfer function.

がＨＬＧ１０画像であり、マッピング関数ＴＭが原画像Ｉ_１に対するＨＬＧ伝達関数であることを示す。 If the information data ID is 3, the information data ID is obtained by applying the preprocessing stage (step 20) to the HDR 10-bit image (input of step 20), and the decoded image (Outside 44)

There is HLG10 image, indicating that the mapping function TM is HLG transfer function for the original image _{I 1.}

  According to the embodiment of step 103, the information data ID is implicitly signaled.

  For example, the syntax element transfer characteristic present in the VUI of HEVC (Annex E) or AVC (Annex E) usually identifies the transfer function (mapping function TM) to be used. Different single layer delivery solutions use different transfer functions (PQ, HLG,...) And therefore implicitly identify the restoration mode to use using the syntax element transfer characteristics. Can do.

  The information data ID can also be implicitly signaled by services defined in higher transports or system layers.

According to another example, the peak luminance value and color space of image I ₃ can be obtained by parsing the MDCV SEI message carried by the bitstream, and the information data ID is the peak luminance value and color space. It can be derived from a specific combination of (primary colors).

  According to the embodiment of step 102, if the parameter P is not present in the bitstream (not obtained from the bitstream), this parameter P is considered lost.

  For example, if the parameter P is carried by a SEI message such as a CVRI or CRI SEI message as described above, the parameter P if the SEI message is not sent in the bitstream or the parsing of the SEI message fails. Is considered lost (or absent).

In accordance with the embodiment of step 103, a parameter is considered corrupted if at least one of the following conditions is met:
The value of the parameter is out of the predetermined value range (for example, when the conforming range is 0 to 6, the saturation_gain_num_val is 10)
The parameter does not have a coherent value according to other parameter values (eg, saturation_gain_y [i] includes outliers, ie, values far away from other saturation_gain_y [i] values). (From [0] to saturation_gain [4] is a value in the range of 0 to 16, and saturation_gain [1] = 255.)

According to an embodiment of the method, the restoration module RM _i is consistent with a decoded image in which only some of the parameters are not corrupted, lost or have graphics or overlays attached. if not, replace all parameters P by reconstructed parameter P _r.

In accordance with an embodiment of the method, different restore mode RM _i is lost, corrupted, or replaced by restoring the parameters P does not match the parameter P _r.

According to an embodiment of the method, the restoration mode RM _i replaces a lost, corrupted or inconsistent parameter P with a value from a set of predetermined parameter values already stored. It is.

  For example, the predetermined set of parameter values may collect a predetermined value for at least one metadata carried by a CRI and / or CVRI SEI message.

  A particular set of predetermined parameter values can be determined, for example, for a delivery solution based on each single layer identified by the information ID.

Table 1 shows a non-limiting example of a specific set of predetermined values for a delivery solution based on three different single layers.

  According to Table 1, three different sets of predetermined values are defined according to the information data ID. These predetermined value sets are the reconstructed values defined for some parameters used by the post-processing stage. The other parameters are set to a common fixed value for the various single layer solutions.

According to the embodiment of step 104, the restoration module RM _i grades at least one characteristic of the original image (image I ₁ ), usually the peak brightness of the original content, or the input image data or the image data to be reconstructed. mastering display peak luminance to be used for, or at least one property of another video, it is typically selected according to the peak brightness of the reconstructed image I _3, or target display.

According to one embodiment, the restoration mode RM _i is a characteristic of the original video (I ₁ ) or a mastering display used for grading input image data or image data to be reconstructed (eg, ST2086). Whether or not there is a characteristic defined in (1) above, and at least one restored parameter is calculated from the characteristic. In the absence the characteristics of the input video, in the absence of the characteristics of mastering device checks whether the reconstructed image I _3, or characteristics of the object of display is present (e.g., CTA-861 .. peak luminance as defined in .3) and at least one restored parameter is calculated from the above characteristics. If the characteristic of the reconstructed image I ₃ does not exist and the characteristic of the target display does not exist, the at least one restored parameter is a video standardization committee (eg, 1000 cd / m ^2). Or a fixed value (as determined by the industry forum).

In accordance with a non-limiting example, Table 2 provides examples of restoration values for some parameters used by post-processing steps that depend on the presence of available information about input / output content and mastering / target displays. To do.

  If the parameter matrix_coefficient_value [i] is present, this parameter is input to the input / output video color space obtained by parsing the MDCV SEI / ST2086 message, BT. 709 or BT. Can be set according to 2020 (input or output video characteristics). The restoration mode depends on the color space.

  If shadow_gain_control is present, this parameter can be calculated according to the value obtained by parsing the MDCV SEI / ST 2086 message.

である。 For example, information indicating the peak luminance of the mastering display is obtained from the MDCV SEI / ST 2086 message, and the parameter shadow_gain_control is calculated by (restoration mode 1).
shadow_gain_control = Clip) (0; 255; Floor (rs (hdrDisplayMaxLuminance) x 127,5 +0,5)
here,

It is.

It is likely that the value of hdrDisplayMaxLuminance is known in the service level information or for a specific workflow. This value can be set to the peak luminance of the target display (for output reproduction) when this characteristic is available. Otherwise (restoration mode 2), an arbitrary default value, usually 1000 cd / m ² is set. The default value corresponds to the reference maximum display mastering brightness that is currently found in most current HDR markets.

およびビットストリームＢから取得されたパラメータＳＰの組から画像Ｉ_３を再構成する方法を使用した別の例を示している。 FIG. 6 illustrates decoded image data (outside 45) according to an example of the present principles.

And another example using a method of reconstructing an image I ₃ from a set of parameters SP obtained from the bitstream B.

にオーバーレイが追加されなければならない旨のイベントを決定モジュールに信号伝達／送信する（通常は、ｏｖｅｒｌａｙ＿ｐｒｅｓｅｎｔ＿ｆｌａｇを１に設定する）任意の（中間）装置で実施されるように意図されている。 This example implements, at least in part, an overlay insertion and mixing mechanism (eg, a set-top box or an Ultra Hd Blu-ray player) and a decoded image (outside 46)

It is intended to be implemented on any (intermediate) device that signals / sends an event that an overlay must be added to the decision module (usually setting overlay_present_flag to 1).

が取得され（ステップ１１）、画像Ｉ_３が再構成される（ステップ１２）。 If no overlay (graphics) needs to be applied to the decoded image, a set of parameters SP is obtained (step 10) and the decoded image (outside 47) as described in FIG. )

There is obtained (step 11), the image _{I 3} is reconstructed (step 12).

に付与されるべき場合には、復号された画像
（外４９）

が取得され（ステップ１１）、ステップ６０において、合成画像Ｉ’_２が復号された画像
（外５０）

にグラフィックス（オーバーレイ）を付与することによって取得される。 Image with overlay decoded (outside 48)

Decoded image (outside 49)

(Step 11), and in step 60, the composite image I ′ ₂ is decoded (outside 50).

Is obtained by adding graphics (overlay) to.

The information data ID is acquired (step 103), the restoration module is selected (step 104), is applied Restore Mode RMi selected (step 105) the reconstructed parameter _{P r} is obtained.

から再構成される Then, the image I ₃ includes the restored parameter P _r and the decoded image (outside 51)

Reconstructed from

  According to one embodiment, the parameter Pr is obtained by training a large number of images of various aspects (bright, rank, logo ...).

にパラメータＰを加えたもの、または、合成画像Ｉ’_２にパラメータＰ_ｒを加えたものが上記ＴＶセットに送信される。 In some cases (not shown in FIG. 6), step 12 may be performed on a remote device such as a TV set. In that case, the decoded image (outside 52)

To which the parameter P is added or the composite image I ′ ₂ to which the parameter _Pr is added is transmitted to the TV set.

  In FIGS. 1-6, a module is a functional unit and may be associated with a distinct physical unit or may not be associated with a physical unit. For example, some of these modules or these modules are a collection of unique components or circuits, while others contribute to software functionality. On the other hand, some modules may be physically configured separately. Devices in accordance with the principles of the present application may be pure hardware, such as Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), and very large scale integrations. It may be implemented using dedicated hardware such as a circuit (VLSI: Very Large Scale Integration), or may be implemented with some integrated electronic components embedded in the device, or hardware And may be implemented by combining software components.

  FIG. 7 represents an exemplary architecture of an apparatus 70 configured to implement the method described in connection with FIGS.

Device 70 is an element linked to each other by data and address bus 71,
For example, a processor 72 (or CPU) that is a DSP (Digital Signal Processor),
ROM (Read Only Memory) 73,
RAM (Random Access Memory) 74,
An I / O interface 75 for transmitting and receiving data to and from the application;
A battery 76.

  According to one example, the battery 76 is external to the device. In each of the above-mentioned memories, where the term “register” is used in the specification, this may correspond to a small capacity area (some bits) or a very large capacity. (For example, the entire program or a large amount of received or decoded data). The ROM 73 includes at least a program and parameters. The ROM 73 can store an algorithm and instructions for executing a technique according to the principle of the present application. When the power is turned on, the CPU 72 uploads the program in the RAM and executes the corresponding command.

  The RAM 74 is executed by the CPU 72 in the register and uploaded after the device 70 is powered on, input data in the register, intermediate data in various states of the method in the register, and further in the register Contains various variables used for execution of the method.

  The implementations described herein can be implemented, for example, in the form of a method or process, an apparatus, a software program, a data stream, or a signal. Whether an embodiment is described only in the context of a single form (eg, only described as a method or apparatus), the described features may be It can also be implemented in a form (for example, a program). The device can be implemented, for example, with suitable hardware, software, and firmware. The method can be implemented, for example, on an apparatus. An apparatus is, for example, a processor, which generally refers to a processing apparatus including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor further includes communication devices such as, for example, computers, cell phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate communication of information between end users.

According to one example, the input video or the original image of the input video is obtained from the source. For example, the source is
Local memory (73 or 74), for example, video memory or RAM (or Random Access Memory), flash memory, ROM (or Read Only Memory), hard disk,
A storage interface (75), for example, an interface to mass storage, RAM, flash memory, ROM, optical disk or magnetic support;
A communication interface (75), for example, a wired interface (eg, a bus interface, a wide area network interface, a local area network interface) or a wireless interface (IEEE 802.11 interface or Bluetooth ™ interface);
A picture capturing circuit (for example, a sensor such as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS));
Is a combination of elements including

  According to an example, a bitstream carrying metadata is transmitted to a destination. By way of example, one or both of these bitstreams are stored in local or remote memory, eg, video memory (74) or RAM (74), hard disk (73). In a variant, at least one of the bitstreams is sent to a storage interface (75), for example an interface with a mass storage device, flash memory, ROM, optical disk or magnetic support, and / or , Transmitted over a communication interface (75), eg, an interface with a point-to-point link, a communication bus, a point-to-multipoint link, or a broadcast network.

  According to another example, a bitstream carrying metadata is obtained from a source. Illustratively, the bitstream is read from local memory, eg, video memory (74), RAM (74), ROM (73), flash memory (73), or hard disk (73). In a variant, the bitstream is received from a storage interface (75), for example an interface with mass storage, RAM, ROM, flash memory, optical disk or magnetic support and / or a communication interface (75). ), For example, from a point-to-point link, bus, point-to-multipoint link, or interface with a broadcast network.

According to an example, the device 70 is configured to perform the method described above,
A mobile device,
A communication device;
Game devices,
A tablet (or tablet computer),
With laptop,
A still image camera,
A video camera,
An encoding chip / decoding chip;
TV set,
A set-top box,
Display,
A still image server;
A video server (eg, broadcast server, video-on-demand server, or web server);
Is a combination of elements including

  The various processing embodiments described herein may be implemented in a variety of different devices and applications. Examples of such equipment are encoders, decoders, post processors that process the output from the decoder, preprocessors that provide input to the encoder, video encoders, video decoders, video codecs, web servers , Set-top boxes, laptops, personal computers, mobile phones, PDAs, and any other device that processes images or video, or other communication devices. It will be clear that the device may be a portable device and may even be installed in a moving vehicle.

  Further, the method may be implemented by instructions executed by a processor, and such instructions (and / or data generated by an implementation) may be stored on a computer readable storage medium. A computer readable storage medium may take the form of a computer readable program product having computer readable program code implemented thereon and executed by one or more computer readable media. As used herein, a computer-readable storage medium is considered to be a non-transitory storage medium that is provided with a unique function of storing information as well as a unique function of storing information. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, such as those listed above. Any combination can be used as appropriate. More specific examples of computer readable storage media to which the present principles can be applied are provided below, but the listed examples are illustrative only and are not exhaustive. It can be easily understood by a trader. Portable computer disk, hard disk, read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), portable compact disk, read only memory (CD-ROM), optical storage, magnetic storage A device, or any combination of the above.

  The instructions can take the form of application programs that are actually implemented on a processor-readable medium.

  For example, the instructions can be hardware, firmware, software, or a combination thereof. The instructions can reside, for example, in the operating system, a separate application, or a combination of the two. Thus, a processor is characterized, for example, as both a device configured to perform processing and a device including a processor-readable medium (such as a storage device) having instructions to perform processing. Further, the processor readable medium may store data values generated by the implementation in addition to or instead of the instructions.

  It will be apparent to those skilled in the art that, depending on the implementation, various signals may be generated that are formatted to carry, for example, information that can be stored and transmitted. The information includes, for example, instructions for performing the method or data generated by one of the above embodiments. For example, the signal may have data as the actual syntax value described by the described example of the present principle, so as to hold the rules for reading and writing the syntax of the described example of the present principle. Can be formatted to hold as Such a signal can be formatted, for example, as an electromagnetic wave (eg, using the radio frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating the carrier with the encoded data stream. The information carried by the signal is, for example, analog information or digital information. The signal can be transmitted over various known wired or wireless links. The signal can be stored on a processor-readable medium.

  Several embodiments have been described. However, it will be understood that various modifications can be made. For example, other embodiments can be created by combining, supplementing, changing, or removing a plurality of different embodiments. Moreover, those skilled in the art can replace the disclosed content with other structures and processes, and the resulting embodiments perform at least substantially the same function in at least substantially the same way. It will be understood that this produces at least substantially the same effects as the disclosed embodiments. Accordingly, these and other embodiments are contemplated by this application.

Claims (13)

Decoded image data (
(Outside 53)

) And image data (I ₃ ) representing original image data (I ₁ ) from (101) parameters acquired from the bitstream, wherein the parameters are derived from the original image data (I ₁ ). Has been processed,
Checking (102) whether the parameters are missing, corrupted, or inconsistent with decoded image data to which graphics or overlays have been applied;
If at least one of the parameters is missing, corrupted, or does not match the decoded image data to which graphics or overlay is applied,
Selecting a restoration mode (RMi) according to information data (ID, 103) indicating how the parameter is being processed;
Applying the reconstruction of the selected restoration mode (RMi), image data (I ₃ ) to the at least one missing, corrupted or mismatched parameter and further restored Reconstructing by taking into account parameters (12),
Said method. An apparatus for reconstructing image data representing original image data from decoded image data and parameters obtained from a bitstream, wherein the parameters are processed from the original image data;
The device is
Means for checking whether the parameters are missing, corrupted, or inconsistent with the decoded image data to which graphics or overlays have been applied;
If at least one of the parameters is missing, corrupted, or does not match the decoded image data to which graphics or overlay is applied,
Means for selecting a restoration mode according to the information data indicating how the parameter is being processed;
Means for restoring the lost, corrupted or inconsistent parameters by applying the selected restoration mode, the reconstruction of image data, and further considering the restored parameters And the apparatus.   The method of claim 1 or the apparatus of claim 2, wherein the information data (ID) is explicitly signaled by syntax elements in a bitstream.   The method of claim 1 or the apparatus of claim 2, wherein the information data (ID) is implicitly signaled. The method or apparatus according to any one of claims 1 to 4, wherein the information data (ID) identifies the content of processing applied to the original image data (I ₁ ) to process parameters.   The method according to any one of claims 1 and 3-5 or any one of claims 2-5, wherein the parameter is considered lost if the parameter is not obtained from the bitstream. Equipment. The parameter is
The value of the parameter is outside the value range;
The parameter does not have a coherent value according to other parameter values;
7. A method according to any one of claims 1 and 3-6 or any one of claims 2-6, which is considered to be broken if at least one of the following conditions is met. Equipment.   The restoration mode is a parameter that is restored even if only some of the parameters are not lost, corrupted or not consistent with the decoded image data with graphics or overlays applied. The method according to any one of claims 1 and 3 to 7, or the apparatus according to any one of claims 2 to 7, wherein all of the parameters are replaced by:   9. The method or any one of claims 1 and 3-8, wherein the restoration mode is to replace each lost, corrupted or mismatched parameter with a restored parameter. Apparatus according to any one of claims 2-8.   10. The restoration mode replaces each missing, corrupted or mismatched parameter with a value from a predetermined set of parameter values already stored. 10. A method according to any one of the preceding claims or an apparatus according to any one of claims 2-9.   The restoration mode is at least one characteristic of the original image data, or at least one characteristic of the mastering display used to grade the original image data or the image data to be reconstructed, or reconstructed 11. A method according to any one of claims 1 and 3-10 or any one of claims 2-10, selected according to at least one characteristic of image data or at least one characteristic of a display of interest. The device described.   A computer program product comprising program code instructions for performing the steps of the method according to any of claims 1, 3 to 11, when the program is executed on a computer.   A non-transitory processor readable medium having stored thereon program code instructions for performing the steps of the method according to any one of claims 1, 3 to 11, when the program is executed on a computer.

Images