RU2818525C2 - High dynamic range video content type metadata - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: group of inventions relates to video playback on a target display. Method includes receiving a digital video bitstream containing metadata, including one or more category fields indicating the category of the video content of the digital video bitstream, and a reference mode flag indicating whether the one or more category fields condition the target display to select display settings for displaying video content in the reference mode. If the flag indicates the selection by the target display of the display settings for displaying the video content in the reference mode, then determining a display mode which corresponds to a certain category of video content, applying display settings of the determined display mode to the target display and displaying the video content.

EFFECT: providing automatic determination of a display mode which corresponds to a certain category of video content.

11 cl, 15 dwg

Description

CROSS REFERENCE TO RELATED APPLICATIONS

[1] This application claims priority to U.S. Provisional Patent Application No. 62/868,070, filed June 28, 2019, and European Patent Application No. 19183269.0, filed June 28, 2019, each of which is incorporated herein by reference throughout completeness.

TECHNICAL FIELD

[2] The present invention relates generally to images. More specifically, an embodiment of the present invention relates to video content type metadata.

BACKGROUND OF THE INVENTION

[3] As used herein, the term "dynamic range" may refer to the ability of the human visual system to perceive a range of intensities (eg, illuminance, brightness) in an image, such as from the darkest blacks (shadows) to the lightest whites (highlights) of an image. In this sense, dynamic range corresponds to "scene-related" intensity. Dynamic range may also refer to the ability of a display device to sufficiently or approximately reproduce a specific width range of intensity. In this sense, dynamic range corresponds to "display-related" intensity. Unless specifically stated to have a particular meaning anywhere in the description herein, it is intended that a term may be used in any sense, such as interchangeably.

[4] As used herein, the term “high dynamic range” (HDR) refers to the width of the dynamic range, which covers approximately 14 to 15 orders of magnitude of the human visual system. In practice, the dynamic range over which a person can simultaneously perceive spatial width across a range of intensity may be reduced slightly relative to HDR. As used herein, the terms “enhanced dynamic range” (EDR) or “visual dynamic range” (VDR) may individually or interchangeably refer to the dynamic range that is simultaneously perceived by the human visual system. As used herein, EDR may refer to a dynamic range that spans five to six orders of magnitude. Thus, although a slightly narrower width may be possible with respect to actual scene-specific HDR, EDR nonetheless represents wide dynamic range width and may also be referred to as HDR.

[5] In practice, images contain one or more color components (eg, luminance Y and chrominance Cb and Cr), with each color component represented to the nearest n bits per pixel (eg, n=8). When using linear luma coding, images in which n<8 (for example, color 24-bit JPEG images) are considered to be standard dynamic range images, while images in which n>8 may be considered to be high dynamic range images. EDR and HDR images can also be stored and distributed using high-precision (e.g., 16-bit) floating point formats, such as the OpenEXR file format developed by Industrial Light and Magic.

[6] Most consumer desktop displays support brightness between 200 and 300 cd/m, or nits. Most consumer HDTVs range from 300 to 1000 cd/m. Thus, such traditional displays are typical representatives of low dynamic range (LDR), also called standard dynamic range (SDR), relative to HDR or EDR. As the availability of EDR content increases due to developments in both capture equipment (such as cameras) and EDR displays (such as the PRM-4200 Professional Reference Monitor from Dolby Laboratories), EDR content can be color sorted and displayed on EDR displays that support wider dynamic ranges (for example, 1000 nits to 5000 nits or more).

[7] As used herein, the term "metadata" refers to any supporting information that is carried as part of the encoded bitstream and helps the decoder present the decoded image. Such metadata may include, but is not limited to, color space or gamma information, source display parameters, and auxiliary signal parameters, such as those described herein.

[8] As used herein, the term “display control” includes, but is not limited to, processing (e.g., tone and gamma mapping) applied to map a first dynamic range video input signal (e.g., one thousand nits) to a second dynamic range display (e.g., five hundred nits).

[9] The approaches described in this section are approaches that can be performed, but not necessarily approaches that have been previously proposed or performed. Therefore, unless otherwise noted, it should not be assumed that any of the approaches described in this section are considered prior art simply because of their inclusion in this section. Likewise, based on this section, it should not be assumed that disadvantages identified with respect to one or more approaches have been addressed in the prior art unless otherwise indicated.

SUMMARY OF THE INVENTION

[10] The apparatus and methods of the present invention provide solutions to the problem of storing the creative intent of a video in a decoder or target display. According to an embodiment of the present invention, the video bitstream includes metadata that conveys to the decoder or target display the type of video content found in the bitstream. This metadata may include numerous fields that indicate characteristics of the video content, such as content type and content subtype, that cause the target display to use the appropriate display mode with associated display settings. The target display can apply additional adjustments to individual display settings based on fields in the metadata that specify, for example, desired white point, whether or not to use video in reference mode, desired sharpness, desired noise reduction, desired MPEG noise reduction, desired frame rate conversion, desired the average brightness level of the image and the desired color. A number of fields in the metadata are added with a specific hierarchy. The hierarchy of fields causes additional refinement of display settings that have already been adjusted by a field belonging to a higher level of the hierarchy. For example, the content type may determine the basic adjustment of specific display settings at the first level of the hierarchy. When you select a more specific content subtype at the second level of the hierarchy, some of the display settings are further adjusted in accordance with the characteristics of the selected content subtype. At the third level of the hierarchy, fields determine individual adjustments to specific display settings. This metadata is intended to allow creators to effortlessly label the content they create. Depending on the effort involved, content creators can apply basic adjustments by simply selecting the appropriate content type and optionally a content subtype. However, they can also apply additional adjustments at the individual level that directly affect specific display settings on the target display.

[11] Metadata allows for multiple points at which metadata can be added to video content. To identify whether the metadata is defined by the original content creator, the metadata contains a reference mode flag that can be set to TRUE or FALSE. The reference mode flag is set to TRUE if the metadata was added by the original content creator, thereby indicating that the application of metadata on the target display preserves creative intent. The flag is set to FALSE if the metadata was added by someone other than the original content creator.

[12] According to a first aspect of the present invention, there is provided a method for generating a digital video bitstream from video content, the method including: providing access to a plurality of video frames of the video content; generate content information metadata for video content; and generating a video bitstream based on the video content and the content information metadata, wherein the content information metadata includes: one or more category fields indicating the category of the video content of the digital video bitstream, wherein the one or more category fields cause the target display to select settings a display for displaying video content in a display mode corresponding to the video content category, and a reference mode flag indicating whether one or more category fields should be applied to cause the target display to select display settings for displaying video content in the reference mode, wherein the reference mode is a display mode having certain display settings, the method including setting one or more category fields in the content information metadata according to a category of the video content and setting a reference mode flag in the content information metadata to indicate whether or not the one or more category fields indicate a reference mode , wherein when content information metadata is generated by a video content creator, the reference mode flag is set to the first Boolean value indicating that one or more category fields should be used in reference mode and in non-reference mode, and when content information metadata content are generated by a third party, the reference mode flag is set to a second Boolean value indicating that one or more category fields should be ignored in reference mode and used in non-reference mode.

In a preferred embodiment of the present invention, one or more category fields in the content information metadata comprise at least one of a video content type and a video content subtype.

In another preferred embodiment of the present invention, setting one or more category fields in the content information metadata comprises: receiving Tech 3293 specification metadata of European Broadcasting Union core metadata and applying a mapping from the received Tech 3293 specification metadata of European Broadcasting Union core metadata to one or more category fields .

In another preferred embodiment of the present invention, the content information metadata further includes: one or more adjustment fields indicating desired display setting adjustments for the video content of the target display when the video content is displayed on the target display, wherein the one or more adjustment fields cause the target display to adjust at least part of the selected display settings to display video content in display mode, thereby specifying the applied display mode by adjustable display settings.

In another preferred embodiment of the present invention, one or more adjustment fields in the content information metadata indicate at least one of the following: desired video content white point, desired video content sharpness, desired video content noise reduction, desired MPEG video content noise reduction, desired video content frame rate conversion, average The brightness level of the video content image and the desired color of the video content.

In another preferred embodiment of the present invention, one or more adjustment fields are set in the content information metadata by operations including: receiving Tech 3293 specification metadata of European Broadcasting Union core metadata and applying a mapping from the received Tech 3293 specification metadata of European Broadcasting Union core metadata into one or more adjustment fields

According to a second aspect of the present invention, there is provided a method for reproducing video content on a target display in a reference mode, wherein the reference mode is a display mode having certain display settings, the method comprising: receiving a digital video bitstream containing the video content and metadata with information about the content, wherein this content information metadata includes: one or more category fields indicating the category of the video content of the digital video bitstream, and a reference mode flag indicating whether one or more category fields should be applied to cause the target display to select display settings for displaying the video content in reference mode, decoding video content at the decoder, extracting the reference mode flag from the content information metadata if the reference mode flag is set to the first Boolean value indicating that the content information metadata is generated by the creator of the video content: applying one or more information metadata category fields about content to determine a category of video content, determine a display mode that corresponds to a certain category of video content, apply the display settings of a certain display mode to the target display, and display the video content; if the reference mode flag is set to a second Boolean value indicating that the content information metadata is generated by a third party: display video content without applying content information metadata that includes one or more category fields.

In a preferred embodiment of the present invention, one or more category fields in the content information metadata comprise at least one of a video content type and a video content subtype.

In another preferred embodiment of the present invention, the content information metadata further includes: one or more adjustment fields indicating desired display setting adjustments for the video content of the target display when the video content is displayed on the target display, wherein the one or more adjustment fields cause the target display to adjust at least part of the selected display settings to display video content in display mode, thereby specifying the applied display mode by adjustable display settings.

In another preferred embodiment of the present invention, one or more adjustment fields in the content information metadata indicate at least one of the following: desired video content white point, desired video content sharpness, desired video content noise reduction, desired MPEG video content noise reduction, desired video content frame rate conversion, average The brightness level of the video content image and the desired color of the video content.

According to a third aspect of the present invention, there is provided a computer-readable medium comprising instructions that, when executed by a processor, perform the method of the first aspect of the present invention.

DESCRIPTION OF GRAPHIC MATERIALS

[13] In FIG. 1 depicts an example sequence process for video delivery, showing the various steps from video capture to display of video content.

[14] In FIG. Figure 2 shows an example of using an expanded metadata flag in the form of a table.

[15] In FIG. Figure 3 shows an example of the use of the reference mode (RM) flag L11 in block diagram form.

[16] In FIG. 4A-AD depicts an example of the disclosed metadata details in tabular form.

[17] In FIG. 5 depicts an example of content type details in the exposed metadata in tabular form.

[18] In FIG. 6 shows an example of typical picture mode settings in the disclosed metadata in tabular form.

[19] In FIG. 7 depicts an example of content subtype details in the disclosed metadata in tabular form.

[20] In FIG. 8 depicts an example of the details of the desired white point in the disclosed metadata in tabular form.

[21] In FIG. Figure 9 shows an example of the details of the desired sharpness in the disclosed metadata in tabular form.

[22] In FIG. 10 depicts an example of the details of the desired noise reduction in the disclosed metadata in tabular form.

[23] In FIG. 11 depicts an example of the details of the desired MPEG noise reduction in the disclosed metadata in tabular form.

[24] In FIG. 12 depicts an example of frame rate conversion details in the disclosed metadata in tabular form.

[25] In FIG. 13 shows an example of the details of the average image brightness level in the disclosed metadata in tabular form.

[26] In FIG. 14 shows an example of desired color details in the disclosed metadata in tabular form.

[27] In FIG. 15 shows an example of displaying the broadcast playback server in the form of a table.

DEFINITIONS

[28] Throughout this specification, “reference mode” (RM) is defined as a rendering state of a target display where the target display presents video content in full accordance with the original creative intent. This playback state can be achieved by setting the target display to a specific display mode with associated display settings. The display settings of a specific display mode can be further adjusted to maintain the original creative intent.

[29] Frame rate conversion refers to the conversion between different video frame rates (eg, from 60 frames per second to 120 frames per second, etc.). Frame rate conversion is duplication potentially involving motion estimation and motion compensation or frame reduction to display content on a video device at a different frame rate.

[30] Perceptual Quantization (PQ) is an efficient way to encode high dynamic range (HDR) images. Each successive pair of code values differs by slightly less than a perceptible step across the entire dynamic range, providing very efficient use of the code values. Examples of using PQ encoding and decoding can be found in the ITU-R BT report. 2390, “High dynamic range television for production and international program exchange,” ITU, 2016. Alternatives to using PQ include using traditional gamma and hybrid logarithmic signal parameters. gamma (HLG).

DETAILED DESCRIPTION

[31] This document describes video content type metadata for high dynamic range (HDR). In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, it will be understood that the subject matter of the invention as claimed herein may be practiced without these specific details. In other examples, well-known structures and devices are not described in great detail in order to avoid unnecessary complexity, confusion, and difficulty in understanding the present invention.

[32] In FIG. 1 depicts an example process sequence 100 for video delivery, showing the various steps from capturing video to displaying video content. A sequence of video frames 102 is captured or generated using an image generating unit 105. Video frames 102 may be captured digitally (eg, through a digital video camera) or computer generated (eg, using computer animation) to provide video data 107. Alternatively, video frames 102 may be captured on film by a movie camera, in which case the film is converted to digital format to provide video data 107. In the production software stage, the video data 107 is edited to provide a video production stream 112.

[33] The video data of the video production stream 112 is then provided to the processor in the post production block 115 for post production editing. The post-production editing block 115 may include adjusting or modifying colors or brightness in specific areas of the image to enhance the quality of the image or to achieve a specific visual appearance for the image in accordance with the creative intent of the video creator. This is sometimes called "color setting" or "color correction". Other editing (eg, scene selection and sequencing, cropping images, adding computer-generated visual special effects, etc.) may be performed in post-production block 115 to produce the final production version 117 for distribution. During editing in post production unit 115, video images are viewed on reference display 125.

[34] After post-production in post-production block 115, the video data of the final version 117 may be delivered to encoder 120 for delivery downstream to decoding and playback devices such as televisions, set-top boxes, movie theaters, and the like. In some embodiments, encoder 120 may include audio and video encoders, such as those defined by ATSC, DVB, DVD, Blu-Ray and other delivery formats, to generate a coded bitstream 122. At the receiver, the coded bitstream 122 is decoded by a decoder 130 to generate a decoded signal 132 representing identical video data or a close approximation of the video data that corresponds to the final version 117. The receiver may be connected to the target display 151, which may have completely different characteristics from the reference display 125. In this case, the display control unit 135 may be used to map the dynamic range of the decoded signal 132 to the characteristics of the target display 151 by generating a signal 137 displayed on the display. The target display 151 may be a reference monitor, a computer monitor or display, a television set (TV), a head-mounted display, a virtual retinal display, or the like.

[35] Dolby Vision™ is a complete package that enables the creation and distribution of content created with high dynamic range and wide color gamut. Dolby Vision display control matches the capabilities of this TV by using a number of algorithms to map the signal to any user's Dolby Vision TV. This creates an optimal and smooth video experience for the viewer.

[36] A complete package that enables the creation and distribution of high dynamic range and wide color content, such as Dolby Vision, and includes metadata as well as video data. For example, in Dolby Vision there are different levels of metadata, which are named from level 1 (L1) to level 10 (L10). Examples of descriptions of such metadata are described in WIPO WO 2019/050972 “Top-curve optimization method and associated video” by R. Atkins et al., which is incorporated herein by reference in its entirety. The digital video bitstream and methods for generating a digital video bitstream and reproducing video content according to the present invention provide solutions to the problem of storing creative intent in a decoder or target display, and also provide a solution for identifying a category of video content in a decoder or target display for adjusting display settings according to the identified category of video content .

[37] A new set of metadata is introduced in a complex for the creation and distribution of high dynamic range and wide color gamut video, such as Dolby Vision, referred to herein as Layer 11 (L11) by which content creators intend to regarding the playback characteristics of video content on the decoder or target display. L11 metadata allows the content creator's creative intent to be conveyed to the decoder or target display. This content information metadata indicates the best way for the target display to present the content. L11 metadata includes a variety of fields to provide effortless creator tagging of the content they create.

[38] It should be understood that the digital video bitstream and methods for generating a digital video bitstream and reproducing video content according to the present invention are generally applicable to complexes for creating and distributing high dynamic range and wide color gamut video, and the term "L11" is used contained herein are for illustrative purposes only and are not intended to limit the digital video bitstream and methods for generating a digital video bitstream and playing video content disclosed herein.

[39] Target displays such as televisions have different display modes, but the choice of display mode depends on the user. For example, when a user is watching a feature film, he/she can change the display mode of the TV. However, most users are not experienced enough or don't think it's worth the time to change the display mode every time they view different types of content. Selecting a particular display mode causes the target display to apply display settings that are associated with the selected display mode.

[40] The present invention provides an apparatus and method for automatically identifying a category of video content, whereby a target display can intelligently determine a display mode to be turned on. The present invention provides an apparatus and method for adding information to video content by introducing L11 metadata. In addition, the present invention allows the determination of the best way to present video content on a target display such that the target display does not need to guess the category of the video content, just as the user does not need to guess the category of the video content. Moreover, the subject matter of the invention claimed here allows the user and the target display to not perform any actions while playing video content. The target display simply does what the L11 metadata tells it to do.

[41] Currently, target displays process video content based on guesswork, but this is not a very accurate process. With metadata containing information about the content, the target display is notified to process the video content correctly. This ensures that the decision process in the target display is informed and that the correct result is achieved every time. For example, the target display receives information from the metadata L11 that the content relates to sports, so the target display during playback is switched to a display mode that corresponds to the sports content, so that the associated display settings of the display mode are set accordingly.

[42] According to an embodiment of the present invention, the video bitstream includes metadata that communicates to the target display the category of video content contained in the bitstream. This metadata may include numerous fields that indicate characteristics of the video content, such as content type and content subtype. In addition, the selected category of video content can be refined by directly specifying adjustments to specific display settings, such as desired white point, whether or not to use video in reference mode, desired sharpness, desired noise reduction, desired MPEG noise reduction, desired frame rate conversion (FRC), desired average image brightness level (APL) and desired color. A number of fields in L11 metadata have been added with a specific hierarchy.

[43] L11 metadata is designed to allow creators to effortlessly mark the content they create: 1) The easiest way for a content creator to do nothing, which triggers the default behavior; 2) The second easiest way is for the content creator to tell the target decoder or target display the content category that the video belongs to, such as a feature film, game, or sports. Depending on this metadata information, the target display, such as a TV, makes a high-level determination of the display mode, such as the TV switching to a feature film, game, or sports mode. The TV searches for the appropriate detailed display mode display settings that correspond to these categories respectively; 3) Finally, L11 metadata allows the content creator to further specify details of the display settings on the target display, which lead to refinement of the selected display mode. For example, the content creator can specify a content subtype, for example, in the case of sports content, whether the sport is a winter sport or a water sport. Moreover, the content creator can specify, for example, that video content will look better on the target display with frame rate conversion set to a high value. L11 metadata is hierarchical, allowing the content type or content subtype to be set to a default value if there is insufficient information. Additionally, L11 metadata allows these default values to be replaced with metadata on a case-by-case basis.

[44] Not only can L11 metadata be specified by content creators, it can also be specified at the application level. For example, if the Smart Blu-Ray Player is playing a movie streaming service such as Netflix® (Netflix® is a registered trademark of Netflix Inc.), the Smart Blu-Ray Player may set metadata to the settings for the movies or TV series; If the Smart Blu-Ray Player is playing from a game, it may label the content as a game; if he is playing from a set-top box, he can set it to sports or any other content. Thus, L11 metadata allows the type of video content to be specified by entities other than the original content creator. In this sense, to preserve creative intent, it is important to not just provide metadata about the content, but also provide an indication of how the metadata was created, such as by the content creator or intermediaries.

[45] L11 metadata allows for multiple points at which L11 metadata can be added to video content. In an embodiment, to indicate the source of the L11 metadata, the L11 metadata includes a reference mode flag (also called the L11 reference mode flag or L11 RM flag for short), which can be set to TRUE or FALSE. The original content creator may add L11 metadata for a specific purpose, for example, the content creator has created a game and wants the game to play in a way that makes the game appear bluer and look very sharp on the target display. The content creator can then set the L11 RM flag to TRUE, thereby implying that even if the target display is in the highest quality mode, it should apply content information metadata to preserve creative intent.

[46] On the other hand, if the content relates to, for example, a game, and L11 metadata was added at subsequent stages by a third party, such as during automatic image processing, the L11 RM flag will be set to FALSE, implying that if the target display is in highest quality mode, it should not apply L11 metadata because the L11 metadata was added by someone other than the content creator, so the target display will not preserve the original content creator's intent. Setting the L11 RM flag to FALSE indicates that the content creator's intent has been changed. Using the L11 RM flag in this way ensures that the metadata is immutable, while its use can be modified depending on the state of the flag. For example, if video content is played in reference mode, in which a lot of processing is done to preserve the creator's original intent and the highest quality is important, then during playback the L11 RM flag is used to tell the target display whether or not to follow the reference mode. mode use content information metadata: i) content information metadata should be used in reference mode if the metadata was created by the original content creator (the flag is set to TRUE); ii) Content information metadata should not be used in reference mode if the metadata was created in subsequent stages by someone other than the original content creator (flag set to FALSE). This is one of the key aspects of the present invention. This makes it important whether or not the target display should take content information metadata into account if the viewer intends to view video content in reference mode.

[47] The use of the L11 RM flag is further illustrated in FIG. 2 as a table 200. In this exemplary table, “reference mode metadata usage” (210) refers to the state of the target display during playback. This table further illustrates the scenarios described in the previous paragraph: 1) There is no metadata in the first row of the table (220), so the metadata cannot be used in reference or non-reference mode (230); 2) The second line (240) has L11 metadata, but the L11 RM flag is FALSE, so to preserve creative intent, the target display is instructed not to use L11 metadata in reference mode. The target display can still use L11 metadata during processing when it is in a non-reference mode; 3) In the third line (260), L11 metadata is present and the L11 RM flag is TRUE, so the target display is commanded that it can use L11 metadata in reference mode and all other non-reference modes.

[48] In FIG. 3 depicts an example of the use of the L11 RM flag in the form of a block diagram 300. The decoder or target display first checks whether L11 metadata is present 310 in the video content. If the answer is no, the decoder or target display does not have to do anything else (320). If the answer is yes, then the decoder or target display checks (330) whether the L11 RM flag is TRUE or FALSE. If the flag is FALSE and the user cares about preserving creative intent, then L11 metadata should not (340) be used in reference mode. If the flag is FALSE and the user does not care about preserving the creative intent, then L11 metadata can (360) be used in a non-reference mode of the target display. If the L11 RM flag is TRUE, the decoder or target display may (350) use metadata in both reference mode and non-reference mode.

[49] The following figures with their corresponding tables illustrate example embodiments of L11 metadata fields and example values used to populate these fields.

[50] In FIG. 4A^GO shows an example of L11 metadata details in tabular form. In fig. 4A, table 400A presents L11 metadata having a content type (410) and a content subtype (412). In fig. 4 The 400V table provides bits for the white point (420), for the RM flag (422), and for future use (424). In fig. 4C, table 400C provides bits for sharpening (430), noise reduction (432), MPEG noise reduction (434), and frame rate conversion (436). In fig. 4D, table 400D provides bits for brightness (440), for color (442), and for future use (444).

[51] In FIG. Figure 5 shows an example of additional content type details in table form. Table 500 describes the varieties of content types (510) in L11 metadata. L11 metadata uses a small payload, such as 4 bits, which is updated for each scene. The reserved values (550) are for future content types and will default to "O" for playout devices that do not have this metadata field defined, to keep the current default user experience unchanged.

[52] In FIG. Figure 6 shows an example of typical L11 categories, each of which corresponds to a specific display mode containing a number of display settings, in table form. Table 600 is populated during the perceptual quantizer (PQ) configuration step, with the collaboration of the owner of the content creation and distribution complex, such as Dolby, with the original equipment manufacturer (OEM) of the target displays, and optionally the content creator.

[53] In FIG. 7 shows an example of L11 content subtype details in table form. L11 metadata uses an additional 4 bits that are updated for each scene. The content subtype (720) modifies the default behavior of the content type (510) by further qualifying the content type, which causes the target display to adjust the specific display settings associated with the selected content subtype. This allows the OEM to further fine-tune post-processing algorithms to enable differentiation. The content creator can simply fill the table with the value "0" if he wants.

[54] In FIG. 8 shows an example of the details of the desired metadata white point L11 in tabular form. The table 800 determines the white point of the image to be represented. The white point can either be bypassed through the audio plug-in API (API) system layer, or can be applied as part of Dolby display management (DM), in which case the target display will be calibrated to its own temperature. Setting the default value to "0" will use the default white point for the content type. The desired white point (810) is filtered in the target display during playback to prevent sudden changes.

[55] In FIG. Figure 9 shows an example of the details of the desired sharpness in the L11 metadata in tabular form. Table 900 defines the algorithm for enhancing detail/sharpness on the target display. Setting the default value to "0" will use the default setting for that content type. "OFF" corresponds to no added sharpness. If necessary, the images are resized using bilinear interpolation. “HIGH” corresponds to maximum sharpening. Setting between "OFF" and “HIGH” controls the sharpness. When applying this table, the transition between sharpness settings should be smooth. The desired sharpness (910) is filtered on the target display during playback to prevent sudden changes.

[56] In FIG. 10 shows an example of the details of the desired noise reduction in the L11 metadata in tabular form. Table 1000 defines a noise reduction algorithm for random noise. Setting the default value to "0" will use the default setting for the content type. "OFF" corresponds to no noise reduction. “HIGH” corresponds to maximum noise reduction. Setting between "OFF" and “HIGH” controls the strength of the noise reduction. When using this table, the transition between different settings should be smooth. The desired noise reduction (1010) is filtered on the target display during playback to prevent sudden changes.

[57] In FIG. 11 depicts an example of the details of the desired MPEG noise reduction in L11 metadata in tabular form. Table 1100 defines a denoising algorithm for compression noise. Setting the default value to "0" will use the default setting for the content type. "OFF" corresponds to no noise reduction. “HIGH” corresponds to maximum noise reduction. Setting between "OFF" and “HIGH” controls the strength of the noise reduction. When using this table, the transition between different settings should be smooth. The desired MPEG noise reduction (1110) is filtered on the target display during playback to prevent sudden changes.

[58] In FIG. 12 shows an example of frame rate conversion (FRC) details in L11 metadata in tabular form. Table 1200 defines the frame rate conversion algorithm. Setting the default value to "0" will use the default FRC setting for the content type. "OFF" corresponds to no frame rate conversion. Frames will be duplicated only when necessary. "HIGH" corresponds to maximum frame rate conversion. Frames will be smoothly interpolated to the maximum refresh rate of the target display. Setting between "OFF" and "HIGH" controls the FRC strength. Frames will be interpolated by a fractional amount between the origin and the midpoint. When applying this table, the transition between different FRC settings should be smooth (i.e. no black frames). As a consequence, full FRC bypass cannot be caused by L11, as it usually results in black frames. There are mechanisms for requesting FRC, such as Auto Low Latency Mode (ALLM) in HDMI 2.1. The desired frame rate conversion (1210) is filtered on the target display during playback to prevent sudden changes.

[59] In FIG. 13 shows an example of average image luminance level (APL) details for L11 metadata in tabular form. Table 1300 determines the average brightness level of the image. Setting the default value to "0" will use the default setting for the content type. "LOW" corresponds to a slightly darker brightness. "MEDIUM" corresponds to medium brightness. “HIGH” corresponds to maximum brightness. When using this table, the transition between different settings should be smooth. The desired average level (1310) of image brightness is filtered on the target display during playback to prevent sudden changes.

[60] In FIG. 14 shows an example of the desired color details in the L11 metadata in tabular form. Table 1400 determines color saturation. Setting the default value to "0" will use the default setting for the content type. "LOW" corresponds to slightly lower saturation. “MEDIUM” corresponds to medium saturation. “HIGH” corresponds to maximum saturation. When using this table, the transition between different settings should be smooth. The desired color saturation (1410) is filtered on the target display during playback to prevent sudden changes.

[61] Default settings in L11 metadata are now further described. For example, when the target display is in a smart mode such as Dolby Smart Mode, video content without L11 metadata may default to "DEFAULT" (eg, in FIG. 5, content type =0). “Everything else” (e.g., in Fig. 5, content type =6) can be considered as the “average best setting” for all content (e.g., sports, drama, TV series, commercials, games, and feature films). During post-production (115), the studio can check the quality of the released product in different display modes (eg, live, cinema/feature film, etc.). If a studio sets its playback display (125) to "home theater" mode to view its content, it may not necessarily provide the same viewing experience as someone using smart mode with the L11s set to "cinema." The same is true for Game, Sports and Standard modes. This may occur because the two displays may differ in how they determine their white point or other calibration parameters.

[62] The generation of L11 metadata is now described. For example, if L11 is not specified in the content, imaging devices may fill in the fields to configure the target display device to the desired mode. Additionally, if L11 is not specified in the content, fields for configuring the target display device may be filled in at the application level (app). For example, in an embodiment, the PS4/Xbox gaming applications are set to game, the Blu-Ray player and Netflix applications are set to cinema, or the NBC sports application is set to sports. Moreover, the authoring toolkit can also fill in fields to configure the target display device. In this case, the authoring toolkit allows and activates the drop-down menu for the content type. Hollywood studios are always in the mood for cinema. Broadcasters are set on "Everything Else" unless they explicitly cover sports or cinema.

[63] One display example of a broadcast playback server is provided in FIG. 15 in the form of a table. Table 1500 depicts the generation of L11 metadata from the European Broadcasting Union (EBU) core metadata specification Tech 3293 (1410) by mapping the L11 metadata from the Tech 3293 EBU core metadata specification (see Tech 3293, "EBU Core metadata set (EBUCore) ", specification v1.9, MIM-AI, EBU, January 2019).

[64] An implementation of L11 playback on a target display is now described. For example, the user selects a smart mode on the target display, such as Dolby Smart Mode™, which may also be the factory default. When you change the content type, the target display remains in smart mode. Metadata changes occur smoothly, meaning there is no flickering of image content. The user can switch to legacy image modes at any time.

[65] The following descriptions of additional embodiments will focus on the differences between them and the previously described embodiment. Accordingly, features that are common to both embodiments will be omitted in the following description, and it is therefore assumed that the features of the previously described embodiment are implemented or at least can be implemented in the additional embodiment unless the following description requires otherwise.

[66] In another embodiment of the present invention, the RM flag is used in more situations, not just during playback. For example, if the metadata does not represent creative intent, the apparatus and methods of this embodiment allow someone downstream (encoder, distributor, playback device) after the creator to modify, create, and/or rewrite the metadata. However, if the metadata represents creative intent, the metadata is not permitted to be modified at subsequent stages.

[67] In an alternative embodiment of the present invention, more information about who completed the metadata is included. The apparatus and methods of this embodiment allow more bits to be used for the RM flag field. For example, information such as the stage of content creation and distribution, as well as the name of the company that populated the L11 metadata, can be stored in the RM flag. This information can be attached to content and metadata. This information can be used to understand how to interpret and apply metadata. Some examples are: i) the application or non-application of metadata in the reproducing device depending on the identification of who filled out the metadata and the reproducing priorities; ii) allowing, or not allowing, various distribution participants, including, for example, Dolby and its partners, to further modify, enhance, or rewrite metadata at various stages of content creation and distribution, depending on who completed it; iii) the disclosure or non-disclosure of some metadata depending on who completed it and where the metadata is potentially disclosed (for example, disclosure of the content type in the consumer's visual graphical user interface (GUI).

[68] In yet another alternative embodiment of the present invention, in a content format that is conceptually similar to a blockchain, participants in subsequent stages may attach additional and/or alternative L11 metadata sets or records. In this case, it is critical to know who completed each set or entry. For example, a content service may have its preferred view defined by one consistent set of L11 metadata for all of its content, while a different set of L11 metadata will be maintained for each piece of content, as defined by the original content creator. This will allow for a different consumer experience depending on which set of L11 metadata is used. The extended RM flag for each L11 metadata set will convey decision information regarding which L11 metadata set to use.

[69] A number of embodiments of the present invention have been described. However, it will be understood that various modifications may be made without departing from the scope of the present invention. Accordingly, other embodiments of the invention are within the scope of the following claims.

[70] The above examples are provided to those skilled in the art as a complete disclosure and description of how to make and use embodiments of the present invention, and are not intended to limit the scope of what the inventor/s consider their invention to be.

[71] It is intended that modifications to the above-described embodiments of the methods and systems disclosed herein that will be apparent to those skilled in the art are within the scope of the following claims. All patents and publications mentioned in the description indicate the levels of skill of specialists in the field to which the present invention relates. All references cited herein are incorporated by reference herein to the same extent as if each reference were incorporated separately herein in its entirety.

[72] It should be understood that the present invention is not limited to specific methods or systems, which, of course, may vary. It should also be understood that the terminology used herein is not intended to describe only specific embodiments and is not intended to be limiting. As used in this specification and in the accompanying claims, the singular forms of a noun include the entities referred to and the plural, unless the content clearly states otherwise. The term “multiple” includes two or more referenced entities unless the content expressly states otherwise. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention relates.

[73] The methods and systems described in the present invention may be implemented as hardware, software, firmware, or a combination thereof. Features described as blocks, modules or components may be implemented together or separately using a combination of hardware, software and firmware. The software portion of the methods of the present invention may include a computer-readable medium that contains instructions (eg, an executable program) that, when executed, at least partially perform the described methods. A computer-readable medium may include, for example, random access memory (RAM) and/or read-only memory (ROM). The instructions may be executed by a processor (eg, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA)).

[74] Various aspects of the present invention can be understood from the following numbered exemplary embodiments (NEX):

VPVO 1. A method of generating a digital video bitstream with metadata, wherein the metadata includes a flag indicating preservation of the creative intent, the method comprising:

setting one or more additional fields in the metadata to indicate the content type of the digital video bitstream, and

setting a flag in the metadata to indicate whether or not one or more additional fields need to be used when the digital video bitstream is decoded in a reference mode in which the digital video bitstream is to be played and/or displayed according to the original creative intent.

VPVO 2. The method according to VPOP 1, wherein one or more additional fields in the metadata comprise at least one of the following: a video content type, a video content subtype, a desired video content white point, a desired video content sharpness, a desired video content noise reduction, a desired MPEG video content noise reduction, a desired conversion of the frame rate of the video content, the average brightness level of the image of the video content, and the desired color of the video content.

VPVO 3. Method according to VPVO 2, in which if the metadata is generated by the creator of the video content, the flag is set to the first Boolean value, and if the metadata is generated by a third party, then the flag is set to the second Boolean value.

VPVO 4. Method according to VPVO 2, in the engine metadata is generated by the video content creator, automatic image processing device or at the application level.

VPVO 5. A method as in any one of VPVO 1^4, wherein the metadata is generated by a video content creator, an automated image processing engine, or at the application level.

VPVO 6. The method as in any one of VPOP 15, wherein if the metadata is generated by the creator of the video content, the flag is set to the first Boolean value, and if the metadata is generated by a third party, then the flag is set to the second Boolean value.

DPVO 7. A method according to any one of DPVO 1-6, in which a mapping from other metadata is used to generate metadata.

DPVO 8. A method according to DPVO 7, in which the other metadata is the Tech 3293 specification of the European Broadcasting Union core metadata.

VPVO 9. A method of reproducing video content with metadata, wherein the metadata includes a flag indicating preservation of creative intent, the method comprising:

receiving video content on the decoder, and

providing to the decoder, based on the state of the flag, commands as to whether or not one or more additional fields are to be used when the video content is decoded in a reference mode in which the digital video bitstream is to be played and/or displayed according to the original creative intent.

VPVO 10. The method according to VPVO 9, in which, if the flag is set to the first logical value, then the decoder is commanded to use metadata to determine the type of video content in the reference mode and in the non-reference mode.

PPVO 11. Method according to PPVO 9, in which if the flag is set to the second logical value, then the decoder is commanded to ignore metadata in the reference mode and use metadata in a non-reference mode.

VPVO 12. The method according to any one of VPVO 9 to 11, wherein the metadata contains at least one of the following: a type of video content, a subtype of video content, a desired white point of the video content, a desired sharpness of the video content, a desired noise reduction of the video content, a desired MPEG noise reduction of the video content, a desired frame rate conversion video content, the average image brightness level of the video content, and the desired color of the video content.

PPVO 13. Digital video bitstream containing:

video content and

metadata including a flag indicating creative intent, wherein the metadata has one or more additional fields to indicate the content type of the digital video bitstream,

wherein

a flag in the metadata indicates whether or not one or more additional fields need to be used when the digital video bitstream is decoded in a reference mode in which the digital video bitstream is to be played and/or displayed according to the original creative intent.

VPVO 14. The digital video bitstream of VPVO 13, wherein one or more additional fields in the metadata comprise at least one of the following: video content type, video content subtype, desired video content white point, desired video content sharpness, desired video content noise reduction, desired MPEG noise reduction video content, the desired frame rate conversion of the video content, the average image brightness level of the video content, and the desired color of the video content.

VPVO 15. A method of indicating the use or non-use of metadata in a video bitstream, including:

provision of a digital video bitstream via PPVO 13 or PPVO 14;

setting the flag to the first boolean value if the metadata is generated by the video content creator, and to the second boolean value if the metadata is generated by a third party.

PPVO 16. A method for reproducing a video bitstream with extended dynamic range, including:

provision of digital video bitstream via PPVO 13 or PPVO 14 and

sending commands to the video decoder to use metadata to determine the type of video content in the reference mode and in a non-reference mode if the flag is set to the first logical value.

PPVO 17. A method for reproducing a video bitstream with extended dynamic range, including:

provision of digital video bitstream via PPVO 13 or PPVO 14 and

sending commands to the video decoder to ignore metadata in the reference mode and use metadata in a non-reference mode if the flag is set to the second logical value.

PPVO 18. A machine-readable medium containing commands that, when executed, perform steps of methods according to any of PPVO 1-12 or PPVO 15-17.

Claims (40)

1. A method for generating a digital video bitstream from video content, the method comprising: providing access to multiple video frames of video content; generate content information metadata for video content; And generating a video bitstream based on the video content and content information metadata, wherein This content information metadata includes: one or more category fields indicating a category of video content of the digital video bitstream, wherein the one or more category fields cause the target display to select display settings for displaying the video content in a display mode corresponding to the category of the video content, and a reference mode flag indicating whether one or more category fields should be applied to cause the target display to select display settings for displaying video content in the reference mode, wherein the reference mode is a display mode having certain display settings, the method including: setting one or more category fields in the content information metadata according to the category of the video content, and setting a reference mode flag in the content information metadata to indicate whether or not one or more of the category fields indicate reference mode, wherein, when the content information metadata is generated by the video content creator, the reference mode flag is set to the first Boolean value indicating that one or more category fields should be used in reference mode and in non-reference mode, and when the content information metadata generated by a third party, the reference mode flag is set to a second Boolean value indicating that one or more category fields should be ignored in reference mode and used in non-reference mode. 2. The method of claim 1, wherein the one or more category fields in the content information metadata comprise at least one of a video content type and a video content subtype. 3. The method according to claim 1, characterized in that setting one or more category fields in the content information metadata includes: acceptance of Tech 3293 specification metadata of the European Broadcasting Union core metadata and applying a mapping from accepted Tech 3293 specification metadata to European Broadcasting Union core metadata into one or more category fields. 4. Method according to any one of paragraphs. 1-3, characterized in that the metadata with information about the content additionally includes: one or more adjustment fields indicating desired display setting adjustments for the video content of the target display when the video content is displayed on the target display, wherein the one or more adjustment fields cause the target display to adjust at least a portion of the selected display settings to display video content in the display mode, thereby specifying the applicable display mode by the adjustable display settings. 5. The method of claim 4, wherein the one or more adjustment fields in the content information metadata indicate at least one of the following: a desired white point of the video content, a desired sharpness of the video content, a desired noise reduction of the video content, a desired MPEG noise reduction of the video content, a desired conversion of the frame rate of the video content, the average brightness level of the image of the video content, and the desired color of the video content. 6. The method of claim 4, wherein one or more adjustment fields are set in the content information metadata through operations including: acceptance of Tech 3293 specification metadata of the European Broadcasting Union core metadata and applying a mapping from accepted Tech 3293 specification metadata to European Broadcasting Union core metadata into one or more adjustment fields. 7. A method for playing video content on a target display in a reference mode, wherein the reference mode is a display mode having certain display settings, the method including: receiving a digital video bitstream containing video content and content information metadata, wherein the content information metadata includes: one or more category fields indicating the category of the video content of the digital video bitstream, and a reference mode flag indicating whether one or more category fields should be applied to cause the target display to select display settings for displaying video content in reference mode, decoding video content on the decoder, extracting the reference mode flag from content information metadata, if the reference mode flag is set to the first boolean value indicating that the content information metadata is generated by the video content creator: applying one or more content information metadata category fields to determine the category of video content, determining the display mode that corresponds to a specific category of video content, applying the display settings of a specific display mode to the target display, and display of video content; if the reference mode flag is set to a second boolean value indicating that the content information metadata is generated by a third party: Display video content without applying content information metadata that includes one or more category fields. 8. The method of claim 7, wherein the one or more category fields in the content information metadata comprise at least one of a video content type and a video content subtype. 9. The method according to claim 7 or 8, characterized in that the metadata with information about the content additionally includes: one or more adjustment fields indicating desired display setting adjustments for the video content of the target display when the video content is displayed on the target display, wherein the one or more adjustment fields cause the target display to adjust at least a portion of the selected display settings to display video content in the display mode, thereby specifying the applicable display mode by the adjustable display settings. 10. The method of claim 9, wherein the one or more adjustment fields in the content information metadata indicate at least one of the following: a desired white point of the video content, a desired sharpness of the video content, a desired noise reduction of the video content, a desired MPEG noise reduction of the video content, a desired conversion of the frame rate of the video content, the average brightness level of the image of the video content, and the desired color of the video content. 11. A computer-readable medium containing instructions that, when executed by a processor, perform the method according to any one of claims. 1-6.