JP2015165704A - Encoding, distributing and displaying video data containing customized video content versions - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method and an apparatus for processing video data containing two or more different versions of video content.SOLUTION: A display apparatus has: an input configured to receive video data and an input configured to receive ambient conditions for viewing the video data; and a processor configured to decode a base layer and at least first and second enhancement layers from the received video data, interpolate between at least two of the enhancement layers on the basis of the received ambient conditions, and generate a video data stream including a corrected base layer in accordance with the interpolated enhancement layers.

Description

  The present invention relates to video data, and more particularly to a method and apparatus for distributing video data. Embodiments of the present invention provide a method for distributing video data; a video data format; an apparatus for distributing video data; an apparatus for receiving video data; and an apparatus for processing video data.

[Cross-reference to related applications]
This application claims priority from US Provisional Patent Application No. 61 / 355,131, filed Jun. 15, 2010. This document is hereby incorporated by reference in its entirety.
[Background technology]
The display includes a television, a computer monitor, a home cinema display, a digital cinema display, a dedicated display such as a tablet computer, a cellular phone, a digital camera, a copy machine, an industrial control device, a medical imaging, a virtual Includes special displays such as reality and vehicle simulation. A color image designated by image data can be displayed using a color display.

  Various display platform technologies can be incorporated into the display. For example, the display may be: a cathode ray tube (CRT) display; a backlit liquid crystal display (LCD); a plasma display; an organic LED display (OLED display); a laser projector; a digital mirror device (DMD) display And electroluminescent displays. Any of these common techniques can use light emitting and / or filtering elements with a wide range of configurations and compositions. The functions of the following areas may vary greatly depending on the display: range of colors that can be displayed (color gamut); maximum brightness; contrast ratio; resolution; color depth; dynamic range; white level, white point, black point, gray Number of steps etc.

  For this reason, if the display to be played is different, the same video content may appear differently. Depending on the display, the video content displayed on the display without modification may be in one or more ways different from the creator's intention to create.

  Some current displays surpass the display that was state of the art when video content was created. For example, some displays can provide images with brighter highlights, larger contrast ratios, and / or wider color gamuts than legacy displays that were available when video production was created. It would be desirable to take advantage of these improved features without significantly departing from the creative intent embodied in the video content.

  You may want to display video content created to use a high-performance display on a low-performance legacy display. It would be desirable to provide a method and apparatus configured to play a video so as to preserve the creative intent embodied in the video content as much as possible.

  The perception of color and brightness is affected by ambient conditions. Video displayed under cinema conditions may be perceived quite differently when viewed in bright ambient light. Furthermore, the characteristics of ambient light (such as color temperature) affect the viewer's perception of video content. It is desirable to adapt the playback of the video content in consideration of the environment in which the video content is viewed so as to keep the creative intention embodied in the video content as much as possible.

  There is a need to provide viewers with a viewing experience that fully exploits the capabilities of the display to view the image (including still and / or video images). There is a need for an apparatus and method that can be applied to the adjustment of a video signal so that video and other image content encoded in the signal will appear as desired during playback.

  The present invention provides a method and apparatus for processing video data that includes two or more different versions of video content. Different versions are optimized for display on different displays and / or for viewing under different ambient conditions.

  One aspect of the present invention provides a method for distributing video data comprising the step of packaging together multiple versions of the same video content. Another aspect of the present invention provides a method of delivering video content optimized for display, the method comprising generating optimized video content from different versions of video content To do. Optimized video content can be generated, for example, by interpolation or extrapolation between two different versions of video content. Another aspect of the present invention provides a data processing stage configured to generate video content optimized for display on a target display from different versions of the video content.

  Another aspect of the present invention is a media server having a data store including multiple versions of the same video content, and a version of the video content version optimized for display on a target display And generating one or both of the optimized video content from different versions of the video content and generating optimized video content for display on a target display, Provide a media server.

  Another aspect provides a method of delivering video content optimized for display on a target display. The method includes providing video data having at least first and second versions of the video content and interpolating or extrapolating between the first and second versions of the video content to provide a third video content. Generating a version of

  Another aspect provides a method of delivering video content optimized for display on a target display. The method includes providing video data including different versions of the video content in a data entity, wherein the different versions have different color gamuts, different frame rates, different bit depths, and different spatial resolutions. Including a version having one or more of the following: The method determines the function of the target display and selects and decodes one of the different versions for display on the target display.

  Another aspect provides a video processing apparatus having a data processing stage. The data processing stage is configured to read video data encoded with different versions of video content. Different versions include versions having one or more of different color gamuts, different frame rates, and different bit depths. The apparatus determines whether one of the versions is optimized for display on the target display, and in response to determining that one version of the version is optimized for display on the target display, A version of the version is configured to be provided for display on the target display.

  Another aspect provides a video processing apparatus. The apparatus includes a decoder configured to extract a base layer and one or more enhancement layers from a video signal and generate a plurality of input signals each encoding a different version of video content; and the plurality of inputs And an interpolator connected to output an output signal that is a different version of the interpolation corresponding to the input signal. Another aspect is a plurality of different video content versions having different color gamuts, different frame rates, different bit depths, or for viewing under different ambient light levels A persistent medium having computer readable video data encoded with different versions optimized for the present invention is provided.

  Another aspect provides a color grading station. The color grading station includes a data storage unit including video content, a plurality of displays, and a controller. The controller displays the video content on each display of the plurality of displays and receives corresponding user input including optimization of the video content for display on each display of the plurality of displays, each of the plurality of the plurality of displays. A controller for storing a plurality of versions of the video content including optimization related to one display of the display and combining the plurality of versions into computer readable video data encoding the different versions of the video content; Have

  Additional aspects of the invention and features of specific embodiments of the invention are described below and / or illustrated in the drawings.

The accompanying drawings illustrate non-limiting embodiments of the invention.
It is a block diagram which shows a video data delivery system. FIG. 6 is a block diagram illustrating a user location including a plurality of different display devices. It is a figure which shows a video production system. FIG. 4 is a diagram illustrating a data processing stage that performs interpolation to provide an optimal video signal. 5 is a flowchart illustrating a method for providing an optimal video signal by interpolation. It is a figure which shows an example of the interpolation method. It is a figure which shows the video delivery system by embodiment. It is a figure which shows the video delivery system by other embodiment.

  In the following description, specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention can be practiced without these specific matters. In other instances, well-known elements have not been shown or described in detail so as not to unnecessarily obscure the present invention. Accordingly, the specification and drawings are to be regarded as illustrative and not restrictive.

One aspect of the present invention relates to distribution of video data. The format provides a base layer and a plurality of enhancement layers. Each enhancement layer provides, for example, the following (s):
-Wide color gamut;
-Optimized for different ambient conditions (e.g. if the base layer is optimized for viewing with ambient light of 100 lux or less, it is optimized for viewing with ambient light of 1000 lux or other illuminance) A single enhancement layer is provided);
-High frame rate;
-Large dynamic range;
-Large bit depth;
-3D information (such as one or more additional viewpoints);
-Large spatial resolution.
In some embodiments, the base layer is a Rec. It corresponds to low function video signals such as 709 compliant video signals. In some embodiments, the enhancement layer is applied to high performance video signals, such as video signals in a format that includes a range of colors sufficient to specify a color gamut that includes all colors perceivable by the human visual system. Correspond.

  Different enhancement layers may be provided to meet the needs of various viewers. FIG. 1 illustrates an embodiment in which video data 10 is carried by a transmission network 14 from a video source 12 to a plurality of user locations 15. In this embodiment, the video data 10 is in a format that includes a base layer 20 and first and second enhancement layers 22A and 22B.

  The transmission network 14 includes, for example, the Internet, a cable television system, a wireless broadcasting system, a distributed medium such as a DVD, other data storage devices including video data, and other video data 10 in a user location. 15 and any data communication system that can be distributed.

  There are various displays 16 at the user location 15. In the illustrated embodiment, user location 15A has a display 16A, user location 15B has displays 16B-1 and 16B-2, and user location 15C has a display 16C.

  The base layer 20 is compatible with the least common multiple display. For example, the base layer may be an ITU-R recommendation BT. (Also known as Rec. 709). 709 or a format specified by other widely accepted formats, or can be easily converted to that format.

  In this embodiment, the first enhancement layer 22A has an extended color gamut suitable for display on a display that can reproduce colors outside the base layer color gamut, and is optimized for display under dim ambient conditions. Provide a video data version. The second enhancement layer 22B is similar to the enhancement layer 22A, but is optimized for display under bright peripheral conditions.

  Video data 10 passes through a data processing stage 18 before being displayed. As shown in the illustrated embodiment, the data processing stage 18 may be in the display 16 (as in the display 16A) or in the user location 15 (as in the user location 15B). It may be in the upstream device 19 or in the upstream device 21 outside the user location (as in the user location 15C). The upstream device 21 may be, for example, in the transmission network 14 or in the video source 12.

  By way of non-limiting example, upstream device 19 includes a transcoder, cable modem, set top box, and the like. The upstream device 21 includes a transcoder, a video distribution system head end, a video server, and the like.

  The data processing stage 18 extracts from the video data 10 the data necessary to generate a video stream 23 that is optimized for display on a specific display 16. In some embodiments, the data processing stage 18 retrieves the selected video stream 23 for the ambient state where the display 16 is viewed. In some embodiments, a light sensor 26 associated with the display 16 detects ambient light (intensity and / or color) and sends ambient light information 27 about the ambient light to the data processing stage 18. The data processing stage 18 uses the ambient light information 27 to set itself so as to distribute an appropriate video stream 23 to the display 16.

  In other embodiments, the data processing stage 18 configures itself to access previously stored account information or user preference information (not shown) and deliver the appropriate video stream 23 to the display 16. To do. Previously stored information indicates using a display, for example, in a dark theater. In other embodiments, the data processing stage 18 receives user input regarding ambient conditions at the display location and uses the user input to configure itself to deliver the appropriate video stream 23 to the display 16. User input is received, for example, by a computer, display, remote controller, portable digital device, smart phone, etc., and sent to the data processing stage 18 by any suitable data communication path.

  Data processing stage 18 includes, for example, a programmable processor executing suitable software and / or firmware, configurable hardware such as a suitably configured field programmable gate array (FPGA), hardware logic circuitry (eg, integrated circuit). Or any suitable combination thereof. The data processing stage 18 is accommodated in a video processing device such as a computer, a transcoder, a set top box, a digital video recorder, or the like, or in a display such as a television, a home theater display, a digital camera projector, a computer video system, etc. Or it may be accommodated in a component of a data transmission network such as a cable television head end, a network access point, or the like. Embodiments provide an apparatus containing a data processing stage 18 described herein.

  In some embodiments, the data processing stage 18 decodes the video data 10 as necessary, and one or more base layers 20 appropriate to the intended intended destination display 16, and One or more enhancement layers 22 are selected. In other embodiments, the data processing stage 18 generates video data based on, but different from, the base layer 20 and the enhancement layer 22. The data processing stage 18 distributes the new data as a video stream 23 or stores the new data as a file for later display on the destination display 16.

  Video data 10 may be streamed as shown in FIG. 1 or may have a file structure. As shown in FIG. 2, the user can hold a file 31 containing video data 10 in a data storage unit 30 that can be accessed by various viewing devices. For example, the user has a smartphone 32A, a tablet computer 32B, a regular television 32C, and a high-end home theater system 32D, all of which can display video content. Each of these devices has a different function for displaying video.

The data processing stage 18 prepares the video content of the video data 10 to be played on the selected device 32 by selecting the appropriate base layer 20 and / or enhancement layer 22 for one of the devices 32, and the device A video file 24 or a stream 23 is generated for playback on the computer. Generation of the file 24 and / or stream 23 is performed by the data processing stage 18 at the following times:
In advance (e.g., when the user downloads or acquires the video data 10, the data processing stage 18 can automatically generate and store a file 24 that is optimal for display on the user's device).
-When video data is downloaded for playback on a device.
When a device is synchronized to a media collection containing video data 10;
-When video data is being streamed to a device.

  As shown in FIG. 3, the appearance of video content to a user by providing video data 10 that includes a version optimized for displaying the video content on different displays and / or under different viewing conditions. It is easier to maintain creative control. A video producer may have a creative intent intended to convey to the user due to factors such as the palette of colors used in a scene, the contrast of the scene, the maximum brightness of the scene, and the like. These and similar factors affect how the scene is perceived by the viewer. The creative intent depends on the content and the producer and is very difficult to judge analytically.

  For example, it is difficult or impossible to program a computer to answer the following question: “A certain producer of video content has been given that video content under specified ambient viewing conditions. How was the content different if it was intended to be viewed on a functional display? As shown in FIG. 3, the video data 10 includes multiple versions of the video content. The producer P may optimize and approve each of the multiple versions separately.

  In the illustrated embodiment, producer P has access to multiple reference displays 26. The reference display 26A is a standard reference display viewed in a dark environment. The producer views the video content of the base layer 20 on the display 26A and uses the color grading station 28 to optimize the appearance of the video content on the display 26A according to the producer's creative intention. The color grading station 28 receives high-quality video data 29 as input. The video data 29 has, for example, a high dynamic range, a wide color gamut, and a large bit depth.

  In the illustrated embodiment, the producer can also access a display 26B with a wide color gamut and a wide dynamic range. The producer looks at the enhancement layer 22A video content on the display 26B under dark ambient conditions and uses the color grading station 28 to determine how the enhancement layer 22A video content appears on the display 26B, depending on the producer's creative intent. Optimize.

  The producer also looks at the enhancement layer 22B video content on the display 26B under bright ambient conditions and uses the color grading station 28 to view the enhancement layer 22B video content on the display 26B according to the producer's creative intentions. Optimize the way. The color grading station 28 has automated or semi-automated tools that assist producers in efficiently generating multiple versions of video content for display on different displays under different ambient conditions.

  When the producer approves the base layer and enhancement layer video content versions, the encoder 25 writes the approved video content version to the video data 10.

  In this way, the video data 10 includes different versions of the same video content that are optimized for use on different displays and / or different viewing conditions, depending on the creative intent of the producer P. As a result, the user can enjoy the video content in a manner that almost matches the producer's creation intention.

  If the base layer or enhancement layer of the video data 10 has already been optimized in a manner suitable for the destination display, the video signal based on that layer is sent to the display for display. In some embodiments, the data processing stage 18 is configured so that the display function and peripheral conditions to be displayed are the functions and peripheral conditions optimized for one of the base layer 20 and the enhancement layer 22 (strictly or within a threshold range). It is configured to determine whether they match. If they match, the video signal obtained from the matching layer is output without correction.

  Video data 10 need not include multiple enhancement layers 22 optimized for any display in any viewing condition. As described above, the data processing stage 18 can generate video data for a display that is based on, but different from, the base layer and the enhancement layer. In some embodiments, this includes interpolation and / or extrapolation between the base layer and the enhancement layer, and / or interpolation and / or extrapolation between the two enhancement layers. Interpolation or extrapolation is based on the function of the target display relative to the functions associated with the two layers.

Data processing stage 18 receives in some way information about the capabilities of the target display and / or the ambient lighting conditions at the target display. For example:
The data processing stage 18 may be hardwired or hardcoded for a certain target display function. This is most appropriate for embodiments where, for example, the data processing stage 18 is integrated into the display and / or is dedicated to a display.
The data processing stage 18 includes or is connected to a memory, a register set, etc. that contains information describing the function of a target display. This is most appropriate, for example, in embodiments where the data processing stage 18 is incorporated into a set top box or other device that is dedicated to one or more displays.
The data processing stage 18 can communicate with the target display via a communication path, which also carries the video signal to the target display.
The data processing stage 18 is in data communication with each other via the Internet or other data communication path and can exchange information including information regarding the function of the target display.
-The target display is associated with one device, even in one device that is docked to the server and otherwise connected periodically, and can receive data uploads. While connected to the server, the device can transfer information regarding the capabilities of the target display to the server or data processing stage 18.

  The data processing stage 18 obtains a video stream 23 or a file 24 from video streams corresponding to two or more base layers 20 and enhancement layers 22 by interpolation and / or extrapolation. If all versions of the video content of the video data 10 are individually optimized to preserve the creative intent (eg, by suitable color timing), the values that are interpolated or extrapolated are those that preserve the creative intent. It is expected to be.

  FIG. 4A is a block diagram illustrating an example component of data processing stage 18 configured to generate optimized video data based on two or more versions of video content from video data 10. FIG. 4B is a flowchart illustrating a method 70 for generating optimized video data based on two or more input versions of video content. The data processing stage 18 includes a decoder 45 that extracts the base layer 20 and / or the enhancement layer 22 from the video data 10. In the illustrated embodiment, the video data 10 includes a base layer 20 and two enhancement layers 22A, 22B. In some embodiments, the decoder 45 extracts only the base layer 20 and enhancement layer 22 that are used to generate video content optimized for a display. The decoder 45 supplies two input signals 47 A and 47 B to the interpolation / extrapolation block 48.

  Each input signal 47 includes a pixel value. The pixel value has, for example, a color coordinate and / or a color coordinate value in the color space. In some embodiments, the color space is a perceptually linear color space.

  The interpolation / extrapolation block 48 is configured to generate an output value 49 by interpolating or extrapolating between corresponding pixel values of the input signal 47 (step 73). Interpolation / extrapolation block 48 provides output value 49 as an output 50 with a modified video signal (step 74). The control input 52 of the interpolation / extrapolation block 48 receives control values that control the operation of the interpolation / extrapolation block 48. In one simple embodiment, in the interpolation / extrapolation block 48, the control value is multiplied by the difference between the corresponding values of the first and second video signals 47A and 47B. The interpolation / extrapolation performed by block 48 is based on the value received at control input 52. The value provided to the control input 52 is determined based on a measure of the function of the display to be displayed and / or based on a measure of ambient conditions in the display to be displayed.

により決定される。（Ｃ−Ａ）は正または負である。｜Ｃ−Ａ｜は｜Ｂ−Ａ｜より大きくても小さくてもよい。 FIG. 4C illustrates one method for performing interpolation / extrapolation. In Figure 4C, the value of the first video signal is y _1, the corresponding value of the second video signal is y _2. The horizontal axis of FIG. 4C shows the scale of the display function or ambient conditions to display. For example, the horizontal axis indicates the dynamic range of the display to be displayed, or the logarithm of the maximum illuminance of the display to be displayed according to some index. The first video signal 47A includes a version of video content that is optimized for display by a display having function A, and the second video signal 47B is a video content that is optimized for display by a display having function B. Includes version. When the display that displays the video signal (destination display) has the function C, the output value is, for example,

Determined by. (CA) is positive or negative. | C-A | may be larger or smaller than | B-A |.

According to equation (1), if y ₁ = y ₂ then V _OUT is equal to y ₁ and y ₂ . Thus, if the creator intentionally sets y ₁ and y ₂ equal in the first and second versions of the video content represented by the first and second video signals, the implementation using equation (1) In form, and in other embodiments having the same characteristics, this value is maintained in the output video signal. On the other hand, when y ₁ ≠ y ₂ , V _OUT is different from y ₁ and y ₂ .

  In some embodiments, the data processing stage 18 is configured to interpolate / extrapolate along the perceptual curve. For example, the data processing stage 18 may be configured to perform interpolation / extrapolation with the logarithm of illuminance. In some embodiments, the video data is given or converted into a perceptually uniform color space (eg, IPT or LogYuv) before interpolation / extrapolation.

により行われる。ここで、ＹはＬｏｇＹｘｙやＬｏｇＹｕｖ鋳などの色空間における照度座標であり、（Ｙ_ＶＤＲなどの）添字ＶＤＲはより高い機能（例えばより高いダイナミックレンジ及び／又はより広い色域）を有するビデオ信号からのデータを識別し、添字ＬＤＲはより低い機能を有するビデオ信号からのデータを識別し、添字ＭＤＲは出力又は目標ビデオ信号を識別し、αとβは補間定数であり、ｘとｙは色座標である（例えば、ＬｏｇＹｘｙ色空間中のｘとｙや、ＬｏｇＹｕ’ｖ’色空間中のｕ’とｖ’）。ここで、αは例えば

により与えられる。ここで、ｍａｘ（Ｙ_ＶＤＲ）は（高機能の）ＶＤＲビデオ信号で指定できるＹの最大値であり、ｍａｘ（Ｙ_ＬＤＲ）は（低機能の）ＬＤＲビデオ信号で指定できるＹの最大値であり、ｍａｘ（Ｙ_{ＴＡＲＧＥＴ}）は目標ディスプレイで再生できるＹの最大値であり、βはβ＝１−αにより与えられる。 In some embodiments, the interpolation between video data is

Is done. Here, Y is luminance coordinates in color space such as cast LogYxy and _LogYuv, from a video signal having a _{(Y VDR} like) subscript VDR higher functionality (e.g., higher dynamic range and / or wider gamut) The subscript LDR identifies the data from the lower function video signal, the subscript MDR identifies the output or target video signal, α and β are interpolation constants, and x and y are color coordinates. (For example, x and y in the LogYxy color space, and u ′ and v ′ in the LogYu′v ′ color space). Where α is, for example,

Given by. Here, max (Y _VDR ) is the maximum value of Y that can be specified in the (high function) VDR video signal, and max (Y _LDR ) is the maximum value of Y that can be specified in the (low function) LDR video signal. , Max (Y _TARGET ) is the maximum value of Y that can be reproduced on the target display, and β is given by β = 1−α.

Example 1:
In one embodiment, between two enhancement layers, each containing one version of video content optimized for viewing in dark ambient conditions and another version of video content optimized for viewing in bright ambient conditions Then, interpolation is performed. Interpolation may be fixed. In alternative embodiments, the interpolation may be variable depending on the ambient lighting in the display to display. In some embodiments, ambient lighting information 27 is supplied to the input 52 of the interpolation / extrapolation engine 48. In such an embodiment, ambient lighting information 27 controls the relative weights provided to the two input signals 47A and 47B.

Example 2:
In another embodiment, the display 16 or device 32 for display has a color gamut wider than the threshold assumed in the base layer 20. In such an embodiment, interpolation is performed between the base layer 20 and the enhancement layer 22 optimized for a wide color gamut display. The relative weight given to the base layer and the enhancement layer in the interpolation is based on a measure of how wide the display color gamut to display compared to the threshold assumed in the base layer 20 and the enhancement layer 22.

Example 3:
In some embodiments, the video signal created for the display to display is based on three or more video content versions of the video data 10. For example, the display to be displayed can reproduce colors having a color gamut wider than that assumed by the base layer and the ambient illuminance in the display. The data processing stage 18 first interpolates between the two enhancement layers based on the peripheral conditions on the display to be displayed, as described in “Example 1” above, and then described in “Example 2” above. Thus, the interpolation between the resulting signal and the base layer is based on the relative function of the display to be displayed.

  Video data 10 has a wide variety of configurations that can include two or more versions of video content optimized for different display and / or viewing conditions. Different versions of video content may be represented individually in the video data 10, but it is much more efficient to encode the enhancement layer with the difference from the base layer and vice versa. In one example, video data 10 includes a bitstream that includes multiple layers that can be selectively decoded. For example, H.264, sometimes referred to as “Scalable Video Coding” or “SVC”. Different layers can be distributed using a scalable video codec similar to the encoding method defined by Annex G of the H.264 / MPEG-4 AVC video compression standard.

  5A and 5B show a video distribution system according to an embodiment. In FIG. 5A, a video signal containing a version of video content optimized for viewing on a display for display is generated at a decoder. The decoder may be local to the display for display. In FIG. 5B, a video signal including a version of video content optimized for viewing on a display for display is generated at a transcoder upstream of a decoder that receives the optimized video signal. In each case, the intent created by the producer P in the version of the video content is distributed to consumers who view the video content distributed by the system.

  Certain embodiments of the present invention include computer processors that execute software instructions that cause the processor to perform the methods of the present invention. For example, a processor in the data processing stage implements the method of FIG. 4B by executing software instructions in accessible program memory. The present invention can also be provided in the form of a program product. A program product includes any medium that carries a set of computer-readable signals that, when executed by a data processor, includes instructions that cause the data processor to perform the methods of the present invention. The program product according to the invention may be of any form. Program products include, for example, magnetic data storage media such as floppy diskettes and hard disk drives, optical data storage media including CD-ROM and DVD, and electronic data storage media including ROM and flash RAM. Media is included. The computer readable signal on the program product may optionally be compressed and encrypted.

  When a component (eg, software module, processor, assembly, device, circuit, etc.) is referred to above, that component (including the case of “means”) is the equivalent of that component unless otherwise specified. As well as any component that performs (i.e., functionally equivalent) the functionality of the components described, and also includes components that are not structurally equivalent to the disclosed configurations that perform the functions of the illustrated embodiments of the invention. It should be understood.

  Accordingly, the present invention includes, or is constituted by, any elements (various parts or features of the present invention) and their equivalents as described herein that currently exist and / or will be developed in the future, or These are basically configured. Furthermore, the invention disclosed herein by way of example can be practiced without any elements, even if not specifically disclosed herein. Obviously, modifications and variations of the present invention are possible in light of the above teachings. Therefore, it should be understood that the invention may be practiced otherwise than as specifically described herein within the scope of the appended claims.

Accordingly, the present invention can be implemented in any of the forms described herein and is not limited to the following enumerated example embodiments that describe the structure, features, and functions of certain parts of the invention.
(Supplementary note 1) A method for delivering video content optimized for display on a target display,
Providing video data comprising at least first and second different versions of the video content;
Interpolating or extrapolating between the first and second versions of the video content to generate a third version of the video content.
(Supplementary note 2) The first version is optimized for a first level of ambient light, the second version is optimized for a second level of ambient light, and the interpolation or extrapolation is The method of claim 1, based on the level of ambient light at the target display.
(Supplementary note 3) A light sensor monitors the level of ambient light at the target display, and the method includes first and second of the video content based on the level of ambient light at the target display detected by the light sensor. The method of claim 2, comprising applying relative weights to versions of:
(Supplementary note 4) The target display is a display between a first color gamut assumed in the first version of the video content and a second color gamut assumed in the second version of the video content. A color gamut, the method weights relative to the first and second versions of the video content based on a measure of the display gamut relative to a corresponding measure of the first and second gamuts. The method of claim 1, comprising the step of applying.
(Supplementary note 5) The method according to supplementary note 1, wherein each version includes a pixel value of a pixel of the image, and the interpolation includes an interpolation between pixel values of corresponding pixels of the first and second versions.
(Supplementary note 6) The method according to supplementary note 5, wherein the pixel value includes a value of a coordinate in a perceptually linear color space.
(Supplementary note 7) The method according to Supplementary note 1, wherein the method comprises the steps of encoding a third version of the video content and sending the encoded third version from the transcoder to a target display, wherein the method is executed in a transcoder. The method described.
(Supplementary Note 8) Receiving identification information of a device on which the third version of the video content is displayed;
The method of claim 1, comprising controlling the interpolation or extrapolation based at least in part on a corresponding display function of the device relative to display functions associated with the first and second versions of the video content. .
(Supplementary note 9) The method according to supplementary note 8, comprising determining a display function of the device in response to the device being connected to receive the download of the video content.
(Supplementary Note 10) A method of delivering video content optimized for display on a target display, comprising:
Providing video data including different versions of the video content in one data entity, wherein the different versions are one of different color gamuts, different frame rates, different bit depths, and different spatial resolutions. Including a version having the above,
Determining the function of the target display;
Selecting and decoding one of the different versions for display on the target display.
(Supplementary note 11) In the supplementary note 10, the step of decoding one of the selected different versions has a step of decoding a base layer and an enhancement layer and modifying the base layer according to the enhancement layer. The method described.
(Supplementary note 12) A video processing apparatus,
A data processing stage, wherein the data processing stage is:
Reading video data encoding different versions of the video content, the versions having one or more of different gamuts, different frame rates, and different bit depths;
Determining if one of the versions is optimized for display on the target display;
A video configured to provide a version of the version for display on the target display in response to determining that a version of the version is optimized for display on the target display. Processing equipment.
(Additional remark 13) The video processing apparatus of Additional remark 12 which has an interpolation part which combines the at least 2 version of the said different versions, and makes it an output signal which is the interpolation of the at least 2 version of the said different versions.
(Supplementary Note 14) The interpolation unit has a control input, and is configured to multiply the difference between corresponding values in the first and second input video signals by an amount based on the value received at the control input. The video processing apparatus according to attachment 13.
(Supplementary note 15) The video processing device according to supplementary note 14, wherein the control input is connected to receive a signal indicating an ambient lighting state in the target display.
(Supplementary note 16) The video processing device according to supplementary note 12, incorporated in the target display.
(Supplementary note 17) The video processing device according to supplementary note 12, incorporated in a transcoder.
(Supplementary Note 18) The data processing stage receives information about ambient lighting conditions on the target display and determines whether one of the versions is optimized for display on the target display under the ambient lighting conditions. 13. A video processing apparatus according to appendix 12, configured to:
(Supplementary note 19) A video processing apparatus,
A decoder configured to extract a base layer and one or more enhancement layers from the video signal and generate a plurality of input signals, each encoding a different version of the video content;
An interpolator connected to receive the plurality of input signals and to output an output signal that is the different version of the interpolation corresponding to the input signals.
(Supplementary note 20) Connected to receive an ambient illumination signal from an optical sensor arranged to detect an ambient illumination condition in the target display and to control the interpolator based at least in part on a value of the ambient illumination signal 20. A video processing apparatus according to appendix 19.
(Supplementary note 21) The video processing device according to supplementary note 19, wherein the interpolation unit is configured to perform interpolation based on a function of the target display with respect to a function related to the input signal.
(Supplementary Note 22) The interpolation unit has a control input, and is configured to multiply the difference between corresponding values in the first and second input video signals by an amount based on the value received at the control input. The video processing apparatus according to appendix 19.
(Supplementary note 23) The video processing device according to supplementary note 19, wherein the interpolation unit is configured to perform interpolation or extrapolation along a perceptual curve.
(Supplementary note 24) The video processing device according to supplementary note 19, wherein the interpolation unit is configured to perform interpolation or extrapolation according to a logarithm of illuminance.
(Appendix 25) Different versions of video content, having different color gamuts, different frame rates, different bit depths, or optimal for viewing under different ambient light levels Persistent medium having computer readable video data encoded with different versions
(Supplementary note 26) The medium according to supplementary note 25, wherein the video data includes a base layer and a plurality of enhancement layers, and the different versions include the base layer and the base layer combined with the enhancement layer. .
(Supplementary note 27) The medium according to supplementary note 25, wherein the video data includes a base layer and a plurality of enhancement layers, and at least one enhancement layer includes three-dimensional information.
(Appendix 28) A color grading station,
A data storage unit containing video content;
Multiple displays,
A controller comprising:
Displaying the video content on each of the plurality of displays;
Receiving corresponding user input including optimization of the video content for display on each of the plurality of displays;
Storing a plurality of versions of the video content, each including optimizations associated with one display of the plurality of displays;
A color grading station comprising: a controller for combining the plurality of versions into computer readable video data encoding the plurality of different video content versions.
(Supplementary note 29) The color grading station according to supplementary note 28, wherein the video content has a base layer and a plurality of enhancement layers.
(Supplementary note 30) A media server comprising a data storage unit including a plurality of versions of the same video content, and selecting one version of the version of the video content optimized for display on a target display; A media server configured to generate video content optimized for display on a target display by one or both of generating the optimized video content from different versions of video content.
(Supplementary note 31) A device having the novel and useful inventive feature or combination of features or partial combination of features described herein or shown in the drawings.
(Supplementary note 32) A method having novel and useful inventive steps and / or actions, or combinations of steps and / or actions, or partial combinations of steps and / or actions as described herein or shown in the drawings.

  Many changes and modifications may be made in the practice of the invention without departing from the spirit and scope of the invention, as will be apparent to those skilled in the art in view of the above disclosure. Accordingly, the scope of the present invention is defined by the following claims.

Claims (16)

A display device,
An input configured to receive video data and an input configured to receive ambient conditions for viewing the video data;
Decodes the base layer and at least the first and second enhancement layers from the received video data, interpolates between at least two enhancement layers based on the received ambient conditions, and modifies according to the interpolated enhancement layer And a processor configured to generate a video data stream including a configured base layer.   The display device according to claim 1, wherein the ambient conditions include ambient lighting conditions.   The display device of claim 1, wherein the ambient condition includes ambient light intensity and color, and the enhancement layer includes an enhancement layer graded under different combinations of light intensity and color.   Ambient conditions include ambient light intensity and color, and enhancement layers include enhancement layers graded under different combinations of light intensity and color, and the enhancement layer extends beyond the gamut capabilities of the graded display The display device according to claim 1, wherein the display device is used to generate an image of a specified color gamut. The display includes a high dynamic range (HDR) display that has a larger color gamut capability than a display graded enhancement layer,
The display device according to claim 4. Including one of set-top boxes, digital video recorders, and displays such as televisions, home theater displays, digital cinema projectors,
The display device according to claim 1. A method of grading video data,
Acquiring video data;
Grading video data according to the director ’s or producer ’s intent,
Viewing graded video data under a first set of lighting conditions;
Generating a first enhancement layer based on a first lighting state set;
Viewing graded video data under a second lighting state set;
Generating a second enhancement layer based on the second lighting state set;
Making the first and second enhancement layers available when adjusting video data to be displayed based on ambient lighting conditions corresponding to the first and second lighting conditions. The step of generating the enhancement layer considers the target layer to view the video data including a high dynamic range (HDR) display having a color gamut expanded to a display that is used to perform color grading, and Including the step of generating,
The method of claim 7.   8. The method of claim 7, wherein the first and second lighting conditions include conditions that normally occur when viewing moving image data on a target display, wherein the first lighting conditions are opposite to the second lighting conditions. The first illumination condition includes light of a first color condition, and the second illumination condition includes light of a second color condition;
The method of claim 9. The first lighting condition includes bright ambient light, the second lighting condition includes dark ambient light, and the target display color gamut of the graded video data and enhancement layer is below the first set of lighting conditions. A display with a wider color gamut than the display used to view the graded video data in the step of viewing the graded video.
11. A method according to any one of claims 7 to 10. A color grading station,
Input configured to receive video data,
A display configured to display video data;
A processor configured to receive input relating to grading adjustments to the video data displayed on the display;
The processor is configured to receive a first set and a second set of inputs related to grading adjustments based on corresponding first and second ambient light conditions at which video data is displayed on the display;
The processor is further configured to package the first set and the second set of adjustments as enhancement layers with the video data.
Color grading station. Ambient conditions include ambient light intensity and color, and enhancement layers include enhancement layers graded under different combinations of light intensity and color.
The color grading station according to claim 12.   Ambient conditions include ambient light intensity and color, and enhancement layers include enhancement layers graded under different combinations of light intensity and color, and the enhancement layer extends beyond the gamut capabilities of the graded display 13. A color grading station according to claim 12, wherein the color grading station is used to generate an image of a specified color gamut. Graded video data and enhancement layer target displays include high dynamic range (HDR) displays,
The color grading station according to claim 12. The first and second lighting conditions include conditions that normally occur when viewing the moving image data on the target display, and the first lighting conditions are opposite to the second lighting conditions.
The color grading station according to claim 15.

Images