RU2813229C1 - Computing dynamic metadata for editing hdr content - Google Patents

Abstract

FIELD: editing video content.

SUBSTANCE: invention relates to a system and method for editing video content. The solution includes receiving video input data; converting input video data to a predefined format; generating a plurality of initial metadata values for a frame of converted video data, the plurality of initial metadata values including the first metadata value corresponding to the first fixed value not calculated from content including the frame, the second metadata value corresponding to an average luminance value of the frame, and the third value metadata corresponding to a second fixed value not calculated from the content, wherein the first metadata value, the second metadata value and the third metadata value include information used by the decoder to prepare by rendering a decoded image on a display.

EFFECT: improving the quality of video content processing.

17 cl, 10 dwg

Description

[0001] This application claims priority to patent application (Europe) number 20178044.2; and U.S. Provisional Application No. 63/034,006, both filed June 3, 2020, each of which is incorporated herein in its entirety by reference.

State of the art

1. Field of technology to which the invention relates

[0002] This application generally relates to images; More specifically, this application relates to the generation of metadata for display control in connection with video content.

2. Description of the prior art

[0003] As used herein, the term "dynamic range (DR)" may refer to the ability of the human visual system (HVS) to perceive a range of intensities (e.g., luminance, luminance signal) in an image, for example, from the darkest blacks ( the darkest areas) to the brightest whites (the brightest areas of the images). In this sense, DR refers to "scene-related" intensity. DR may also refer to the ability of a display device to adequately or approximately prepare by rendering an intensity range of a particular latitude. In this sense, DR refers to "display-related" intensity. Unless a particular sense is clearly stated to have a particular meaning at any point in the description herein, it is understood that the term may be used in any sense, such as interchangeably.

[0004] As used herein, the term “high dynamic range (HDR)” refers to DR latitude that spans approximately 14-15 or more orders of magnitude of HVS. In practice, DR, by which a person can simultaneously perceive a wide latitude over a range of intensities, may be truncated to a certain extent, relative to HDR. As used herein, the terms "enhanced dynamic range (EDR)" or "visual dynamic range (VDR)" may separately or interchangeably refer to DR, which is perceived in a scene or image by HVS, which includes eye movements, providing some light adaptation changes in a scene or image. As used herein, EDR may refer to DR that spans 5-6 orders of magnitude. Thus, although perhaps narrower to a certain extent than the referenced HDR of the true scene, EDR nevertheless represents significant DR breadth and may also be referred to as "HDR".

[0005] In practice, images contain one or more color components (e.g., RGB, a luminance signal Y, and a chrominance signal Cb and Cr), where in a quantized digital system each color component is represented by a precision of n bits per pixel (e.g. n=8). A bit depth of n≤8 (for example, 24-bit color JPEG images) may be used with standard dynamic range (SDR) images, while a bit depth of n>8 may be considered for enhanced dynamic range (EDR) images to avoid contouring and staircase artifacts. In addition to integer data types, EDR and high dynamic range (HDR) images can also be stored and distributed using high-precision (e.g., 16-bit) floating point formats, such as the OpenEXR file format developed by Industrial Light and Magic.

[0006] A reference electro-optical transfer function (EOTF) for a given display characterizes the relationship between color values (eg, brightness) of an input video signal to output screen color values (eg, screen brightness) generated by the display. For example, ITU Rec. ITU-R BT.1886 "Reference electro-optical transfer function for flat panel displays used in HDTV studio production" (March 2011), which is contained in this document in its entirety by reference, specifies a reference EOTF for flat panel displays based on the measured characteristics of the electro-optical radiation tube (CRT). Given a video stream, information regarding its EOTF is typically embedded in the bitstream as metadata.

[0007] Many consumer desktop displays render non-HDR content at a maximum brightness of 200-300 cd/ ^cm2 (“nits”), and consumer high-definition and ultra-high-definition television receivers (“HDTV” and “UHD TV” ") - from 300 to 500 nits. This displayed output therefore clearly shows the low dynamic range (LDR), also called "SDR", relative to HDR or EDR. As the availability of HDR or EDR content increases due to improvements in both image capture hardware (such as cameras) and HDR displays (such as Dolby Laboratories' PRM-4200 professional reference monitor or 4K flagship Sony Trimaster HX 31'' HDR Monitor), HDR content can be color corrected and displayed on HDR displays that support wider dynamic ranges (for example, 700 nits to 5000 nits or more). Such displays may be specified using alternative EOTFs that support high brightness specifications (eg, 0 to 10,000 nits). An example of such an EOTF is specified in SMPTE ST 2084:2014 "High Dynamic Range EOTF of Mastering Reference Displays", which is contained in this document by reference in its entirety.

[0008] As used herein, the term "display control" refers to the processing (eg, tone and gamma conversion) required to convert an input video signal with a first dynamic range (eg, 1000 nits) to a display with a second dynamic range ( for example 500 nits). Examples of display control processes are described in PCT application serial number PCT/US2014/016304 (the '304 application), filed February 13, 2014, "Display management for high dynamic range video" by R. El Atkins et al., which is contained in its entirety in this document by reference. Display control may be performed by or controlled by input metadata generated by the incoming content source and multiplexed into a coded bitstream.

[0009] The approaches described in this section are approaches that may be considered, but do not necessarily represent approaches that have been previously conceived or considered. Therefore, unless otherwise noted, any approach described in this section should not be assumed to be prior art simply by virtue of its inclusion in this section. Likewise, problems identified with one or more approaches should not be assumed to be identified in the prior art based on this section unless otherwise noted.

Brief summary of the invention

[0010] Various aspects of the present disclosure relate to devices, systems and methods for generating metadata that can be used to provide increased video quality, increased temporal stability, low computational requirements and reduced latency.

[0011] In one aspect of the present disclosure, there is provided a method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format to thereby generate converted video data; generating a plurality of initial metadata values for a frame of converted video data, the plurality of initial metadata values including a first metadata value corresponding to a first fixed value not calculated from content including the frame, a second metadata value corresponding to an average luminance value of the frame, and a third value metadata corresponding to a second fixed value not calculated from the content, wherein the first metadata value, the second metadata value and the third metadata value include information used by the decoder to prepare by rendering a decoded image on a display.

[0012] In another aspect of the present disclosure, there is provided a video editing system comprising: a storage device; and a processor configured to instruct the video editing system to: receive input video data, convert the input video data into a predetermined format thereby generating converted video data, generate a plurality of initial metadata values for a frame of converted video data, the plurality of initial metadata values including the first a metadata value corresponding to a first fixed value not calculated from the content including the frame, a second metadata value corresponding to an average luminance value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content.

[0013] In another aspect of the present disclosure, there is provided a method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format to thereby generate converted video data; generating a first metadata value for a frame of converted video data, the first metadata value corresponding to an average brightness value of the frame; defining a time window including the frame, wherein the time window has a predetermined length in time; calculating a maximum metadata range and a minimum metadata range over a time window to thereby generate a first filtered metadata value and a second filtered metadata value; and smoothing the first filtered metadata value and the second filtered metadata value to thereby generate a first smoothed and filtered metadata value and a second smoothed and filtered metadata value.

[0014] Thus, various aspects of the present disclosure provide improvements to at least the fields of video capture, video editing, and signal processing technology, as well as related fields of video and other content delivery technology.

Brief description of drawings

[0015] These and other more detailed and specific features of the various embodiments are more fully set forth in the description below, with reference to the accompanying drawings, in which:

[0016] FIG. 1 illustrates an exemplary video delivery pipeline in accordance with various aspects of the present disclosure;

[0017] FIG. 2 illustrates a processing flow for an exemplary method of generating metadata in accordance with various aspects of the present disclosure;

[0018] FIG. 3 illustrates exemplary metadata generated by the method of FIG. 2;

[0019] FIG. 4 illustrates a processing flow for another exemplary method of generating metadata in accordance with various aspects of the present disclosure;

[0020] FIG. 5 illustrates exemplary metadata generated by the method of FIG. 2;

[0021] FIG. 6 illustrates a processing flow for another exemplary method of generating metadata in accordance with various aspects of the present disclosure;

[0022] FIG. 7A-7C, respectively, illustrate exemplary metadata generated by the method of FIG. 6; And

[0023] FIG. 8 illustrates an exemplary system in accordance with various aspects of the present disclosure.

Detailed Description of the Invention

[0024] This disclosure and aspects thereof may be embodied in various forms, including hardware, devices or circuits controlled by computer-implemented methods, computer program products, computer systems and networks, user interfaces and application programming interfaces; as well as hardware-based methods, signal processing circuits, memory matrices, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and the like. The above summary is intended only to give a general idea regarding various aspects of the present disclosure, and does not limit the scope of the disclosure in any way.

[0025] In the following description, numerous details are set forth, such as optical device configurations, timing synchronizations, operations, and the like, so as to provide an understanding of one or more aspects of the present disclosure. It will be apparent to those skilled in the art that these specific details are merely exemplary and are not intended to limit the scope of this application.

[0026] Additionally, although the present disclosure focuses primarily on examples in which various schemas are used in generating content, it should be understood that this represents merely one example implementation. Additionally, it should be understood that the disclosed systems and methods may be used in any device in which there is a need to generate, modify, or display image data and/or associated metadata; for example, in cinema, home and other commercial projection systems, smartphone and other consumer electronic devices, head-up displays, virtual reality style displays, etc.

[0027] Metadata

[0028] As used herein, the term "metadata" refers to any supporting information that is carried as part of the encoded bitstream and helps the decoder render the decoded image. Such metadata may include, but is not limited to, color space or gamma information, reference display parameters, and auxiliary signal parameters, for example, as described herein, and may be collectively referred to as “imaging metadata.”

[0029] Such metadata may be divided or classified into different levels of metadata. In one specific example, in which the present disclosure is implemented in a Dolby Vision architecture, the transform metadata may include L0 metadata, L1 metadata, L2/L8 metadata, L3 parameters, L4 metadata, L11 metadata, etc. .P. In some examples, L0 metadata provides or describes information regarding the dynamic range of video data for entire video content (ie, multiple pictures and/or scenes). For example, the L0 metadata may include L0min and L0max, which respectively represent the minimum luminance and maximum luminance for the video content, respectively. In one specific example, in which the video content is HDR content, L0min and L0max may correspond to 0.01 nits and 1000 nits, respectively.

[0030] In some examples, L1 metadata provides or describes information regarding the distribution of luminance values in a source image, source scene, source GOP, etc., as represented in the video data. The distribution of luminance values may be represented by histograms extracted based on image content (eg, pixel values, luminance values, chrominance values, Y values, Cb/Cr values, RGB values, etc.) for an image, scene , GOP, etc. The L1 metadata may include parameters L1min, L1mid and L1max, which respectively represent the minimum brightness ("crease"), grayscale brightness ("mid") and maximum brightness ("clip") for one or more images, scenes, GOP, etc., in video content, respectively.

[0031] Other levels of metadata may also be specified. For example, L2 provides or describes information regarding adjustments to video characteristics that are initiated or traced back to adjustments made by the director, color corrector, video editor, etc., in the production studio and/or during post-production. L8 metadata is similar to L2 metadata and in some cases may be equivalent to L2 metadata (eg, depending on the corresponding tone curves). The L2 and L8 metadata may be referred to as "cropping parameters" and may indicate or relate to the gain/offset/power of the image data. The L2 metadata may correspond to a first reference display having a first reference dynamic range.

[0032] L3 metadata provides or describes information regarding adjustments to video characteristics that are initiated or traced back to adjustments made by a director, color corrector, video processor, or the like. Compared with L2 metadata, L3 metadata may correspond to a second reference display with a reference dynamic range different from the reference dynamic range of the first reference display. The L3 metadata may include, for example, offsets or adjustments from the L1 metadata, including offsets or adjustments for crease, mid, and/or clip luminance values.

[0033] L4 metadata provides or describes information regarding global dimming operations. L4 metadata can be calculated by the encoder during pre-processing and can be calculated using primary RGB colors. In one example, the L4 metadata may include data that dictates the global brightness level of the display panel backlight on a frame-by-frame basis. Other generated metadata, for example, L11 metadata, may provide or describe information that should be used to identify the source of video data, for example, cinematic content, computer game content, sports content, and the like. Such metadata may further provide or describe intended movie frame settings, such as intended white point, sharpness, and the like.

[0034] Taken together, the imaging metadata may include transform data for converting from a first dynamic range to a second dynamic range that is different from the first dynamic range. In some aspects of the present disclosure, the first dynamic range may be wider than the second dynamic range (eg, when converting from HDR to SDR). In other aspects of the present disclosure, the second dynamic range may be wider than the first dynamic range (eg, when converting from SDR to HDR).

[0035] Comparative methods for generating imaging metadata may include calculating metadata per frame (eg, L1 metadata per frame) and either applying IIR filtering (eg, for live or game broadcast) or combining the metadata into scenes by calculating the minimum, middle and maximum of the scene (for example, for studio or professional offline content). However, none of these methods provide an instant preview of your content as you edit. For example, a user might want to brighten an HDR movie frame and preview the result on an SDR monitor as a reference display. To do this, the following steps are carried out: regulation is applied to the HDR film frame; L1 metadata is recalculated for the adjusted HDR movie frame; tone mapping operations are modified; and the adjusted tone mapping is applied to the adjusted HDR movie frame to generate an SDR preview movie frame.

[0036] This set of operations may be satisfactory if applied to a single image, since one image may conveniently be stored in a storage device such that the operations may be applied consecutively. However, the L1 metadata (and therefore the resulting tone curve) can vary significantly between frames, causing flickering in the converted image. This may be called "temporary instability". To ensure temporal stability of the tonal curve, some filtering may be performed. However, if the metadata recalculation operation depends on multiple images, then the throttling operation and the metadata recalculation operation must be applied to multiple images, which may be impractical or impossible at high video bit rates.

[0037] Thus, comparative methods for generating imaging metadata may result in slowdowns in the iterative cycle of adjusting, updating, and previewing results. In addition, the comparative IIR filtering method may require a very large number of frames that must be analyzed prior to the current frame, since the current frame requires all previous frames for the IIR filter. The studio or professional comparison method requires all frames of a scene, which in some cases can be on the order of minutes or even hours in length.

[0038] With this in mind, various aspects of the present disclosure provide methods for generating metadata for efficient editing; those. for editing that results in high quality images and high temporal stability. Various aspects of the present disclosure can perform such editing with low computational requirements and low or no latency. In some aspects, methods for generating metadata are included or used in/with the video delivery pipeline.

[0039] Video delivery pipeline

[0040] FIG. 1 illustrates an example video delivery pipeline and shows the various stages from video capture to display of video content. Moreover, although the following description is provided in terms of video (ie, moving images), the present disclosure is not limited to this. In some examples, the image content may be still images or combinations of video and still images. Image content may be represented using raster (or pixel) graphics, vector graphics, or a combination of raster and vector graphics. Fig. 1 illustrates an image generating unit 101, a production unit 102, a post-production unit 103, an encoding unit 104, a decoding unit 105, and a display control unit 106. The various blocks illustrated in FIG. 1 may be implemented as/via hardware, software, firmware, or combinations of the above. In addition, different groups of illustrated blocks may have combined respective functions and/or may be executed in different devices and/or at different times. Individual or groups of illustrated blocks may be implemented by circuitry, including, but not limited to, central processing units (CPUs), graphics processing units (GPUs), ASICs, FPGAs, and combinations of the foregoing. Operations performed by one or more blocks may be processed locally, remotely (e.g., in the cloud), or a combination of locally and remotely.

[0041] As illustrated in FIG. 1, the video delivery pipeline further includes a reference display 111 that may be provided to assist or monitor operations conducted in the post-production unit 103 and the target display 112. For purposes of explanation, the imaging unit 101, the production unit 102, the unit The post-production unit 103 and encoding unit 104 may be referred to as “upstream blocks or components,” while the decoding unit 105 and display control unit 106 may be referred to as “lower blocks or components.”

[0042] In the example illustrated in FIG. 1, video content in the form of a sequence of video frames 121 is captured or generated in the imaging unit 101. Video frames 121 may be captured digitally (eg, by a digital camera) or generated by a computer (eg, using computer animation) to form video data 122. Alternatively, video frames 121 may be captured on film by a film camera and then converted to digital format so as to provide video data 122. In either case, video data 122 is provided to production unit 102, where it is edited to provide production stream 123. Video frames 121 may be grouped into sequences, such as shots and scenes. A snapshot, for example, is a collection of temporarily connected frames. Snapshots can be separated by "snapshot switches" (eg, points in time at which all of the image's content is changed instead of just part of it). A scene, for example, is a sequence of shots that describe a narrative segment of larger content. In one specific example, in which the video content is an action movie, the video content may include (among others) a chase scene, which in turn includes a sequence of shots (e.g., shot of the driver of the pursuing vehicle, shot of the driver of the pursued vehicle, shot the street where the chase is taking place, etc.).

[0043] The video data in production stream 123 is then provided to the processor or processors in post-production block 103 for post-production editing. Editing performed in the post-production block 103 may include adjusting or modifying colors or brightness in specific areas of the image in order to enhance the quality of the images or achieve a particular appearance for the image in accordance with the creative intent of the video creator (or editor). This may be called "color sync" or "color grading." Other editing (eg, selecting and arranging scenes, cropping images, adding machine-generated visual effects or overlays, etc.) may be performed in post-production block 103 to result in a distributed stream 124. In some examples, post-production block 103 may provide intermediate stream 125 to the reference display 111 to allow images to be viewed on its screen, for example, to assist in the editing process. One, two, or all of the production unit 102, the post-production unit 103, and the encoding unit 104 may further include processing for adding metadata to the video data. This post-processing may include, but is not limited to, statistical analysis of content properties. Post-processing can be done locally or remotely (eg as cloud processing).

[0044] Following post-production operations, the distributed stream 124 may be delivered to an encoder 104 for downstream delivery to decoding and playback devices such as televisions, set-top boxes, movie theaters, laptop computers, tablet computers, and the like. In some examples, encoding unit 104 may include audio and video encoders, for example, encoders defined by the Advanced Television System Committee (ATSC) standard, Digital Video Broadcast (DVB), Digital Versatile Disc (DVD), Blu -Ray and other delivery formats, in order to generate a 126-bit encoded stream. At the receiver, the coded bit stream 126 is decoded by decoding module 105 to generate a decoded signal 127 representing an identical or good approximation of the propagated stream 124. The receiver may be coupled to a target display 112, which may have characteristics that differ from the reference display 111. If the reference display 111 and target display 112 have different characteristics, display control unit 106 may be used to convert dynamic range or other characteristics of decoded signal 127 into characteristics of target display 112 by generating display-converted signal 128. Display control unit 106 may additionally or alternatively be used to provide power control to the target display 112.

[0045] The target display 112 generates an image using an array of pixels. The specific matrix structure depends on the architecture and resolution of the display. For example, if the target display 112 operates in a liquid crystal architecture, it may include a relatively low resolution backlight array (e.g., an array of LEDs or other light emitting elements) and a relatively high resolution liquid crystal array and color filter array to selectively attenuate white light from the backlight array and provide colored light (often called "dual modulation display technology"). If the target display 112 operates in an OLED architecture, it may include a high-resolution array of self-emissive color pixels.

[0046] The communication link between the upper blocks and the lower blocks (i.e., the path over which the 126 bit coded stream is provided) may be via live or real-time transmission, for example, via broadcast over an air interface using electromagnetic waves or via content delivery line, for example, through optical fiber, twisted pair (Ethernet) and/or coaxial cables. In other examples, the communication link may be accomplished through time-independent transmission, for example, by recording an encoded bitstream onto a physical medium (eg, a DVD or hard drive) for physical delivery to an end user device (eg, a DVD player). A decoder unit 105 and a display control unit 106 may be included in a device associated with the target display 112; for example, in the form of a smart TV that includes decoding, display control, power control and display functions. In some examples, decoder block 105 and/or display control block 106 may be included in a device separate from target display 112; for example, in the form of a set-top box or multimedia player.

[0047] The decoder block 105 and/or the display control block 106 may be configured to receive, parse, and operate in response to metadata included or added to the underlying blocks. Such metadata may therefore be used to provide additional adjustment or control to the target display 112. The metadata may include imaging metadata described above (eg, Dolby Vision metadata). Metadata may be generated in post-production block 103 using reference display 111 and production stream 123 and/or intermediate stream 125.

[0048] Semi-dynamic generation of metadata

[0049] Imaging metadata may be generated using a "semi-dynamic" method, an example of which is illustrated in FIG. 2. The steps illustrated in FIG. 2 may be performed automatically by hardware, software, firmware, or combinations thereof associated with post-production unit 103 of FIG. 1.

[0050] As illustrated in FIG. 2, the semi-dynamic metadata generation process may begin at step 201, where the video signal is converted to a predefined format (eg, a specific EOTF). For example, the video signal may be converted from hybrid log-gamma (HLG) or SLOG-3 HDR to perceptual RGB quantizer (PQ)-based HDR, for example, as described in Rec. ITU-R BT.2100-1 (06/2017). In some implementations, at step 202, spatial smoothing and/or subsampling may be applied. Step 202 may be implemented using a two-dimensional smoothing kernel (eg, [1 1]/2 or [1 3 3 1]/8). This reduces sensitivity to individual pixels. In some examples, the image may be downsized to a smaller size (eg, half the size) after filtering. In other implementations, step 202 may be omitted, and the semi-dynamic metadata generation process may proceed directly from step 201 to step 203. At step 203, initial L1mid values are calculated (eg, on a frame-by-frame basis). For example, L1mid may represent the average value of the maximum RGB brightness. In other words, L1mid may represent the average value of the maximum of the R, G, B color component values for each pixel. The L1min and L1max values are not calculated from the content; instead, at step 204, fixed values are used that correspond to the minimum and maximum allowable range of image content as represented in the L0 metadata (eg, Level0.Smin and Level0.Smax). For example, L1mid may represent the average brightness value of the Y component values when the YCbCr representation is used.

[0051] As one specific example of the steps of FIG. 2, the following pseudocode is represented using MATLAB format:

% Conversion to PQ (example step 201)
if strcmpi(SEOTF,'HLG')
imPQ=L2PQ(HLG2L(im, SMax,Smin));
else
imPQ=im;
end
% Downsample by 50% (example step 202)
imPQHalf=Half(imPQ);
% Compute metadata per pixel (example step 203, 204)
maxRGB=max(imPQHalf, [],3);
L1Min=L2PQ(MD{n}.Level0.Smin);
L1Mid=mean(MAXRGB(:));
L1Max=L2PQ(MD{n}.Level0.Smax);

[0052] FIG. 3 illustrates an example output of the above steps for metadata corresponding to the values "crease", "middle" and "clip". Each plot illustrates the normalized luminance signal as a function of frame number. The lines 310 _min , 310 _mid and 310 _max illustrate the minimum, average and maximum brightness values of a given frame, respectively. The 320 _min , 320 _mid and 320 _max lines illustrate the L1min, L1mid and L1max values for a given frame.

[0053] Since L1min and L1max are set equal to the values of L0min and L0max, respectively, the crease and clip metadata values do not strictly match the actual luminance values for all frames. In the specific example illustrated in FIG. 3, the minimum luminance value per frame (line 310 _min ) varies between approximately 0.01 and 0.48, while the value L1min (line 320 _min ) is set to approximately 0. The maximum luminance value per frame (line 310 _max ) varies between approximately 0.53 and 0.82, while the L1max value (line 320max) is set to approximately 0.82. However, since L1mid is determined by a dynamic value, the "mid" metadata value closely matches the actual brightness value for all frames. As illustrated in FIG. 3, the average brightness value per frame (310 _mid line) and the L1mid value (320 _mid line) have a high agreement.

[0054] Semi-dynamic metadata generation requires processing of only one frame. This method results in high efficiency; however, as can be seen from FIG. 3, Using static values for the "crease" and "clip" metadata values results in slightly lower converted image contrast for images that do not use the full dynamic range of the container.

[0055] Dynamic generation of metadata

[0056] Imaging metadata may be generated using a "dynamic" method, an example of which is illustrated in FIG. 4. The steps illustrated in FIG. 4 may be performed automatically by hardware, software, firmware, or combinations thereof associated with post-production unit 103 of FIG. 1.

[0057] As illustrated in FIG. 4, the dynamic metadata generation process may begin at step 401, where the video signal is converted to a predefined format (eg, a specific EOTF). For example, the video signal may be converted from HLG or SLOG-3 HDR to RGB PQ HDR, for example, as described in Rec. ITU-R BT.2100-1 (06/2017). In some implementations, at block 402, spatial smoothing and/or subsampling may be applied. Step 402 may be implemented using a two-dimensional smoothing kernel (eg, [1 1]/2 or [1 3 3 1]/8). This reduces sensitivity to individual pixels. In some examples, the image may be downsized to a smaller size (eg, half the size) after filtering. In other implementations, step 402 may be omitted, and the semi-dynamic metadata generation process may proceed directly from step 401 to step 403. At step 403, the initial values L1min, L1mid and L1max are calculated (eg, on a frame-by-frame basis). For example, L1min may represent a minimum RGB brightness value, L1mid may represent an average of a maximum RGB brightness value, and L1max may represent a maximum RGB brightness value.

[0058] Then, at step 404, the L1min and L1max values are recalculated using the dynamic values for the given shot or scene. For example, each frame of a photo or scene can be analyzed to determine the minimum and maximum RGB brightness for that photo or scene. At step 405, the L1min and L1max values are recalculated using the value that corresponds to the minimum and maximum for the entire “clip” determined at step 404. Compared to the semi-dynamic method described above, the dynamic method can more closely match the image data because L1min and L1max are determined relative to the actual minimum and maximum RGB luminance of the content, rather than the permissible (i.e., possible) minimum and maximum RGB luminance of the content.

[0059] As one specific example of the steps of FIG. 4, the following pseudocode is represented using MATLAB format:

% Convert to PQ (example step 401)
if strcmpi(SEOTF,'HLG')
imPQ=L2PQ(HLG2L(im, SMax,Smin));
else
imPQ=im;
end
% Downsample by 50% (example step 402)
imPQHalf=Half(imPQ);
% Compute metadata per pixel (example step 403)
maxRGB=max(imPQHalf, [],3);
L1Min=min(imPQHalf(:));
L1Mid=mean(MAXRGB(:));
L1Max=max(imPQHalf(:));
% Determine the minimum and maximum of the entire "clip" (example step 404)
minPQ=1;
maxPQ=0;
for n=1:N
if L1Min(n)<minPQ
minPQ=L1Min(n)
end
if L1Max(n)>maxPQ
maxPQ=L1Max(n)
end
end
% Update dynamic metadata (example step 405)
for n=1:N
fprintf("Updating metadata for frame%d%d/n", n, N);
NewMinF(n)=minPQ;
NewMidF(n)=L1Mid(n);
NewMaxF(n)=maxPQ;
end

[0060]a

Fig. 5 illustrates an exemplary output of the above steps for metadata corresponding to the above steps for metadata corresponding to the values "wrinkle", "middle" and "clip". Each plot illustrates the normalized luminance signal as a function of frame number. The lines 510 _min , 510 _mid and 510 _max illustrate the minimum, average and maximum brightness values of a given frame, respectively. The lines 520 _min , 520 _mid and 520 _max illustrate the values of L1min, L1mid and L1max for a given frame.

[0061] Because L1min and L1max are set to the minimum values for the entire frame of the clip, the crease and clip metadata values do not strictly match the actual luminance values for all frames. However, compared to the semi-dynamic method described above, the crease and clip metadata values more closely match the actual brightness values. In the specific example illustrated in FIG. 5, the minimum luminance value per frame (510 _min line) varies between approximately 0.02 and 0.48, while the L1min value (510 _min line) is set to approximately 0.02. In comparison, in the semi-dynamic method as illustrated in FIG. 3, the L1min value (line 310 _min ) is set to 0. The maximum luminance value per frame (line 510 _max ) varies between approximately 0.53 and 0.82, while the value L1max (line 520max) is set to approximately 0.82 . As described above, since L1mid is determined by a dynamic value, the "mid" metadata value closely matches the actual brightness value for all frames. As illustrated in FIG. 5, the average brightness value per frame (510 _mid line) and the L1mid value (520 _mid line) have a high agreement.

[0062] Dynamic metadata generation requires processing of all frames in a given picture or scene and may therefore result in higher computational requirements than semi-dynamic metadata generation. However, as can be seen from FIG. 5, the use of dynamic values for the "crease" and "clip" metadata values does not suffer from the same type of contrast reduction in the converted images for images that do not use the full dynamic range of the container.

[0063] Windowing FIR metadata

[0064] Imaging metadata may be generated using a windowing method, an example of which is illustrated in FIG. 6. The steps illustrated in FIG. 6 may be performed automatically by hardware, software, firmware, or combinations thereof associated with post-production unit 103 of FIG. 1.

[0065] As illustrated in FIG. 6, the metadata windowing process may begin at step 601, where the AV video signal is converted to a predefined format (eg, a specific EOTF). For example, the video signal may be converted from HLG or SLOG-3 HDR to RGB PQ HDR, for example, as described in Rec. ITU-R BT.2100-1 (06/2017). Thereafter, at step 602, spatial smoothing and/or subsampling is applied. Step 402 may be implemented using a two-dimensional smoothing kernel (eg, [1 1]/2 or [1 3 3 1]/8). This reduces sensitivity to individual pixels. In some examples, the image may be downsized to a smaller size (eg, half the size) after filtering. At step 603, the initial values L1min, L1mid and L1max are calculated (eg, per pixel). For example, L1min may represent the minimum value of the minimum RGB brightness, L1mid may represent the average value of the maximum RGB brightness, and L1max may represent the maximum value of the maximum RGB brightness.

[0066] Next, at step 604, a list of frames in the sliding window (k) is determined. This can be specified as the maximum length T in time multiplied by the frame rate. The time window T (also called a "time window") may be statement configurable. In some examples, the time window T is configurable between 0.4 seconds and 8 seconds. In one specific example, the time window T is 4 seconds. As discussed in more detail below, a particular time window value T is selected to balance temporal stability and quality of movie frames; for example, smaller windows may introduce a degree of flicker, while larger windows may result in a more static experience. However, the window itself is configured in such a way that it does not intersect fast scene switches and is therefore limited by the first and last frame of the current scene.

[0067] Step 605 is the first pass where the minimum and maximum of the window are calculated. For example, for each frame f in sliding window k, step 605 may smooth the metadata using a one-dimensional smoothing kernel, for example, [1 1]/2, [1 2 1]/4, [1 3 3 1]/8, or [ 1 3 8 3 1]/16, in order to thereby obtain smoothed metadata L1minS(f) and L1maxS(f). This reduces sensitivity to individual frames. Thereafter, step 605 may determine the maximum metadata range over the sliding window k; for example, by calculating L1minM(f)=min(L1minS(k)) (i.e. the minimum smoothed minimum metadata value) and L1maxM(f)=max(L1max(S(k)) (i.e. the maximum smoothed maximum metadata values).

[0068] Step 606 is a second pass in which the minimum and maximum of the window are smoothed. For example, block 606 may compute a smoothing filter F as a Gaussian with L taps and standard deviation s=L/6 and then, for each frame f in the sliding window k, smooth the filtered metadata computed at block 605 using the smoothing filter F. to obtain smoothed and filtered L1minF and L1maxF metadata values. Step 606 may also, for each frame f, set L1min and L1max equal to the smoothed and filtered metadata values and set L1mid equal to its initial value determined at step 603.

[0069] Since steps 605 and 606 apply to a sliding window of frames, steps 604, 605 and 606 may be repeated such that all possible windows in a given scene are processed.

[0070] As one specific example of the steps of FIG. 6, the following pseudocode is represented using MATLAB format:

% Convert to PQ (example step 601)
if strcmpi(SEOTF,'HLG')
imPQ=L2PQ(HLG2L(im, SMax,Smin));
else
imPQ=im;
end
% Downsample by 50% (example step 602)
imPQHalf=Half(imPQ);
% Calculate metadata per pixel (example step 603)
maxRGB=max(imPQHalf, [],3);
L1Min=min(imPQHalf(:));
L1Mid=mean(MAXRGB(:));
L1Max=max(imPQHalf(:));
% Modification for window metadata (example steps 604, 605)
for n=1:N
fprintf("Updating metadata for frame%d%d/n", n, N);
% Retrieve local window (example step 604)
kmin=n-FIRWindow/2;
kmax=n+FIRwindow/2;
klastscenecut=SceneCuts(find(n>=SceneCuts,1,'last'));
kmin=max(kmin, klastscenecut);
knextscenecut=SceneCuts(find(n<SceneCuts,1,'first'));
if ~isempty(knextscenecut)
kmax=min(kmax, knextscenecut-1);
end
%Calculate the middle and maximum of the scene (example step 605)
padding=[[1 1]*k(1) k[1 1]*k(end)];
L1MinF=conv(L1Min(padding), [1 3 8 3 1]/16,'valid');
L1MaxF=conv(L1Max(padding), [1 3 8 3 1]/16,'valid');
WindowedMin(n)=min(L1MinF);
WindowedMax(n)=max(L1MaxF);
end
% Smoothing (example step 606)
for n=1:N
fprintf("Updating metadata for frame%d%d/n", n, N);
% Retrieve local window (example step 604)
kmin=n-FIRWindow/2;
kmax=n+FIRwindow/2;
klastscenecut=SceneCuts(find(n>=SceneCuts,1,'last'));
kmin=max(kmin, klastscenecut);
knextscenecut=SceneCuts(find(n<SceneCuts,1,'first'));
if ~isempty(knextscenecut)
kmax=min(kmax, knextscenecut-1);
end
% Filtering (example step 605)
H=exp(-(kn).hathathat2/(2*(1/6*FIRWindow)hathathat2))';
H=H/sum(H);
NewMinF(n)=WindowedMin(k)*H;
NewMidF(n)=L1Mid(n);
NewMaxF(n)=WindowedMax(k)*H;
end

[0071] FIG. 7A-7C illustrate an example output of the above steps for metadata corresponding to the values of "crease", "middle" and "clip" for various window lengths. Each plot illustrates the normalized luminance signal as a function of frame number. In each of FIG. 7A-7C, lines 710 _min , 710 _mid and 710 _max illustrate the minimum, average and maximum luminance values of a given frame, respectively. In fig. 7A, lines 721 _min , 721 _mid and 721 _max illustrate the values of L1min, L1mid, L1max for a given frame, calculated according to the methods described above with a window length T of 1 second. In fig. 7B, lines 722 _min , 722 _mid and 722 _max illustrate the values of L1min, L1mid, L1max for a given frame, calculated according to the methods described above with a window length T of 4 seconds. In fig. 7C, lines 723 _min , 723 _mid and 723 _max illustrate the values of L1min, L1mid, L1max for a given frame, calculated according to the methods described above with a window length T of 8 seconds.

[0072] By comparing FIG. 7A-7C with each other, it can be seen that shorter window lengths result in increased quality of movie frames (eg, _max line 721 more closely tracks _max line 710); however, longer window lengths result in increased temporal stability (e.g., the 723 _max line exhibits a greater degree of variability). In one particular implementation of the present disclosure, the window length is preferably 4 seconds, as illustrated in FIG. 7B.

[0073] Metadata windowing generates metadata that is the outer "crease" and "clip" envelope of a sliding window. Compared to semi-dynamic metadata generation, it requires more frames to compute, but is finite in the number of frames required to compute metadata for any one frame, unlike the comparative methods described above. By applying the filter twice in succession, the resulting filter essentially triples the length of the specified window. Sequential operations can be combined into a single operation or thread, for example using the pseudocode illustrated above.

[0074] Exemplary Computer System Implementation

[0075] Various aspects of the present disclosure may be implemented by a computer system, systems configured in electronic circuits and components, an integrated circuit (IC) device such as a microcontroller, FPGA or other configurable programmable logic device (PLD), discrete timing circuit or a digital signal processor (DSP), ASIC and/or hardware that includes one or more such systems, devices and components. The computer and/or IC may execute, control, or carry out instructions related to the above-described processes and operations. The computer and/or IC may calculate any variety of parameters or values that relate to the processes and operations described above.

[0076] FIG. 8 illustrates one example of a computer system 800 in which one or more aspects of the present disclosure may be implemented. As illustrated, computer system 800 includes a bus 801 or other communication mechanism for transmitting information and a hardware processor 802 coupled to bus 801 for processing the information. Processor 802, for example, may be a general purpose microprocessor.

[0077] The computer system 800 also includes a main storage device 803, such as random access memory (RAM) or other dynamic storage device, coupled to the bus 801 for storing information and instructions to be executed by the processor 802. Main storage device 803 may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 802. Such instructions, when stored on non-volatile storage media accessible to the processor 802, make the computer system 800 a special purpose machine that individually configured with the ability to perform operations specified in the instructions.

[0078] The computer system 800 further includes a read-only memory (ROM) 804 or other static data storage device coupled to a bus 801 for storing static information and instructions for the processor 802. A data storage device 805, such as a magnetic disk or an optical disk , can be provided and connected to the 801 bus to store information and instructions. The computer system 800 may couple via a bus 801 to a display 811, such as a liquid crystal display, to display information to a computer user. An input device 812, including alphanumeric and other keys, is coupled to a bus 801 to transmit information and command selections to the processor 802. The computer system 800 may further be coupled via the bus 801 to a cursor control means 813, such as a mouse, trackball, or cursor control keys for transmitting directional information and command selections to processor 802 and for controlling cursor movement on display 811.

[0079] The computer system 800 may implement the technologies described herein using custom hardware-based logic, one or more ASICs or FPGAs, firmware, and/or software logic that, in combination with the computer system, instructs or programs the computer system 800 in such a way that it is a special purpose machine. According to one embodiment, the technologies described herein are performed by the computer system 800 in response to the execution, by the processor 802, of one or more sequences of one or more instructions contained in the main storage device 803. Such instructions may be read into the main storage device. device 803 from another storage medium, for example, from storage device 805. Execution of the sequences of instructions contained in the main storage device 803 instructs the processor 802 to carry out the process steps described herein. In alternative embodiments, hardware-implemented circuitry may be used instead of or in combination with software instructions.

[0080] The term "storage media" as used herein means any non-volatile media that stores data and/or instructions that instruct a machine to operate in a particular manner. Such storage media may include non-volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage device 805. Volatile media includes dynamic storage media such as main storage device 803. Common forms of storage media include, for example, floppy disk, floppy disk, hard disk, semiconductor drive, magnetic tape or any other magnetic storage media, CD- ROM, any other optical storage media, any physical media with hole patterns, RAM, PROM and EPROM, flash EPROM, NVRAM, any other memory chip or cartridge.

[0081] Storage media are different from, but may be used in combination with, transmission media. Transmission media are involved in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wire, and fiber optics, including wires that contain bus 801. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared communications.

[0082] Various forms of media may be provided in carrying one or more sequences of one or more instructions to processor 802 for execution. For example, the instructions may initially be carried on a magnetic disk or on a semiconductor drive of a remote computer. The remote computer can load instructions into its dynamic storage device and send instructions over a communication line such as coaxial cable, optical fibers, etc. A communications interface 806 local to the computer system 800 may receive data over a communications link, and associated circuitry may place the data on a bus 801. The bus 801 carries data to main storage 803, from which processor 802 retrieves and executes instructions. Instructions received by main storage device 803 may optionally be stored on storage device 805 before or after execution by processor 802.

[0083] Communications interface 806 provides a bidirectional data connection to a network communications link 821 that connects to a local area network 822. For example, communications interface 806 may be an integrated services digital network (ISDN) card, a cable modem, a satellite modem or a modem to provide a data connection to the appropriate type of telephone line. As another example, communications interface 806 may be a local area network (LAN) card that provides a data connection to a compatible LAN. Wireless communication lines can also be implemented. In any such implementation, communications interface 806 sends and receives electrical, electromagnetic, or optical signals that carry digital data streams representing various types of information.

[0084] Network communication link 821 typically allows communication through one or more networks with other data communications devices. For example, the network link 821 may provide a connection through a local network 822 to a host computer 823 or to data communications equipment managed by an Internet service provider (ISP) 824 . ISP 824 in turn provides data services over the worldwide packet data network, now commonly referred to as "Internet 825". Local area network 822 and Internet 825 use electrical, electromagnetic, or optical signals that carry digital data streams. Signals through various networks and signals on network communication link 821 and communication interface 806 that carry digital data to/from computer system 800 are exemplary forms of transmission media.

[0085] The computer system 800 may send messages and receive data, including program code, through the network(s), network link 821, and communications interface 806. In the Internet example, server 826 may transmit requested code for an application program via Internet 825, ISP 824, local network 822, and communications interface 806. Received code may be executed by processor 802 as it is received and/or stored in storage device 805 or other non-volatile storage device for later execution.

[0086] Applications and Benefits

[0087] The above aspects of the present disclosure can provide generation of metadata that can be used to provide increased video quality, increased temporal stability, low computational requirements, and reduced latency.

[0088] Systems, methods, and devices in accordance with the present disclosure may take any one or more of the following configurations.

[0089] (1) A method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format to thereby generate converted video data; generating a plurality of initial metadata values for a frame of converted video data, the plurality of initial metadata values including a first metadata value corresponding to a first fixed value not calculated from content including the frame, a second metadata value corresponding to an average luminance value of the frame, and a third value metadata corresponding to a second fixed value not calculated from the content, wherein the first metadata value, the second metadata value and the third metadata value include information used by the decoder to prepare by rendering a decoded image on a display.

[0090] (2) The method according to (1), further comprising applying a spatial smoothing operation to the transformed video data before generating a plurality of initial metadata values.

[0091] (3) The method according to (2), wherein applying a spatial smoothing operation includes subsampling the converted video data.

[0092] (4) The method according to (2) or (3), wherein applying a spatial smoothing operation includes decimating the converted video data to a smaller size.

[0093] (5) The method according to any of (1) to (4), further comprising: outputting distributed video data including a portion of video data corresponding to a frame, a first metadata value, a second metadata value, and a third metadata value.

[0094] (6) The method according to any of (1) to (5), wherein the first fixed value corresponds to the minimum permissible brightness value of the content, and the second fixed value corresponds to the maximum permissible brightness value of the content.

[0095] (7) The method according to any of (1) to (6), wherein the predetermined format is a high dynamic range format based on a perceptual quantizer.

[0096] (8) A non-transitory computer-readable medium storing instructions that, when executed by a computer system processor, instruct the computer system to perform operations comprising a method according to any of (1) to (7).

[0097] (9) A video editing system, comprising: a storage device; and a processor configured to instruct the video editing system to: receive input video data, convert the input video data into a predetermined format thereby generating converted video data, generate a plurality of initial metadata values for a frame of converted video data, the plurality of initial metadata values including the first a metadata value corresponding to a first fixed value not calculated from the content including the frame, a second metadata value corresponding to an average luminance value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content.

[0098] (10) The video editing system according to (9), wherein the processor is configured to instruct the video editing system to: apply a spatial smoothing operation to the converted video data before generating a plurality of initial metadata values.

[0099] (11) The video editing system according to (10), in which the spatial smoothing operation includes downsampling the converted video data.

[0100] (12) The video editing system according to (10) or (11), in which the spatial smoothing operation includes decimating the converted video data to a smaller size.

[0101] (13) The video editing system according to any one of (9)-(12), wherein the processor is configured to further instruct the video editing system to: output distributed video data including a portion of video data corresponding to a frame, a first metadata value, a second a metadata value and a third metadata value.

[0102] (14) The video editing system according to any one of (9) to (13), wherein the first fixed value corresponds to the minimum permissible brightness value of the content, and the second fixed value corresponds to the maximum permissible brightness value of the content.

[0103] (15) The video editing system according to any one of (9) to (14), wherein the predetermined format is a high dynamic range format based on a perceptual quantizer.

[0104] (16) A method for editing video content, comprising: receiving input video data; converting the input video data into a predetermined format to thereby generate converted video data; generating a first metadata value for a frame of converted video data, the first metadata value corresponding to an average brightness value of the frame; defining a time window including the frame, wherein the time window has a predetermined length in time; calculating a maximum metadata range and a minimum metadata range over a time window to thereby generate a first filtered metadata value and a second filtered metadata value; and smoothing the first filtered metadata value and the second filtered metadata value to thereby generate a first smoothed and filtered metadata value and a second smoothed and filtered metadata value.

[0105] (17) The method according to (16), in which the predetermined length in time is four seconds.

[0106] (18) The method according to (16) or (17), in which the window is configured so as not to intersect a fast scene switch in the input video data.

[0107] (19) The method according to any of (16)-(18), further comprising applying a spatial smoothing operation to the transformed video data before generating the first metadata value.

[0108] (20) A non-transitory computer-readable medium storing instructions that, when executed by a computer system processor, instruct the computer system to perform operations comprising a method according to any of (16)-(19).

[0109] With respect to the processes, systems, methods, heuristic procedures, etc. described herein, it should be understood that although the steps of such processes, etc. described as being carried out according to some ordered sequence, such processes may be practiced with the described steps performed in an order different from the order described herein. Additionally, it should be understood that some steps may be performed concurrently, that other steps may be added, or that some steps described herein may be omitted. In other words, the process descriptions herein are provided for purposes of illustrating certain embodiments and should in no way be construed as limiting the claims.

[0110] Accordingly, it should be understood that the foregoing description is intended to be illustrative and not limiting. Many embodiments and uses other than the examples provided should become apparent upon reading the foregoing description. The scope should not be determined by reference to the foregoing description, but instead should be determined by reference to the appended claims, along with the full scope of equivalents to which such claims are authorized. It is intended and implied that future developments will be made in the technologies explained herein and that the disclosed systems and methods will be included in such future embodiments. In general, it should be understood that the application is subject to modification and variation.

[0111] All terms used in the claims are to be given their broadest reasonable construction and their ordinary meaning as understood by those skilled in the art described herein, unless otherwise expressly stated herein. In particular, the use of singular articles such as "a", "the", "said", etc., should be considered as listing one or more specified elements, unless the claim lists another express limitation.

[0112] An abstract of the disclosure is provided to enable the reader to quickly identify the nature of the technical disclosure. It is submitted with the understanding that it is not to be used to interpret or limit the scope or meaning of the claims. In addition, in the above detailed description, it can be seen that various features are grouped in various embodiments for the purpose of simplifying the disclosure. This disclosure should not be interpreted as indicating that the claimed embodiments include more features than are expressly set forth in each claim. On the contrary, as reflected in the accompanying claims, the subject matter of the invention is not all of the features of a single disclosed embodiment. Accordingly, the appended claims are hereby incorporated into the detailed description, each claim being independently claimed as subject matter.

Various aspects of the present invention may be taken into account from the following enumerated exemplary embodiments (EEE):

1. A method for editing video content, containing:

- receiving input video data;

- converting input video data into a predetermined format in order to thereby generate converted video data;

- generating a plurality of initial metadata values for a frame of converted video data, the plurality of initial metadata values including a first metadata value corresponding to a first fixed value not calculated from the content including the frame, a second metadata value corresponding to an average luminance value of the frame, and a third the metadata value corresponding to the second fixed value not calculated from the content,

- wherein the first metadata value, the second metadata value and the third metadata value include information used by the decoder to prepare a decoded image on the display by rendering.

2. A method according to EEE 1, further comprising applying a spatial smoothing operation to the transformed video data before generating a plurality of initial metadata values.

3. A method according to EEE 2, wherein applying a spatial smoothing operation includes downsampling the converted video data.

4. A method according to EEE 2 or EEE 3, wherein applying a spatial smoothing operation involves decimating the converted video data to a smaller size.

5. A method according to any one of EEE 1-4, further comprising:

- outputting distributed video data, including a portion of the video data corresponding to the frame, a first metadata value, a second metadata value, and a third metadata value.

6. A method according to any one of EEE 1-5, wherein the first fixed value corresponds to the minimum permissible brightness value of the content, and the second fixed value corresponds to the maximum permissible brightness value of the content.

7. A method according to any one of EEE 1-6, wherein the predetermined format is a high dynamic range format based on a perceptual quantizer.

8. A non-transitory computer-readable medium storing instructions that, when executed by a computer system processor, instruct the computer system to perform operations comprising a method according to any of EEE 1-7.

9. Video editing system containing:

- Memory device; And

- a processor configured to instruct the video editing system to:

- accept input video data,

- convert input video data into a predefined format in order to thereby generate converted video data,

- generating a plurality of initial metadata values for a frame of converted video data, the plurality of initial metadata values including a first metadata value corresponding to a first fixed value not calculated from the content including the frame, a second metadata value corresponding to an average luminance value of the frame, and a third the metadata value corresponding to the second fixed value not calculated from the content.

10. A video editing system according to EEE 9, wherein the processor is configured to instruct the video editing system:

- apply a spatial smoothing operation to the converted video data before generating a set of initial metadata values.

11. An EEE 10 video editing system in which the spatial anti-aliasing operation involves downsampling the converted video data.

12. A video editing system according to EEE 10 or EEE 11, in which the spatial anti-aliasing operation involves decimating the converted video data to a smaller size.

13. A video editing system according to any one of EEE 9-12, wherein the processor is configured to further instruct the video editing system:

- output distributed video data including a portion of video data corresponding to a frame, a first metadata value, a second metadata value, and a third metadata value.

14. A video editing system according to any one of EEE 9-13, wherein the first fixed value corresponds to a minimum permissible luminance value of the content, and the second fixed value corresponds to a maximum permissible luminance value of the content.

15. A video editing system according to any one of EEE 9-14, wherein the predefined format is a high dynamic range format based on a perceptual quantizer.

Claims (37)

1. A method for editing video content, containing stages in which: receive input video data; converting the input video data into a predetermined format that is different from the format of the input video data with respect to the dynamic range, thereby generating converted video data; applying a spatial smoothing operation to the converted video data; generating a plurality of initial metadata values for a frame of converted video data, wherein the plurality of initial metadata values includes a first metadata value corresponding to a first fixed value not calculated from the content including the frame, wherein the first fixed value corresponds to a minimum acceptable brightness value of the content, the second a metadata value corresponding to the average brightness value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content, wherein the second fixed value corresponds to the maximum allowed brightness value of the content, wherein the first metadata value, the second metadata value and the third metadata value include information used by the decoder to prepare a decoded image on a display by rendering. 2. The method of claim 1, wherein the predetermined format is a standard dynamic range format, wherein the input video data is optionally in a high dynamic range format. 3. The method of claim 1, wherein applying the spatial smoothing operation includes downsampling the converted video data. 4. The method of claim 1 or 3, wherein applying the spatial smoothing operation includes cutting down the converted video data to a smaller size. 5. Method according to any one of paragraphs. 1-4, also containing a step in which: outputting distributed video data including a portion of video data corresponding to a frame, a first metadata value, a second metadata value, and a third metadata value. 6. Method according to any one of paragraphs. 1-5, wherein the predefined format is a high dynamic range format based on a perceptual quantizer. 7. A non-transitory computer-readable medium storing instructions that, when executed by a computer system processor, instruct the computer system to perform operations comprising the method of any one of claims. 1-6. 8. Video editing system containing: Memory device; And a processor configured to instruct the video editing system to: accept input video data, convert the input video data into a predetermined format that is different from the format of the input video data with respect to the dynamic range, thereby generating converted video data, apply a spatial smoothing operation to the converted video data, generate a plurality of initial metadata values for a frame of converted video data, the plurality of initial metadata values including a first metadata value corresponding to a first fixed value not calculated from the content including the frame, wherein the first fixed value corresponds to a minimum acceptable luminance value of the content, a second a metadata value corresponding to the average brightness value of the frame, and a third metadata value corresponding to a second fixed value not calculated from the content, wherein the second fixed value corresponds to the maximum allowed brightness value of the content, 9. The video editing system of claim 8, wherein the predefined format is a standard dynamic range format, wherein the input video data is optionally in a high dynamic range format. 10. The video editing system of claim 8, wherein the spatial smoothing operation includes downsampling the converted video data. 11. The video editing system of claim 8 or 10, wherein the spatial smoothing operation includes decimating the converted video data to a smaller size. 12. Video editing system according to any one of paragraphs. 8-11, wherein the processor is also configured to instruct the video editing system: output distributed video data including a portion of video data corresponding to a frame, a first metadata value, a second metadata value, and a third metadata value. 13. Video editing system according to any one of paragraphs. 8-12, wherein the predetermined format is a high dynamic range format based on a perceptual quantizer. 14. A method for editing video content, containing the stages of: receive input video data; converting the input video data into a predetermined format to thereby generate converted video data; applying a spatial smoothing operation to the converted video data; generating a first metadata value for a frame of converted video data, the first metadata value corresponding to the average brightness value of the frame; determining a time window including the frame, the time window having a predetermined length in time; calculating a maximum metadata range and a minimum metadata range over a time window to thereby generate a first filtered metadata value and a second filtered metadata value; And smoothing the first filtered metadata value and the second filtered metadata value to thereby generate the first smoothed and filtered metadata value and the second smoothed and filtered metadata value. 15. The method of claim 14, wherein the predetermined length in time is four seconds. 16. Method according to any one of paragraphs. 14 or 15, in which the window is configured so as not to intersect a fast scene switch in the input video data. 17. A non-transitory computer-readable medium storing instructions that, when executed by a computer system processor, instruct the computer system to perform operations comprising the method of any one of claims. 14-16.