GB2619096A

GB2619096A - Enhancement interlacing

Info

Publication number: GB2619096A
Application number: GB2207887.7A
Authority: GB
Inventors: Ciccarelli Lorenzo; Maurer Florian; Morena Amaya; Dorovic Katarina; Ferrara Simone
Original assignee: V Nova International Ltd
Current assignee: V Nova International Ltd
Priority date: 2022-05-27
Filing date: 2022-05-27
Publication date: 2023-11-29
Also published as: WO2023227911A1; GB202207887D0

Abstract

A method of encoding a signal for use in reproducing an interlaced or non-interlaced video. The method comprises obtaining an initial rendition of an input signal having a first colour range and converting the initial rendition of the input signal from the first colour range to a second colour range to generate a colour converted rendition. This is followed by processing the colour converted rendition to generate an interlaced colour converted rendition and sending it to a base encoder. A decoded rendition of an encoded rendition of the interlaced colour converted rendition is obtained from the base encoder and is processed to generate a reconstructed decoded rendition, the reconstructed decoded rendition being a de-interlaced rendition of the decoded rendition. The reconstructed decoded rendition is compared to a version of the initial rendition to generate a set of residuals and then instructing an encoding of the set of residuals using an enhancement encoder to generate an encoded enhancement signal for later decoding the residuals for combination with a decoded version of a base encoded signal so as to reconstruct a rendition of the initial rendition of the input signal.

Description

ENHANCEMENT INTERLACING

The encoding techniques in the following specification are particularly suited to be used with existing Low Complexity Enhancement Video Coding (LCEVC) techniques.

A standard specification for LCEVC is provided in the Text of ISO/IEC 23094-2 Ed 1 Low Complexity Enhancement Video Coding published in November 2021, and many possible implementation details of LCEVC are described in patent publications WO 2020/188273 and WO 2020/188229. Each of these earlier documents is incorporated herein by reference.

Broadly speaking, LCEVC enhances the reproduction fidelity of a decoded video after encoding and decoding using an existing codec. This is achieved by combining a base layer with an enhancement layer, where the base layer contains the video encoded using the existing codec, and the enhancement layer indicates a residual difference between the original video and a predicted decoded video produced by decoding the base layer using the existing codec. The enhancement layer can be combined with the decoded base layer to more accurately reproduce the original video.

Colour conversion within a hierarchical video coding scheme has previously been described in W02020/074896, the contents of which are incorporated herein by reference.

Examples of implementation of LCEVC may be described in W02022/023747 and W02022/023739, which are incorporated herein by reference.

It remains an objective to effectively and efficiently integrate enhancement coding into existing ecosystems. Examples according to the present disclosure provide an integration of enhancement coding (i.e. LCEVC) in current TV specifications to enhance an available interlaced 1080i SDR stream to a progressive 1080p HDR stream to improve current TV use cases, particularly within TV hardware legacy ecosystems. One of the key advantages of enabling a conversion from an SDR interlaced video to an HDR progressive video lies in the ability to move the production of the input video from interlaced to progressive, thus simplifying the production of videos for those services that currently needs to produce both an interlaced and a progressive sequence due to the fact that many legacy systems still support interlace whereas more modern ones are progressive-based. In this sense, this solution simplifies the technical workflow for the input video, and uniform it towards a single format (typically HDR progressive 4K or higher).

It should be noted that solutions presented herein are also suitable for enhancing a lower resolution interlaced video (e.g., 576i or other).

In a specific current use case, in the majority of Brazil the TV 2.0 standard is used for broadcasting as of today. While TV 3.0 will replace this in the near future, it remains an aim to enable the distribution of an enhancement layer on top of a TV 2.0 base to deliver a 10 bit HDR stream to supported receivers. TV 2.0 currently uses AVC 1080i30 SDR (BT.709 colour space) and will likely be required for backwards compatibility.

Examples disclosed herein enable a TV 2.0 base to be enhanced with LCEVC for playback as 1080p60 HDR with 81.2020 colour space.

Initial research activities around deinterlacing and HDR/colour space conversion have investigated the feasibility of using enhancement coding and leveraging that deinterlacing and tone mapping blocks are not necessarily standardized and an available algorithm can be different depending on the chipset.

The number of supported decoding devices (TVs, Set Top Boxes (STBs)) and their similarity (i.e., deinterlacing process, position of colour space conversion block within framework) can impact decoder development.

Figure 1 shows an example of an encoding system according to the present 25 disclosure.

In some examples, an input sequence, or an input frame is obtained. The input sequence may be obtained with various resolutions, frame rates, SDR/HDR characteristics, bit depth and colour gamut. In some examples the input sequence in obtained at a resolution of 2160p60, i.e. in 4K resolution. The input sequence may be a 10 bit HDR input signal which makes use of a HDR colour space such as BT.2020.

Throughout the present description, we will refer to colour conversion and the conversion from one definition or type of colour to another At time we will use the terms gamut, space and range. These are not necessarily interchangeable but where the text describes a conversion, for example a colour space conversion, a colour range conversion or a colour gamut conversion, it will be understood that all conversions may be provided in the same step, either separately, independently or at the same time. For example, where the text describes a colour conversion, the conversion may be from HDR to SDR, from 10 bit to 8bit, a narrowing or widening of the colour space, e.g. BT202 to 709, or narrowing/widening colour gamut. For brevity in places only one term is used but it should be understood that all conversions may be performed at the same time or separately, or at different locations in the process. For example, a range may be changed before a scaling and a space changed at another point in the process.

The input sequence is optionally provided to a downsampler, which converts the resolution of the input frame to a lower resolution. In some examples, the resolution may be downsampled from 2160p to 1080p. However, these are just examples, and it is to be understood that the downsampler converts the input frame from a relatively high resolution to a relatively low resolution.

HDR mapping may then optionally be performed on the downsampled signal. This may include providing HDR metadata to an enhancement encoder. For example, the enhancement encoder may be an LCEVC encoder, and the HDR metadata 25 may be static HDR10 metadata.

The input frame is then converted. This may comprise a colour conversion to a different colour gamut. This may additionally or alternatively comprise converting dynamic range, for example the downsampled input signal may be converted from a HDR colour space to an SDR colour space. In some examples, this may involve converting the downsampled signal from BT.2020 to BT.709. In general, this can be seen as a 'down-conversion' because it is converting a first colour range to a second colour range, wherein the first colour range has a larger volume (e.g. a 'wider' colour space and/or larger dynamic range) than the second colour range. The 'down conversion' may thus be seen as reducing a volume of the colour range of the signal.

Optionally, the bit depth of the signal may be lowered. For example, the bit depth may be decreased from 10 bit to 8 bit.

In examples, the input frame is provided to a colour conversion block for colour conversion (i.e. 'down-conversion and in response a colour converted frame is received. In this example, the input frame has a wider colour gamut (i.e. a larger area of colour space) than the colour converted frame. The conversion may be provided by an independent entity or module to the entity or module providing the coding process and method.

After colour conversion, the colour converted signal is interlaced. In examples, the colour converted signal is provided to an interlacer for interlacing, and in response, an interlaced signal is received.

Such processing (i.e. conversion and then interlacing) may provide for a high quality final reconstructed output image that does not utilise the enhancement layer but also provide for a final reconstructed output image that does utilise the enhancement layer.

The interlaced signal is provided to a base codec for encoding and decoding. In some examples the base codec may be an AVC 8 bit encoder. Different base codecs may of course be used to the same effect. The base codec may produce a base signal which is transmitted as part of a bitstream. Example base codecs include MPEG standards such as AVC/H.264, HEVC/H.265, etc. as well as nonstandard algorithm such as VP9, AV1, and others. In examples, the method may include performing the base encoding and decoding. However, base encoding and decoding is often performed by specialist hardware block, and thus we describe the step as providing a signal to a base codec for encoding and decoding.

A decoded rendition of the downsampled, colour converted and interlaced signal is then de-interlaced. Functionally, this may be the inverse of the interlacing operation performed before encoding. Accordingly, the decoded signal is converted from 1080i30 to 1080p60. Thus, in examples, described examples may include de-interlacing the decoded rendition of the downsampled, colour converted and interlaced signal. However, de-interlacing may be performed by a specialist (e.g. hardware) block, and thus in described examples we instead describe providing the decoded rendition of the downsampled, colour converted and interlaced signal to a de-interlacer, and in response, receiving a decoded de-interlaced rendition of the downsampled, colour converted and interlaced signal.

It should be noted that each of the blocks illustrated in the figures may be performed by the coding process or independent or separable modules or entities. For example, modules existing on pre-existing SoC may be utilised in the pipeline and their functionality managed by instructing their function or by passing the module the appropriate data and receiving an output in response.

After de-interlacing, the decoded signal is colour converted. Functionally, the colour range conversion may be the inversion of the previous colour range conversion. That is, the decoded signal is colour range converted back to the colour range of the original input frame. In some examples, the signal is converted from an SDR signal (e.g. BT.709) to a HDR signal (e.g. BT.2020). In general, this can be seen as a 'up-conversion' because it is converting a third colour range to a fourth colour range, wherein the fourth colour range has a larger volume than the third colour range. The 'up-conversion' can thus be seen as increasing the colour range of the signal.

Many different techniques to convert from one colour range to another colour space (i.e. 'up-conversion' and 'down-conversion') are known, e.g. Recommendation ITU-R BT.2087 https://www.itu.int/rec/R-REC-BT.2087-0-201510-I/en, Recommendation ITU-R BT. 709 https://www.itu.int/rec/R-RECBT.709, Recommendation ITU-R BT.2020 https://www.itu.int/rec/R-REC-BT.2020, Recommendation ITU-R BT.2100 https://www.itu.int/rec/R-REC-BT.2100, are all ways to convert one colour range to another colour range.

The de-interlaced and colour converted rendition of the decoded signal may be referred to as a reconstructed signal. Thus, in examples, described examples may include colour converting the decoded de-interlaced rendition of the downsampled, colour converted and interlaced signal. However, colour conversion may be performed by a specialist (e.g. hardware) block, and thus in described examples we instead describe providing the decoded de-interlaced rendition of the downsampled, colour converted and interlaced signal to a colour space converter, and in response, receiving a colour converted decoded de-interlaced rendition of the downsampled, colour converted and interlaced signal.

In examples, the de-interlacing and colour conversion may be asymmetric, that is, they may use different approaches or parameters. This may introduce different errors or artifacts to be corrected.

Optionally, the bit depth of the encoded signal may be converted (e.g. increased). In examples the bit depth is converted from 8 bit to 10 bit, this may introduce different errors or artifacts to be corrected.

A residual signal is then generated by taking the difference between a rendition of the input signal and the reconstructed signal. The residual signal is provided to an enhancement encoder which may also be configured to receive the HDR metadata in the optional configuration.

The enhancement encoder generates an enhancement signal which may be transmitted with the base signal in a bitstream. The enhancement signal may optionally further include supplementary HDR data, for example HDR10 data as SEINUI data.

It is to be noted that the steps of downsampling, HDR mapping, and the base coding may be found in conventional encoding systems and so the modules may be re-used. However, the process of colour converting and interlacing the signal before passing the signal to the base coder, and the subsequent de-interlacing and inverse colour space conversion, as well as generating residuals and performing an enhancement encoding are specific to the enhancement encoding pipeline. In particular, the steps may be particular to LCEVC processing.

In examples, colour space conversation takes place before interlacing in the cdownscale' path and after de-interlacing in the 'upscale' or 'reconstruction' path, that is, after the base encoding and decoding. In this way, colour range conversion maximises the correction of the de-interlaced signal to improve overall reconstructed video quality.

It will be understood that throughout, where a resolution, bit depth or frame rate is presented, this is merely exemplary and further resolutions are possible.

It should be noted that the examples herein do not include a typical LCEVC spatial upscaling step in the encoding pipeline. It will of course be understood that the spatial upscaling included within LCEVC may also be utilised with the examples herein so that the colour space converted de-interlaced signal is further upscaled for combination with a higher resolution input signal to generate a set of residuals.

This may be in addition to, or instead of the set of residuals encoded as illustrated in Figure 1.

The bitstream is transmitted and received by a decoding system. An example of a decoding system is illustrated in Figure 2. The bitstream typically comprises a base signal and an enhancement signal is received by the decoding system. The base signal is passed to a base decoder, and the enhancement signal is passed to an enhancement decoder. In certain examples, the base decoder may be an 8 bit AVC decoder and the enhancement decoder could be a 10 bit LCEVC decoder.

The base signal is decoded by the base decoder and de-interlaced. For example, the decoded rendition of the signal may be temporally scaled from a resolution of 1080i30 to 1080p60. In examples, this is achieved by passing the decoded rendition of the signal to a de-interlacer (e.g. a de-interlacing hardware block) and in response receiving a de-interlaced decoded signal.

After the de-interlacing, the signal undergoes colour range conversion. For example, the signal may be converted from an SDR colour space to a HDR colour space. Additionally or alternatively the signal may be converted from a narrow colour gamut to a wide colour gamut. In a specific example, the colour gamut may be converted from BT.709 to BT.2020 or BT.2087. In examples, this is achieved by passing the decoded de-interlaced rendition of the signal to a colour range converter (e.g. a colour gamut conversion hardware block and/or a dynamic range converter block) and in response receiving a colour range converted de-interlaced decoded signal.

Optionally, the bit depth of the signal may be upscaled for example from 8 bit to 10 bit.

In some examples, a spatial upsampling may be performed after the signal is converted to a HDR colour space. It should be noted that solutions presented herein are therefore also suitable for enhancing a lower resolution interlaced video (e.g., 576i or other). The spatial upsampling may be in accordance with existing LCEVC technology and may be located at various different points of the pipeline to achieve the intended effect. In these examples, the upsampled frame is compared with a frame of a similar resolution input video to generate a set of residuals for encoding using the enhancement coding technology, e.g. LCEVC.The colour converted rendition of the decoded rendition of the base signal is passed to the enhancement decoder The enhancement decoder combines the colour converted rendition of the decoded rendition of the base signal with the enhancement signal to produce an enhanced video signal, in examples, this provides an advantage that the enhancement layer can correct for artefacts/errors introduced in the colour range conversion and/or the de-interlacing process. In other examples, colour range conversion is performed (i.e. by a decoder) after the enhancement layer is applied to the base layer, this may provide for a process that is compatible with a wide range of decoding devices.

The enhancement decoder may also produce a stream of HDR metadata, for example static HDR10 metadata, if such metadata is included along with the enhancement stream.

The video signal and metadata are passed to an output device. For example, the video signal and the metadata are passed to a TV for display. In some examples the TV is a TV which supports the TV 2.5 specification. That is, the TV supports some aspects of HDR.

The enhanced video signal may have a resolution of 1080p, have HDR10 picture quality and possess a colour gamut of BT.2020.

To ensure backwards compatibility, in some examples, the decoded base signal may be passed to an output device without being upsampled or colour gamut converted. For example, this may be required for TVs adhering to the TV 2.0 specification. In this case, where no enhancement signal is used, the output signal is provided at a resolution of 1080i30, in SDR and a colour gamut of BT.709.

It is to be noted that the base codec, the upsampling and the colour conversion may be part of an existing video decoder pipeline, for example as implemented by an SoC. The enhancement decoder is specific to an enhancement decoding pipeline, such as an LCEVC pipeline.

Certain notable features of the examples of the present disclosure include: - Interlacing and de-interlacing.

- In examples, there is upsampling/downsampling within the enhancement coding pipeline. In the above example, downsampling is optionally performed before the colour conversion/interlace/deinterlace/conversion pipeline -but not within that pipeline. This is somewhat unusual because typically if downsampling occurs in the pipeline then an upsampling would be expected somewhere else in the pipeline (which is then compared to obtain a set of residuals).

-The combination and order of de-interlacing and SDR conversion to HDR is notable, in particular (at the decoder) de-interlacing is configured before colour range conversion.

In summary, the following are exemplary steps of proposed examples: a. Colour convert an input frame, optionally a rendition of the input frame, (e.g. spatially downsampled rendition). The conversion may, for example, be from HDR to SDR, further optionally converted from 10 bit to 8 bit.

b. Interlace the (colour converted) frame.

c. Send the interlaced colour converted (i.e. SDR) frame to a base codec (to encode and decode). Further, and in response to said sending, a decoded encoded rendition of the interlaced colour converted frame may be received.

d. Deinterlace the decoded encoded rendition of the interlaced colour converted frame.

e. Colour convert the de-interlaced frame to generate a 'reconstructed' frame. In practice, this colour conversion may be thought of functionally as the reverse of the previous colour conversion.

f. Generate residuals between the 'reconstructed' frame and (rendition of) the input frame.

g. Encode the residuals. For example, encoding using an enhancement encoder such as LCEVC.

The residuals may be encoded in a step of generating an encoded enhancement signal for the encoded video signal, the encoded enhancement signal comprising one or more layers of residual data, the residual data being generated based on a comparison of data derived from the decoded video signal and data derived from an input video signal.

It should be noted that, although the steps above describe converting from SDR to HDR at the encoder after de-interlacing and before computing the residuals, in implementations the conversion may be performed after the residual computation (for example the hardware block that does the SDR-to-HDR conversion in the decoder may occur on the output). This may provide for a process that is compatible with a wide range of decoding devices.

In examples, there may be further optional steps, such as: h. Encode HDR metadata alongside the residuals.

i. Downsample the input frame to obtain the 'rendition of' the input frame.

In examples, the input frame may be 10 bit, and the colour range conversion may comprise converting from 10 bit to 8 bit, and optionally vice versa. The first colour range conversion may be from a first space (e.g. 3T2020) to a second space (e.g. BT709), and the second conversion may convert from the second space (e.g. BT709) to the first space (e.g. BT2020). In this way the conversion may be symmetrical, but in other examples, it may be asymmetrical.

Implementation of the above is non-trivial for many reasons, not least because, in practical embodiments, the de-interlacer at the decoder side is often different to the interlacer at the encoder.

Figures 3 to 11 illustrate example variations of the steps of Figures 1 and 2 In Figure 3, HDR metadata is sent from the input sequence to the enhancement encoder for combination into the bitstream. In Figure 3, the colour range conversion is an SL-HDR1 decomposition. The input sequence undergoes SL-HDR1 decomposition with the SL-HDR1 metadata being provided to the AVC encoding and decoding step, as well as the SL-HDR1 reconstruction step being performed after de-interlacing. As with Figure 1, after the colour range conversion of the SL-HDR1 decomposition, the signal is interlaced and provided to an AVC encoder. The encoded version of the signal is output and decoded. The decoded version is provided to a deinterlacer and SL-HDR1 reconstruction step to reconstruct the video before comparison with the input sequence to create the residuals for LCEVC encoding. Figure 4 illustrates a corresponding decoding process in which SL-HDR1 metadata is provided from the AVC decoder to the colour range conversion step, i.e. the SL-HDR1 reconstruction, performed after de-interlacing.

In the variations of Figures 5 and 6, a post-production step occurs after downsampling which provides the static HDR10 metadata to the LCEVC encoder.

The colour range conversion is an HDR decomposition with the metadata being provided to the base encoder. At the decoder side, like Figure 4, the AVC decoder provides the HDR metadata to an HDR reconstruction step occurring after de-interlacing.

Figure 7 illustrates a decoding variation in which HDR reconstruction is performed on the base decoded signal and provided to the TV separately.

Figure 8 illustrates an 8 bit approach. A rendition of the input signal after colour range conversion is combined with the de-interlaced base decoded rendition to generate the residuals. In this example, in the reconstruction process, the colour range is not recreated, but instead the residuals are formed using the output of the colour range conversion before the interlace step.

Figure 9 illustrates a decoding of the 8 bit approach of Figure 8, in which colour range conversion occurs after the enhancement decoding.

Figures 10 and 11 illustrate decoding modes in which the de-interlacing and colour gamut conversion are selectively disabled.

Colour depth may also be referred to as bit depth. Colour depth may represent number of total bits used to indicate a colour of a single pixel (bpp), e.g. in a bit-mapped image or video frame buffer. Colour depth may represent a number of bits used for each of the (e.g. red, green and blue) colour components that make up a single pixel. A colour depth determines the total/maximum amount of distinct colours that it can display. This does not mean that the image necessarily uses all of these available colours, but that it can instead specify colours with that level of precision.

A colour space may also be referred to as a colour model or a colour system.

A colour range may be referred to as a colour volume. A colour range may therefore comprise and/or be associated with one or more of: a dynamic range (i.e. of the colour range), a colour gamut (i.e. of the colour range), a colour depth (i.e. of the colour range). A colour range may therefore determine which specific colours can be displayed and also a luminosity, which will have an effect on its "brightness", and "colourfulness" (intensity and saturation). A colour range may further determine how many distinct colour variations (tones/shades) can be viewed. A larger colour gamut may be referred to a wider colour gamut.

In examples, we describe a system comprising a modified downscaler/downsampler and a modified upscaler/upsampler. The modified downsampler/downscaler is configured to: downscale a colour range (e.g. a colour volume) of a signal (e.g. frame of video) to produce a downscaled signal (e.g. input video); and temporally downsample the downscaled signal to produce a downscaled downsampled signal. The modified downscaler may be configured to input the downscaled downsampled signal to a base codec, which in turn returns a decoded encoded rendition of what is input into the base codec. The modified upsampler/upscaler is configured to: upscale a colour range (e.g. a colour volume) of a signal to produce a upscaled signal; and temporally upsample the upscaled signal to produce a upscaled upsampled signal. The system may be configured to compare the a upscaled upsampled signal with a rendition of the (e.g. input) signal to generate residuals. The system may be configured to encode the residuals, optionally using a LCEVC encoding scheme.

We describe a corresponding system, the corresponding system in communication with the system and configured to receive the encoded stream.

The corresponding system comprises the modified upscaler/upsampler. The corresponding system comprises a decoder configured to: decode, using a base decoder, a base signal; upsample/upscale, using the modified upscaler/upsampler, the decoded base signal; decode the encoded residual enhancement signal; and combine the decoded residuals with the upsampled/upscaled base signal.

In all examples described herein, the base video may be output together with, or separately from the LCEVC encoded signal. That is, the base may be encoded with the LCEVC signal or the LCEVC may be encoding the residuals as a separate signal for independent transmission and the LCEVC coder does not include the base signal in the output enhancement sequence.

In certain cases, a base codec may be used. The base codec may comprise an independent codec that is controlled in a modular or "black box" manner. The methods described herein may be implemented by way of computer program code that is executed by a processor and makes function calls upon hardware and/or software implemented base codecs.

In general, the term "residuals" as used herein refers to a difference between a value of a reference array or reference frame and an actual array or frame of data. The array may be a one or two-dimensional array that represents a coding unit. For example, a coding unit may be a 2x2 or 4x4 set of residual values that correspond to similar sized areas of an input video frame. It should be noted that this generalised example is agnostic as to the encoding operations performed and the nature of the input signal. Reference to "residual data" as used herein refers to data derived from a set of residuals, e.g. a set of residuals themselves or an output of a set of data processing operations that are performed on the set of residuals. Throughout the present description, generally a set of residuals includes a plurality of residuals or residual elements, each residual or residual element corresponding to a signal element, that is, an element of the signal or original data. The signal may be an image or video. In these examples, the set of residuals corresponds to an image or frame of the video, with each residual being associated with a pixel of the signal, the pixel being the signal element. Examples disclosed herein describe how these residuals may be modified (i.e. processed) to impact the encoding pipeline or the eventually decoded image while reducing overall data size. Residuals or sets may be processed on a per residual element (or residual) basis, or processed on a group basis such as per tile or per coding unit where a tile or coding unit is a neighbouring subset of the set of residuals. In one case, a tile may comprise a group of smaller coding units. A tile may comprise a 16x16 set of picture elements or residuals (e.g. an 8 by 8 set of 2x2 coding units or a 4 by 4 set of 4x4 coding units). Note that the processing may be performed on each frame of a video or on only a set number of frames in a sequence.

In general, each or both enhancement streams may be encapsulated into one or more enhancement bitstreams using a set of Network Abstraction Layer Units (NALUs). The NALUs are meant to encapsulate the enhancement bitstream in order to apply the enhancement to the correct base reconstructed frame. The NALU may for example contain a reference index to the NALU containing the base decoder reconstructed frame bitstream to which the enhancement has to be applied. In this way, the enhancement can be synchronised to the base stream and the frames of each bitstream combined to produce the decoded output video (i.e. the residuals of each frame of enhancement level are combined with the frame of the base decoded stream). A group of pictures may represent multiple NALUs.

The following sets out particularly preferred examples of the present disclosure as a set of numbered clauses. It will be understood that these are examples helpful in understanding the invention.

Numbered Clause 1. A method of encoding a signal for use in reproducing an interlaced or non-interlaced video, comprising: obtaining an initial rendition of an input signal having a first colour range; converting the initial rendition of the input signal from the first colour range to a second colour range to generate a colour converted rendition; processing the colour converted rendition to generate an interlaced colour converted rendition, the interlaced colour converted rendition being an interlaced rendition of the colour converted rendition; sending the interlaced colour converted rendition to a base encoder; obtaining a decoded rendition of an encoded rendition of the interlaced colour converted rendition from the base encoder; processing the decoded rendition to generate a reconstructed decoded rendition, the reconstructed decoded rendition being a de-interlaced rendition of the decoded rendition; comparing the reconstructed decoded rendition to a version of the initial rendition to generate a set of residuals,; and, instructing an encoding of the set of residuals using an enhancement encoder to generate an encoded enhancement signal for later decoding the residuals for combination with a decoded version of a base encoded signal to reconstruct a rendition of the initial rendition of the input signal.

Numbered Clause 2. A method according to Numbered Clause 1, wherein the reconstructed decoded rendition and/or the version of the initial rendition use a colour range different from the decoded rendition.

Numbered Clause 3. A method according to Numbered Clause 1 or 2, wherein the step of comparing comprises comparing the reconstructed decoded rendition to the colour converted rendition.

Numbered Clause 4. A method according to Numbered Clause 1 or 2, wherein the step of processing the decoded rendition comprises: converting a de-interlaced decoded rendition from a third colour range to a fourth colour range to generate the reconstructed decoded rendition.

Numbered Clause 5. A method according to Numbered Clause 4, wherein the second colour range is the same as the third colour range.

Numbered Clause 6. A method according to Numbered Clause 4 or 5, wherein the first colour range is the same as the fourth colour range.

Numbered Clause 7. A method according to any of Numbered Clauses 4 to 6, wherein the conversion from the third colour range to the fourth colour range is the reverse as the conversion from the first colour range to the second colour range.

Numbered Clause 8. A method according to any of Numbered Clauses 4 to 7 wherein the fourth colour range comprises a larger colour volume than the third colour range.

Numbered Clause 9 A method according to any of Numbered Clauses 4 to 8 wherein converting the de-interlaced decoded rendition from the third colour range to the fourth colour range comprises increasing the colour volume.

Numbered Clause 10. A method according to any preceding Numbered Clause, wherein the input signal is a frame of an input video.

Numbered Clause 11. A method according to any preceding Numbered Clause, wherein the step of an initial rendition of an input signal having a first colour range comprises: receiving the input signal; and, spatially downsampling the input signal.

Numbered Clause 12. A method according to any preceding Numbered Clause, wherein the method further comprises: performing HDR mapping on the initial rendition to generate a set of HDR metadata; and, providing the HDR metadata to the enhancement decoder for encoding with the encoded enhancement signal.

Numbered Clause 13. A method according to any preceding Numbered Clause, wherein the enhancement encoder is LCEVC.

Numbered Clause 14. A method according to any preceding Numbered Clause, wherein the step of converting the initial rendition from the first colour range to a second colour range is a conversion that decreases a dynamic range of the first colour space, optionally wherein the step of converting the initial rendition from the first colour range to a second colour range is a conversion from HDR to SDR.

Numbered Clause 15. A method according to any preceding Numbered Clause, wherein the step of converting the initial rendition from the first colour range to a second colour range is a conversion from a first colour space to a second colour space, wherein the first colour space comprises a wider colour gamut than the second colour space.

Numbered Clause 16. A method according to any preceding Numbered Clause, wherein the first colour range has a higher colour depth than the second colour range, in particular wherein the first colour range is a 10 bit colour range and the second colour range is an 8 bit colour range.

Numbered Clause 17. A method according to any preceding Numbered Clause wherein the first colour range comprises a larger colour volume than the second colour range.

Numbered Clause 18. A method according to Numbered Clause 17, wherein converting the initial rendition of the input signal from the first colour range to a second colour range to generate a colour converted rendition comprises one or more of: decreasing a colour depth of the first colour space; decreasing a colour gamut of the first colour space; and decreasing a dynamic range of the first colour space.

Numbered Clause 19. A method according to any preceding Numbered Clause, wherein the colour ranges comprise colour space data.

Numbered Clause 20. A method according to any preceding Numbered Clause, wherein the colour ranges comprise dynamic range data.

Numbered Clause 21. A method according to any preceding Numbered Clause, wherein the colour ranges comprise colour gamut data.

Numbered Clause 22. A method of decoding an encoded enhancement signal for reproducing a rendition of a video signal, the method comprising: obtaining an encoded signal comprising an encoded enhancement signal and a base encoded signal; passing the base encoded signal to a base decoder to generate an interlaced base decoded video signal; processing the base decoded video signal to generate a reconstructed rendition of the video signal, the reconstructed rendition being a de-interlaced rendition of the base decoded video signal; combining the reconstructed rendition with a set of residuals decoded from the encoded enhancement stream to generate a reconstructed output video, wherein the reconstructed output video is in a sixth colour range different from the base decoded video signal in a fifth colour range.

Numbered Clause 23. A method according to Numbered Clause 22, wherein the step of combining comprises converting the reconstructed rendition from the fifth colour range to the sixth colour range.

Numbered Clause 24. A method according to Numbered Clause 22, wherein the method further comprises: converting a de-interlaced decoded rendition of the video signal from the fifth colour range to the sixth colour range to generate the reconstructed rendition of the video signal.

Numbered Clause 25 The method according to any one of Numbered Clauses 22 to 24, wherein the sixth colour range comprises one or more of: a larger colour depth than the fifth colour range; a larger colour gamut than the fifth colour range; and a larger dynamic range than the fifth colour range.

Numbered Clause 26. A method according to any of Numbered Clauses 22 to 25, further comprising: outputting the interlaced base decoded video.

Numbered Clause 27. A method according to any of Numbered Clauses 22 to 26, wherein the fifth colour range is the same as a third colour range, the third colour range being used to recontruct a rendition of the video signal during encoding Numbered Clause 28. A method according to any of Numbered Clauses 22 to 27, wherein the sixth colour range is the same as a fourth colour range, the fourth colour range being used to convert a de-interlaces base decoded rendition of a video signal during encoding.

Numbered Clause 29. A method according to any of Numbered Clauses 22 to 28, further comprising: obtaining HDR metadata from the encoded enhancement signal and providing the HDR metadata along with the reconstructed output video.

Numbered Clause 30. A method according to any of Numbered Clauses 22 to 29, wherein the encoded enhancement signal is encoded using LCEVC.

Numbered Clause 31. A method according to any of Numbered Clauses 22 to 30, wherein the step of converting comprises converting the de-interlaced decoded rendition of the video signal from SDR to HDR.

Numbered Clause 32. A method according to any of Numbered Clauses 22 to 31, wherein the step of converting comprises converting the de-interlaced decoded rendition of the video signal from one colour space to another colour space.

Numbered Clause 33. A method according to any of Numbered Clauses 22 to 32, wherein the colour ranges comprise colour space data.

Numbered Clause 34. A method according to any of Numbered Clauses 22 to 33, wherein the colour ranges comprise dynamic range data.

Numbered Clause 35. A method according to any of Numbered Clauses 22 to 34, wherein the colour ranges comprise colour gamut data.

Numbered Clause 36. An encoder configured to perform the method of any of Numbered Clauses 1 to 21.

Numbered Clause 37. A decoder configured to perform the method of any of Numbered Clauses 22 to 35.

Numbered Clause 38. A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, cause the processors to perform a method according to any of Numbered Clauses 1 to 21 or 22 to 35.

Numbered Clause 39. A method substantially as herein described or a method or product as shown in any one of Figures 1 to 11.

Claims

CLAIMS1 A method of encoding a signal for use in reproducing an interlaced or non-interlaced video, comprising: obtaining an initial rendition of an input signal having a first colour range; converting the initial rendition of the input signal from the first colour range to a second colour range to generate a colour converted rendition; processing the colour converted rendition to generate an interlaced colour converted rendition, the interlaced colour converted rendition being an interlaced rendition of the colour converted rendition; sending the interlaced colour converted rendition to a base encoder; obtaining a decoded rendition of an encoded rendition of the interlaced colour converted rendition from the base encoder; processing the decoded rendition to generate a reconstructed decoded rendition, the reconstructed decoded rendition being a de-interlaced rendition of the decoded rendition; comparing the reconstructed decoded rendition to a version of the initial rendition to generate a set of residuals; and, instructing an encoding of the set of residuals using an enhancement encoder to generate an encoded enhancement signal for later decoding the residuals for combination with a decoded version of a base encoded signal to reconstruct a rendition of the initial rendition of the input signal.
2. A method according to claim 1, wherein the reconstructed decoded rendition and/or the version of the initial rendition use a colour range different from the decoded rendition.
3. A method according to claim 1 or 2, wherein the step of comparing comprises comparing the reconstructed decoded rendition to the colour converted rendition.
4. A method according to claim 1 or 2, wherein the step of processing the decoded rendition comprises: converting a de-interlaced decoded rendition from a third colour range to a fourth colour range to generate the reconstructed decoded rendition.
5. A method according to claim 4, wherein the second colour range is the same as the third colour range.
6. A method according to claim 4 or 5, wherein the first colour range is the same as the fourth colour range.
7. A method according to any of claims 4 to 6, wherein the conversion from the third colour range to the fourth colour range is the reverse as the conversion from the first colour range to the second colour range.
8. A method according to any of claims 4 to 7 wherein the fourth colour range comprises a larger colour volume than the third colour range.preferably wherein converting the de-interlaced decoded rendition from the third colour range to the fourth colour range comprises increasing the colour volume.
9. A method according to any preceding claim, wherein the step of an initial rendition of an input signal having a first colour range comprises: receiving the input signal; and, spatially downsampling the input signal.
10.A method according to any preceding claim, wherein the method further comprises: performing HDR mapping on the initial rendition to generate a set of HDR metadata; and, providing the HDR metadata to the enhancement decoder for encoding with the encoded enhancement signal.
11 A method according to any preceding claim, wherein the step of converting the initial rendition from the first colour range to a second colour range is a conversion that decreases a dynamic range of the first colour space, optionally wherein the step of converting the initial rendition from the first colour range to a second colour range is a conversion from HDR to SDR.
12. A method according to any preceding claim, wherein the step of converting the initial rendition from the first colour range to a second colour range is a conversion from a first colour space to a second colour space, wherein the first colour space comprises a wider colour gamut than the second colour space.
13. A method according to any preceding claim, wherein the first colour range has a higher colour depth than the second colour range, in particular wherein the first colour range is a 10 bit colour range and the second colour range is an 8 bit colour range.
14.A method according to any preceding claim wherein the first colour range comprises a larger colour volume than the second colour range.
15. A method according to claim 14, wherein converting the initial rendition of the input signal from the first colour range to a second colour range to generate a colour converted rendition comprises one or more of: decreasing a colour depth of the first colour space; decreasing a colour gamut of the first colour space; and decreasing a dynamic range of the first colour space.
16.A method of decoding an encoded enhancement signal for reproducing a rendition of a video signal, the method comprising: obtaining an encoded signal comprising an encoded enhancement signal and a base encoded signal; passing the base encoded signal to a base decoder to generate an interlaced base decoded video signal; processing the base decoded video signal to generate a reconstructed rendition of the video signal, the reconstructed rendition being a de-interlaced rendition of the base decoded video signal; combining the reconstructed rendition with a set of residuals decoded from the encoded enhancement stream to generate a reconstructed output video, wherein the reconstructed output video is in a sixth colour range different from the base decoded video signal in a fifth colour range.
17.A method according to claim 16, wherein the step of combining comprises converting the reconstructed rendition from the fifth colour range to the sixth colour range.
18. A method according to claim 16, wherein the method further comprises: converting a de-interlaced decoded rendition of the video signal from the fifth colour range to the sixth colour range to generate the reconstructed rendition of the video signal.
19. The method according to any one of claims 16 to 18, wherein the sixth colour range comprises one or more of: a larger colour depth than the fifth colour range; a larger colour gamut than the fifth colour range; and a larger dynamic range than the fifth colour range.
20. A method according to any of claims 16 to 19, further comprising: outputting the interlaced base decoded video.
21. A method according to any of claims 16 to 20, wherein the fifth colour range is the same as a third colour range, the third colour range being used to recontruct a rendition of the video signal during encoding.
22. A method according to any of claims 16 to 21, wherein the sixth colour range is the same as a fourth colour range, the fourth colour range being used to convert a de-interlaces base decoded rendition of a video signal during encoding.
23. An encoder configured to perform the method of any of claims 1 to 15.
24. A decoder configured to perform the method of any of claims 16 to 22.
25. A non-transitory computer-readable storage medium storing instructions which, when executed by one or more processors, cause the processors to perform a method according to any of claims 1 to 15 or 16 to 22.