GB2577350A

GB2577350A - Image data encoding and decoding

Info

Publication number: GB2577350A
Application number: GB1821010.4A
Authority: GB
Inventors: James Sharman Karl; Mark Keating Stephen
Original assignee: Sony Corp
Current assignee: Sony Corp
Priority date: 2018-09-24
Filing date: 2018-12-21
Publication date: 2020-03-25
Also published as: GB201909139D0; GB201821010D0; GB201815559D0; GB2577338A

Abstract

An image data encoder configured to encode samples of one or more colour components for respective image regions so as to generate encoded image data. The image data encoder comprises: a region controller configured to select one or both of a region size and a region shape for a current image region, and an intra-image predictor configured to derive predicted blocks of samples of colour components for the current image region. The intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples for a target set (e.g. chroma component) of colour components in dependence upon decoded samples of a source set (e.g. luma component) of the colour components by applying a component model function to the decoded source samples. This component model function being defined by model data (i.e. model parameters). A model function generator is configured to generate model data at least partially defining the component model function from input samples of the source set of the colour components and input samples of the target set of the colour components.

Description

IMAGE DATA ENCODING AND DECODING

BACKGROUND Field

This disclosure relates to image data encoding and decoding.

Description of Related Art

The "background" description provided herein is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, is neither expressly or impliedly admitted as

prior art against the present disclosure.

There are several video data encoding and decoding systems which involve transforming video data into a frequency domain representation, quantising the frequency domain coefficients and then applying some form of entropy encoding to the quantised coefficients. This can achieve compression of the video data. A corresponding decoding or decompression technique is applied to recover a reconstructed version of the original video data.

SUMMARY

The present disclosure addresses or mitigates problems arising from this processing. Respective aspects and features of the present disclosure are defined in the appended claims.

It is to be understood that both the foregoing general description and the following detailed description are exemplary, but are not restrictive, of the present technology.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: Figure 1 schematically illustrates an audio/video (AN) data transmission and reception system using video data compression and decompression; Figure 2 schematically illustrates a video display system using video data decompression; Figure 3 schematically illustrates an audio/video storage system using video data compression and decompression; Figure 4 schematically illustrates a video camera using video data compression; Figures 5 and 6 schematically illustrate storage media; Figure 7 provides a schematic overview of a video data compression and decompression apparatus; Figure 8 schematically illustrates a predictor; Figure 9 schematically illustrates a partially-encoded image; Figure 10 schematically illustrates a linear model chrominance (LM chroma) process; Figures 11 and 12 are schematic flowcharts illustrating respective methods; Figures 13 to 15 schematically illustrate variants of an LM chroma process; Figures 16 and 17 schematically illustrate linear models; Figures 18 and 19 are schematic flowcharts illustrating respective methods; Figure 20 schematically illustrates a block splitting process; Figure 21 is a schematic flowchart illustrating block splitting or partitioning; Figures 22 and 23 schematically illustrate respective variations of part of an encoder; Figure 24 schematically illustrates a mode selector; Figure 25 is a schematic flowchart illustrating a mode selection process; Figure 26 schematically illustrates the use of previously stored model data; Figures 27 to 29 are schematic flowcharts illustrating the operation of the apparatus of Figure 26; Figure 30 schematically illustrates different encoding resolutions; Figures 31 and 32 schematically illustrate inter-component dependencies in a linear model; Figures 33-42 are schematic flowcharts illustrating respective methods; Figure 43 is a summary schematic diagram of an encoding apparatus; Figure 44 is a summary schematic diagram of a decoding apparatus; and Figure 45 schematically illustrates an image region.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings, Figures 1-4 are provided to give schematic illustrations of apparatus or systems making use of the compression and/or decompression apparatus to be described below in connection with embodiments of the present technology.

All of the data compression and/or decompression apparatus to be described below may be implemented in hardware, in software running on a general-purpose data processing apparatus such as a general-purpose computer, as programmable hardware such as an application specific integrated circuit (ASIC) or field programmable gate array (FPGA) or as combinations of these. In cases where the embodiments are implemented by software and/or firmware, it will be appreciated that such software and/or firmware, and non-transitory data storage media by which such software and/or firmware are stored or otherwise provided, are considered as embodiments of the present technology.

Figure 1 schematically illustrates an audio/video data transmission and reception system using video data compression and decompression. In this example, the data values to be encoded or decoded represent image data.

An input audio/video signal 10 is supplied to a video data compression apparatus 20 which compresses at least the video component of the audio/video signal 10 for transmission along a transmission route 30 such as a cable, an optical fibre, a wireless link or the like. The compressed signal is processed by a decompression apparatus 40 to provide an output audio/video signal 50. For the return path, a compression apparatus 60 compresses an audio/video signal for transmission along the transmission route 30 to a decompression apparatus 70.

The compression apparatus 20 and decompression apparatus 70 can therefore form one node of a transmission link. The decompression apparatus 40 and decompression apparatus 60 can form another node of the transmission link. Of course, in instances where the transmission link is uni-directional, only one of the nodes would require a compression apparatus and the other node would only require a decompression apparatus.

Figure 2 schematically illustrates a video display system using video data decompression. In particular, a compressed audio/video signal 100 is processed by a decompression apparatus 110 to provide a decompressed signal which can be displayed on a display 120. The decompression apparatus 110 could be implemented as an integral part of the display 120, for example being provided within the same casing as the display device. Alternatively, the decompression apparatus 110 maybe provided as (for example) a so-called set top box (STB), noting that the expression "set-top" does not imply a requirement for the box to be sited in any particular orientation or position with respect to the display 120; it is simply a term used in the art to indicate a device which is connectable to a display as a peripheral device.

Figure 3 schematically illustrates an audio/video storage system using video data compression and decompression. An input audio/video signal 130 is supplied to a compression apparatus 140 which generates a compressed signal for storing by a store device 150 such as a magnetic disk device, an optical disk device, a magnetic tape device, a solid state storage device such as a semiconductor memory or other storage device. For replay, compressed data is read from the storage device 150 and passed to a decompression apparatus 160 for decompression to provide an output audio/video signal 170.

It will be appreciated that the compressed or encoded signal, and a storage medium such as a machine-readable non-transitory storage medium, storing that signal, are considered as embodiments of the present technology.

Figure 4 schematically illustrates a video camera using video data compression. In Figure 4, an image capture device 180, such as a charge coupled device (CCD) image sensor and associated control and read-out electronics, generates a video signal which is passed to a compression apparatus 190. A microphone (or plural microphones) 200 generates an audio signal to be passed to the compression apparatus 190. The compression apparatus 190 generates a compressed audio/video signal 210 to be stored and/or transmitted (shown generically as a schematic stage 220).

The techniques to be described below relate primarily to video data compression and decompression. It will be appreciated that many existing techniques may be used for audio data compression in conjunction with the video data compression techniques which will be described, to generate a compressed audio/video signal. Accordingly, a separate discussion of audio data compression will not be provided. It will also be appreciated that the data rate associated with video data, in particular broadcast quality video data, is generally very much higher than the data rate associated with audio data (whether compressed or uncompressed). It will therefore be appreciated that uncompressed audio data could accompany compressed video data to form a compressed audio/video signal. It will further be appreciated that although the present examples (shown in Figures 1-4) relate to audio/video data, the techniques to be described below can find use in a system which simply deals with (that is to say, compresses, decompresses, stores, displays and/or transmits) video data. That is to say, the embodiments can apply to video data compression without necessarily having any associated audio data handling at all.

Figure 4 therefore provides an example of a video capture apparatus comprising an image sensor and an encoding apparatus of the type to be discussed below. Figure 2 therefore provides an example of a decoding apparatus of the type to be discussed below and a display to which the decoded images are output.

A combination of Figure 2 and 4 may provide a video capture apparatus comprising an image sensor 180 and encoding apparatus 190, decoding apparatus 110 and a display 120 to which the decoded images are output.

Figures 5 and 6 schematically illustrate storage media, which store (for example) the compressed data generated by the apparatus 20, 60, the compressed data input to the apparatus 110 or the storage media or stages 150, 220. Figure 5 schematically illustrates a disc storage medium such as a magnetic or optical disc, and Figure 6 schematically illustrates a solid state storage medium such as a flash memory. Note that Figures 5 and 6 can also provide examples of non-transitory machine-readable storage media which store computer software which, when executed by a computer, causes the computer to carry out one or more of the methods to be discussed below.

Therefore, the above arrangements provide examples of video storage, capture, transmission or reception apparatuses embodying any of the present techniques.

Figure 7 provides a schematic overview of a video or image data compression and decompression apparatus, for encoding and/or decoding image data representing one or more images.

A controller 343 controls the overall operation of the apparatus and, in particular when referring to a compression mode, controls a trial encoding processes by acting as a selector to select various modes of operation such as block sizes and shapes, and whether the video data is to be encoded losslessly or otherwise. The controller is considered to part of the image encoder or image decoder (as the case may be). Successive images of an input video signal 300 are supplied to an adder 310 and to an image predictor 320. The image predictor 320 will be described below in more detail with reference to Figure 8. The image encoder or decoder (as the case may be) plus the intra-image predictor of Figure 8 may use features from the apparatus of Figure 7. This does not mean that the image encoder or decoder necessarily requires every feature of Figure 7 however.

The adder 310 in fact performs a subtraction (negative addition) operation, in that it receives the input video signal 300 on a "+" input and the output of the image predictor 320 on a " input, so that the predicted image is subtracted from the input image. The result is to generate a so-called residual image signal 330 representing the difference between the actual and projected images.

One reason why a residual image signal is generated is as follows. The data coding techniques to be described, that is to say the techniques which will be applied to the residual image signal, tend to work more efficiently when there is less "energy" in the image to be encoded. Here, the term "efficiently" refers to the generation of a small amount of encoded data; for a particular image quality level, it is desirable (and considered "efficient") to generate as little data as is practicably possible. The reference to "energy" in the residual image relates to the amount of information contained in the residual image. If the predicted image were to be identical to the real image, the difference between the two (that is to say, the residual image) would contain zero information (zero energy) and would be very easy to encode into a small amount of encoded data. In general, if the prediction process can be made to work reasonably well such that the predicted image content is similar to the image content to be encoded, the expectation is that the residual image data will contain less information (less energy) than the input image and so will be easier to encode into a small amount of encoded data.

Therefore, encoding (using the adder 310) involves predicting an image region for an image to be encoded; and generating a residual image region dependent upon the difference between the predicted image region and a corresponding region of the image to be encoded. In connection with the techniques to be discussed below, the ordered array of data values comprises data values of a representation of the residual image region. Decoding involves predicting an image region for an image to be decoded; generating a residual image region indicative of differences between the predicted image region and a corresponding region of the image to be decoded; in which the ordered array of data values comprises data values of a representation of the residual image region; and combining the predicted image region and the residual image region.

The remainder of the apparatus acting as an encoder (to encode the residual or difference image) will now be described. The residual image data 330 is supplied to a transform unit or circuitry 340 which generates a discrete cosine transform (DCT) representation of blocks or regions of the residual image data. The DCT technique itself is well known and will not be described in detail here. Note also that the use of DCT is only illustrative of one example arrangement. Other transforms which might be used include, for example, the discrete sine transform (DST). A transform could also comprise a sequence or cascade of individual transforms, such as an arrangement in which one transform is followed (whether directly or not) by another transform. The choice of transform may be determined explicitly and/or be dependent upon side information used to configure the encoder and decoder.

Therefore, in example, an encoding and/or decoding method comprises predicting an image region for an image to be encoded; and generating a residual image region dependent upon the difference between the predicted image region and a corresponding region of the image to be encoded; in which the ordered array of data values (to be discussed below) comprises data values of a representation of the residual image region.

The output of the transform unit 340, which is to say, a set of DCT coefficients for each transformed block of image data, is supplied to a quantiser 350. Various quantisation techniques are known in the field of video data compression, ranging from a simple multiplication by a quantisation scaling factor through to the application of complicated lookup tables under the control of a quantisation parameter. The general aim is twofold. Firstly, the quantisation process reduces the number of possible values of the transformed data. Secondly, the quantisation process can increase the likelihood that values of the transformed data are zero. Both of these can make the entropy encoding process, to be described below, work more efficiently in generating small amounts of compressed video data.

A data scanning process is applied by a scan unit 360. The purpose of the scanning process is to reorder the quantised transformed data so as to gather as many as possible of the non-zero quantised transformed coefficients together, and of course therefore to gather as many as possible of the zero-valued coefficients together. These features can allow so-called run-length coding or similar techniques to be applied efficiently. So, the scanning process involves selecting coefficients from the quantised transformed data, and in particular from a block of coefficients corresponding to a block of image data which has been transformed and quantised, according to a "scanning order" so that (a) all of the coefficients are selected once as part of the scan, and (b) the scan tends to provide the desired reordering. One example scanning order which can tend to give useful results is a so-called up-right diagonal scanning order.

The scanned coefficients are then passed to an entropy encoder (EE) 370. Again, various types of entropy encoding may be used. Two examples are variants of the so-called CABAC (Context Adaptive Binary Arithmetic Coding) system and variants of the so-called CAVLC (Context Adaptive Variable-Length Coding) system. In general terms, CABAC is considered to provide a better efficiency, and in some studies has been shown to provide a 10- 20% reduction in the quantity of encoded output data for a comparable image quality compared to CAVLC. However, CAVLC is considered to represent a much lower level of complexity (in terms of its implementation) than CABAC. Note that the scanning process and the entropy encoding process are shown as separate processes, but in fact can be combined or treated together. That is to say, the reading of data into the entropy encoder can take place in the scan order. Corresponding considerations apply to the respective inverse processes to be described below.

The output of the entropy encoder 370, along with additional data (mentioned above and/or discussed below), for example defining the manner in which the predictor 320 generated the predicted image, provides a compressed output video signal 380.

However, a return path is also provided because the operation of the predictor 320 itself depends upon a decompressed version of the compressed output data.

The reason for this feature is as follows. At the appropriate stage in the decompression process (to be described below) a decompressed version of the residual data is generated. This decompressed residual data has to be added to a predicted image to generate an output image (because the original residual data was the difference between the input image and a predicted image). In order that this process is comparable, as between the compression side and the decompression side, the predicted images generated by the predictor 320 should be the same during the compression process and during the decompression process. Of course, at decompression, the apparatus does not have access to the original input images, but only to the decompressed images. Therefore, at compression, the predictor 320 bases its prediction (at least, for inter-image encoding) on decompressed versions of the compressed images.

The entropy encoding process carried out by the entropy encoder 370 is considered (in at least some examples) to be "lossless", which is to say that it can be reversed to arrive at exactly the same data which was first supplied to the entropy encoder 370. So, in such examples the return path can be implemented before the entropy encoding stage. Indeed, the scanning process carried out by the scan unit 360 is also considered lossless, but in the present embodiment the return path 390 is from the output of the quantiser 350 to the input of a complimentary inverse quantiser 420. In instances where loss or potential loss is introduced by a stage, that stage may be included in the feedback loop formed by the return path. For example, the entropy encoding stage can at least in principle be made lossy, for example by techniques in which bits are encoded within parity information. In such an instance, the entropy encoding and decoding should form part of the feedback loop.

In general terms, an entropy decoder 410, the reverse scan unit 400, an inverse quantiser 420 and an inverse transform unit or circuitry 430 provide the respective inverse functions of the entropy encoder 370, the scan unit 360, the quantiser 350 and the transform unit 340. For now, the discussion will continue through the compression process; the process to decompress an input compressed video signal will be discussed separately below.

In the compression process, the scanned coefficients are passed by the return path 390 from the quantiser 350 to the inverse quantiser 420 which carries out the inverse operation of the scan unit 360. An inverse quantisation and inverse transformation process are carried out by the units 420, 430 to generate a compressed-decompressed residual image signal 440.

The image signal 440 is added, at an adder 450, to the output of the predictor 320 to generate a reconstructed output image 460. This forms one input to the image predictor 320, as will be described below.

Turning now to the process applied to decompress a received compressed video signal 470, the signal is supplied to the entropy decoder 410 and from there to the chain of the reverse scan unit 400, the inverse quantiser 420 and the inverse transform unit 430 before being added to the output of the image predictor 320 by the adder 450. So, at the decoder side, the decoder reconstructs a version of the residual image and then applies this (by the adder 450) to the predicted version of the image (on a block by block basis) so as to decode each block. In straightforward terms, the output 460 of the adder 450 forms the output decompressed video signal 480. In practice, further filtering may optionally be applied (for example, by a filter 560 shown in Figure 8 but omitted from Figure 7 for clarity of the higher level diagram of Figure 7) before the signal is output.

The apparatus of Figures 7 and 8 can act as a compression (encoding) apparatus or a decompression (decoding) apparatus. The functions of the two types of apparatus substantially overlap. The scan unit 360 and entropy encoder 370 are not used in a decompression mode, and the operation of the predictor 320 (which will be described in detail below) and other units follow mode and parameter information contained in the received compressed bit-stream rather than generating such information themselves.

Figure 8 schematically illustrates the generation of predicted images, and in particular the operation of the image predictor 320.

There are two basic modes of prediction carried out by the image predictor 320: so-called intra-image prediction and so-called inter-image, or motion-compensated (MC), prediction. At the encoder side, each involves detecting a prediction direction in respect of a current block to be predicted, and generating a predicted block of samples according to other samples (in the same (intra) or another (inter) image). By virtue of the units 310 or 450, the difference between the predicted block and the actual block is encoded or applied so as to encode or decode the block respectively.

(At the decoder, or at the reverse decoding side of the encoder, the detection of a prediction direction may be in response to data associated with the encoded data by the encoder, indicating which direction was used at the encoder. Or the detection may be in response to the same factors as those on which the decision was made at the encoder).

I ntra-image prediction bases a prediction of the content of a block or region of the image on data from within the same image. This corresponds to so-called I-frame encoding in other video compression techniques. In contrast to I-frame encoding, however, which involves encoding the whole image by intra-encoding, in the present embodiments the choice between intra-and inter-encoding can be made on a block-by-block basis, though in other embodiments the choice is still made on an image-by-image basis.

Motion-compensated prediction is an example of inter-image prediction and makes use of motion information which attempts to define the source, in another adjacent or nearby image, of image detail to be encoded in the current image. Accordingly, in an ideal example, the contents of a block of image data in the predicted image can be encoded very simply as a reference (a motion vector) pointing to a corresponding block at the same or a slightly different position in an adjacent image.

A technique known as "block copy" prediction is in some respects a hybrid of the two, as it uses a vector to indicate a block of samples at a position displaced from the currently predicted block within the same image, which should be copied to form the currently predicted block.

Returning to Figure 8, two image prediction arrangements (corresponding to intra-and inter-image prediction) are shown, the results of which are selected by a multiplexer 500 under the control of a mode signal 510 (for example, from the controller 343) so as to provide blocks of the predicted image for supply to the adders 310 and 450. The choice is made in dependence upon which selection gives the lowest "energy" (which, as discussed above, may be considered as information content requiring encoding), and the choice is signalled to the decoder within the encoded output data-stream. Image energy, in this context, can be detected, for example, by carrying out a trial subtraction of an area of the two versions of the predicted image from the input image, squaring each pixel value of the difference image, summing the squared values, and identifying which of the two versions gives rise to the lower mean squared value of the difference image relating to that image area. In other examples, a trial encoding can be carried out for each selection or potential selection, with a choice then being made according to the cost of each potential selection in terms of one or both of the number of bits required for encoding and distortion to the picture.

The actual prediction, in the intra-encoding system, is made on the basis of image blocks received as part of the signal 460, which is to say, the prediction is based upon encoded-decoded image blocks in order that exactly the same prediction can be made at a decompression apparatus. However, data can be derived from the input video signal 300 by an intra-mode selector 520 to control the operation of the intra-image predictor 530.

For inter-image prediction, a motion compensated (MC) predictor 540 uses motion information such as motion vectors derived by a motion estimator 550 from the input video signal 300. Those motion vectors are applied to a processed version of the reconstructed image 460 by the motion compensated predictor 540 to generate blocks of the inter-image prediction.

Accordingly, the units 530 and 540 (operating with the estimator 550) each act as detectors to detect a prediction direction in respect of a current block to be predicted, and as a generator to generate a predicted block of samples (forming part of the prediction passed to the units 310 and 450) according to other samples defined by the prediction direction.

The processing applied to the signal 460 will now be described. Firstly, the signal is optionally filtered by a filter unit 560, which will be described in greater detail below. This involves applying a "deblocking" filter to remove or at least tend to reduce the effects of the block-based processing carried out by the transform unit 340 and subsequent operations. A sample adaptive offsetting (SAO) filter may also be used. Also, an adaptive loop filter is optionally applied using coefficients derived by processing the reconstructed signal 460 and the input video signal 300. The adaptive loop filter is a type of filter which, using known techniques, applies adaptive filter coefficients to the data to be filtered. That is to say, the filter coefficients can vary in dependence upon various factors. Data defining which filter coefficients to use is included as part of the encoded output data-stream.

The filtered output from the filter unit 560 in fact forms the output video signal 480 when the apparatus is operating as a decompression apparatus. It is also buffered in one or more image or frame stores 570; the storage of successive images is a requirement of motion compensated prediction processing, and in particular the generation of motion vectors. To save on storage requirements, the stored images in the image stores 570 may be held in a compressed form and then decompressed for use in generating motion vectors. For this particular purpose, any known compression / decompression system may be used. The stored images are passed to an interpolation filter 580 which generates a higher resolution version of the stored images; in this example, intermediate samples (sub-samples) are generated such that the resolution of the interpolated image is output by the interpolation filter 580 is 4 times (in each dimension) that of the images stored in the image stores 570 for the luminance channel of 4:2:0 and 8 times (in each dimension) that of the images stored in the image stores 570 for the chrominance channels of 4:2:0. The interpolated images are passed as an input to the motion estimator 550 and also to the motion compensated predictor 540.

The way in which an image is partitioned for compression processing will now be described. At a basic level, an image to be compressed is considered as an array of blocks or regions of samples. The splitting of an image into such blocks or regions can be carried out by a decision tree, such as that described in SERIES H: AUDIOVISUAL AND MULTIMEDIA SYSTEMS Infrastructure of audiovisual services -Coding of moving video High efficiency video coding Recommendation ITU-T H.265 12/2016. Also: High Efficiency Video Coding (HECV) algorithms and Architectures, Editors: Madhukar Budagavi, Gary J. Sullivan, Vivienne Sze; ISBN 978-3-319-06894-7; 2014 which is incorporated herein in its entirety by reference. In some examples, the resulting blocks or regions have sizes and, in some cases, shapes which, by virtue of the decision tree, can generally follow the disposition of image features within the image. This in itself can allow for an improved encoding efficiency because samples representing or following similar image features would tend to be grouped together by such an arrangement. In some examples, square blocks or regions of different sizes (such as 4x4 samples up to, say, 64x64 or larger blocks) are available for selection. In other example arrangements, blocks or regions of different shapes such as rectangular blocks (for example, vertically or horizontally oriented) can be used. Other non-square and non-rectangular blocks are envisaged. The result of the division of the image into such blocks or regions is (in at least the present examples) that each sample of an image is allocated to one, and only one, such block or region.

The intra-prediction process will now be discussed. In general terms, intra-prediction involves generating a prediction of a current block of samples from previously-encoded and decoded samples in the same image.

Figure 9 schematically illustrates a partially encoded image 800. Here, the image is being encoded from top-left to bottom-right on a block by block basis. An example block encoded partway through the handling of the whole image is shown as a block 810. A shaded region 820 above and to the left of the block 810 has already been encoded. The intra-image prediction of the contents of the block 810 can make use of any of the shaded area 820 but cannot make use of the unshaded area below that.

In some examples, the image is encoded on a block by block basis such that larger blocks (referred to as coding units or CUs) are encoded in an order such as the order discussed with reference to Figure 9. Within each CU, there is the potential (depending on the block splitting process that has taken place) for the CU to be handled as a set of two or more smaller blocks or transform units (TUs). This can give a hierarchical order of encoding so that the image is encoded on a CU by CU basis, and each CU is potentially encoded on a TU by TU basis. Note however that for an individual TU within the current coding tree unit (the largest node in the tree structure of block division), the hierarchical order of encoding (CU by CU then TU by TU) discussed above means that there may be previously encoded samples in the current CU and available to the coding of that TU which are, for example, above-right or below-left of that TU.

The block 810 represents a CU; as discussed above, for the purposes of intra-image prediction processing, this may be subdivided into a set of smaller units. An example of a current TU 830 is shown within the CU 810. More generally, the picture is split into regions or groups of samples to allow efficient coding of signalling information and transformed data. The signalling of the information may require a different tree structure of sub-divisions to that of the transform, and indeed that of the prediction information or the prediction itself. For this reason, the coding units may have a different tree structure to that of the transform blocks or regions, the prediction blocks or regions and the prediction information. In some examples the structure can be a so-called quad tree of coding units, whose leaf nodes contain one or more prediction units and one or more transform units; the transform units can contain multiple transform blocks corresponding to luma and chroma representations of the picture, and prediction could be considered to be applicable at the transform block level. In examples, the parameters applied to a particular group of samples can be considered to be predominantly defined at a block level, which is potentially not of the same granularity as the transform structure.

The intra-image prediction takes into account samples coded prior to the current TU being considered, such as those above and/or to the left of the current TU. Source samples, from which the required samples are predicted, may be located at different positions or directions relative to the current TU. To decide which direction is appropriate for a current prediction unit, the mode selector 520 of an example encoder may test all combinations of available TU structures for each candidate direction and select the prediction direction and TU structure with the best compression efficiency.

The picture may also be encoded on a "slice" basis. In one example, a slice is a horizontally adjacent group of CUs. But in more general terms, the entire residual image could form a slice, or a slice could be a single CU, or a slice could be a row of CUs, and so on. Slices can give some resilience to errors as they are encoded as independent units. The encoder and decoder states are completely reset at a slice boundary. For example, intra-prediction is not carried out across slice boundaries; slice boundaries are treated as image boundaries for this purpose.

Figure 10 schematically represents a previously proposed process applicable to a so-called linear mode (LM) chrominance operation.

In LM chroma, predicted samples for the chrominance channel are derived by a linear model from reconstructed samples of the luminance channel.

This can allow greater coding efficiency because there is no need to code values for individual predicted chrominance samples, and in some instances (for examples where the linear model is derived at both the encoder and the decoder) there would be no need to encode particular parameters for the linear model itself, simply a flag or indicating data that the LM chroma data is in use. However, previous proposed LM chroma operations do potentially impose a significant latency on the encoding and decoding. This issue will be discussed with reference to Figure 10. An upper portion 1000 of Figure 10 relates to the encoding, using an arrangement such as that shown in Figure 7 and reference numerals are used where they correspond to Figure 7) of the luminance channel. A lower portion 1010 represents the encoding of a chrominance channel. In these techniques, input luminance data Yisig is processed to generate reconstructed luminance data Yi" from which a linear model processor 1020 derives a linear model (for example, expressed as a gradient and intercept value) between reconstructed luminance samples and reconstructed chrominance samples. Note that in order to use the linear model in the encoding and decoding of a particular chrominance block, the reconstructed chrominance and luminance samples are generally co-cited outside of the block in question so that the linear model can be obtained before the encoding or decoding of the block in question on the chrominance channel. Based on the linear model, a prediction is generated of the chrominance samples and this prediction 1030 is subtracted from input chrominance samples Cod, to form the basis of the chrominance encoding.

An issue with this arrangement is that the encoding (at the encoder) or decoding (at the decoder) of the chrominance samples cannot start until the reconstructed luminance samples have been obtained, and in fact the linear model processing 1020 cannot commence without the reconstructed luminance samples. This implies a potentially significant latency in the system, in that an entire cycle of encoding or decoding on the luminous channel has to be performed before the corresponding chrominance samples can be processed.

Previously proposed techniques are disclosed by "Chroma Intra Prediction based on Inter-Channel Correlation for HEVC", Zhang et al, IEEE Transactions on Image Processing, Vol. 23. No. 1, January 2014, the contents of which are hereby incorporated by reference.

The previously proposed LM chroma operation is summarised in Figures 11 and 12. In Figure 11, at a step 1100 luminance samples are encoded and reconstructed luminance samples generated. At a step 1110, a detection is made as to whether even use the LM chroma mode. If it is used then its use is flagged in the data stream to inform the decoder.

At a step 1120, the linear model (s) is derived. Note that more than one linear model can be derived, for example using techniques to be discussed below with reference to Figures 16 and 17.

At a step 1130, the chrominance block is predicted using the linear model (s) and then at a step 1140 the chroma residual data is generated and encoded.

Therefore, the chrominance model function represents a linear relationship between predicted chrominance sample values and the decoded luminance sample values.

At the decoding side for chrominance data, in Figure 12, at a step 1200 the luminance data is decoded and reconstructed luminance samples are generated. At a step 1210, a detection is made as to whether the LM chroma mode has been used (for example, in response to the flag mentioned above) and at a step 1220, unless the linear model data was included in the data stream, the linear model is again derived from the same source data as would have been used at the encoder.

At a step 1230 the chrominance block is predicted using the linear model (s) and at a step 1240 reconstructed chrominance samples are generated using the prediction generated at the step 1230 and chroma residual data decoded separately at a step 1250.

Figure 13 again represents the previously proposed arrangement in which the linear model for use in generating the predicted chrominance sample is generated by, for example, a linear regression process 1300 from the reconstructed luminance and chrominance samples 1310. The outputs of the linear regression process are a gradient value (alpha) and an intercept value (beta) which are processed by a linear model processor 1320 to generate predicted chrominance samples 1330 from co-cited reconstructed luminance samples 1340.

Note that the whole arrangement of Figure 13 would be repeated at the encoder and at the decoder.

In an alternative arrangement forming an embodiment of the present disclosure shown in Figure 14, the linear regression process 1400 is in fact carried out on original luminance and chrominance samples 1410. Again, alpha and beta values are generated but because the original samples are not available at the decoder, the alpha and beta values needs to be transmitted as data 1420 in associated with the encoded data stream for use at the decoder. The linear model 1430 uses the alpha and beta values (generated by the linear model generator 1400) and the reconstructed luminance values (which will be available at the decoder) to generate predicted chrominance values 1440.

Therefore, the model function generator may be configured to provide the model data 1420 in association with an encoded data stream generated by the apparatus.

At the decoder, the linear regression unit 1400 is neither required nor possible (as the original samples are not available) so the predictor at the decoder would comprise the linear model unit 1430 with the inputs of the data 1420. Here, the received model data defines a gradient and an intercept of the linear relationship.

In another alternative example of the present disclosure, the linear regression process 1500 is performed again on original luminance and chrominance values 1510 to generate an alpha value (gradient) as an output 1520 which is in turn communicated 1530 to the decoder.

The alpha value is used as an input to an intercept detector 1540 which detects an intercept value by multiplying reconstructed luminance values by the alpha values and subtracting them for reconstructing chrominance samples to detect a beta value 1550. The linear model 1560 uses the alpha and beta values generated by the linear model generator 1500, 1540. The same reconstruction process can be carried out at the decoder to generate the beta value so that the only additional data to be transmitted as part of the data stream to the decoder is the alpha value 1530. The alpha generated from the original samples and the beta generated from the reconstructed samples are used by the linear model 1560 to generate predicted chrominance samples 1570.

In other words, in these examples, the model function generator is configured to generate the model data from input samples of the source set of the colour components and input samples of the target set of the colour components.

Again, at the decoder, the linear regression unit 1500 is neither required nor possible (as the original samples are not available) so the predictor at the decoder would comprise the linear model unit 1560 with the inputs of the data 1530, and the unit 1540. Here, at the decoder, the received model data comprises gradient data defining at least a gradient of the linear relationship; and the decoder comprises a model function generator configured to detect intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded luminance samples and the decoded chrominance samples. For example, the model function generator at the encoder and the decoder may be configured 1540 to generate the intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded luminance samples and decoded chrominance samples, at least the decoded chrominance samples being from outside the current image region. In other words, in some examples, reference chrominance and luminance samples may be used to derive the linear model, because these are available at the stage at which the linear model is required. In other examples, however, the linear model derivation may be carried out at the encoder and decoder by applying a linear regression process to reference chrominance samples and luminance samples in the current region. This can reduce the need for storage for the luminance samples.

In the examples of Figures 14 and 15 as applied to Figure 7 and operating under the control of the controller 343, there is provided an image encoding apparatus comprising: an intra-image predictor 320 configured to derive predicted luminance and chrominance 30 samples; a residual data encoder (340...370) configured to encode respective luminance and chrominance residual data representing a difference between input luminance and chrominance samples for the current image region and the respective predicted samples; a decoding stage (420... 450) configured to decode the residual data and to generate decoded luminance samples by combining the decoded residual data and the predicted luminance samples; the intra-image predictor being selectively operable in a chrominance model mode in which the intra-image predictor is configured to generate the predicted chrominance samples from the decoded luminance samples by applying a chrominance model function to the decoded luminance samples; and a model function generator 1400, 1500, 1540 configured to generate model data 1420, 1530 at least partially defining the chrominance model function from the input luminance samples and the input chrominance samples.

At the decoder side, this is an example of an image decoding apparatus responsive to an input encoded data stream, the apparatus comprising: an intra-image predictor 320 configured to derive predicted luminance and chrominance samples; a residual data decoder 470...430 configured to decode respective luminance and chrominance residual data representing a difference between input luminance and chrominance samples for the current image region and the respective predicted samples; a combiner450 configured to generate decoded luminance samples by combining the decoded residual data and the predicted luminance samples; the intra-image predictor being selectively operable in a chrominance model mode in which the intra-image predictor is configured to generate the predicted chrominance samples from the decoded luminance samples by applying a chrominance model function to the decoded luminance samples, the chrominance model function being defined at least partially by model data1420, 1530 received by the apparatus in association with the input encoded data stream.

The model function generator 1400, 1500, 1540 may be configured to detect a linear regression between the input luminance samples and the input chrominance samples so as to detect gradient data defining at least a gradient of the linear relationship.

in examples, the model function generator 1540 may be configured to generate intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded luminance samples and the decoded chrominance samples. As mentioned above, more than one linear model may be used. Figure 16 schematically illustrates a set of luminance, chrominance) sample pairs and a single model 1600 arrived at by a linear aggression. However, referring to Figure 17, it may in fact be appropriate to express this relationship as multiple models relating to respective sub-groups of the (luminance, chrominance sample pairs). Note that in each of these situations the techniques are also applicable (as described below) to systems in which the source samples are not necessarily luminance samples and the target samples are not necessarily chrominance samples.

Figure 18 is a schematic flowchart illustrating an image encoding method comprising: predicting (at a step 1800) luminance and chrominance samples; encoding (at a step 1810) respective luminance and chrominance residual data representing a difference between input luminance and chrominance samples for the current image region and the respective predicted samples; decoding (at a step 1820) the residual data and to generate decoded luminance samples by combining the decoded residual data and the predicted luminance samples; the predicting step being selectively operable in a chrominance model mode in which the predicting step comprises generating the predicted chrominance samples from the decoded luminance samples by applying a chrominance model function to the decoded luminance samples; and generating (at a step 1830) model data at least partially defining the chrominance model function from the input luminance samples and the input chrominance samples.

Figure 19 is a schematic flowchart illustrating an image decoding method responsive to an input encoded data stream, the method comprising: predicting (at a step 1900) luminance and chrominance samples; decoding (at a step 1910) respective luminance and chrominance residual data representing a difference between input luminance and chrominance samples for the current image region and the respective predicted samples; and generating (at a step 1920) decoded luminance samples by combining the decoded residual data and the predicted luminance samples; the predicting step being selectively operable in a chrominance model mode in which the predicting step comprises generating the predicted chrominance samples from the decoded luminance samples by applying a chrominance model function to the decoded luminance samples, the chrominance model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.

Further examples of these and similar techniques will now be described.

As background to some of these alternative examples, some aspects of an encoding block structure will first be described.

In general, in the present examples, all three components (such as Y,Cb,Cr) may be encoded, for example using an identical block structure, although different block structures for the different components may be employed. A highest level block can be referred to as a Coding Tree Unit (CTU). A CU refers to a coding unit; a PU refers to a prediction unit and defines the parameters for the prediction unit; and a TU refers to a transform unit, or in other words the block of data that is to be transformed The HEVC system uses a system of a so-called quad tree (having a root node and a set of leaf nodes) of CUs, where each leaf CU represents one or more PUs. Each leaf CU represents the root of a transform quad tree.

The JEM system uses QTBT (quad tree binary tree), in that each root CU splits down by a quad tree. Each leaf then optionally splits further with a binary tree. Each of those leaves then represents a PU, CU, TU without distinction.

As an example, Figure 20 schematically illustrates a so-called root coding unit (CU) 2000, shown as a region 2010 on the right hand side of Figure 20. A quad tree division of the root CU 2000 produces four sub CUs 2020, one of which 2030 is shown as being divided by a further four-way division into smaller CUs 2040, one of which 2050 is split by a binary division into coding units 2060. Note that the blocks resulting from these divisions can be square or non-square.

This arrangement provides an example in which a current image region is a sub-region of a larger image region in a hierarchy of image regions.

Figure 21 schematically illustrates an example technique relating to such block or region division. The example of Figure 21 uses the tree-based decision (whether quad tree or binary tree) of Figure 20, in which a higher level block such as a root block or the blocks 2030, 2050 2900 is partitioned using the example processes shown in 20 at a step 2110. If further partitioning is required at a step 2120 then control returns to this step 2110. Criteria for determining whether further partitioning is required will be discussed below.

At a step 2140 the partitioned blocks are encoded.

The process of partitioning the blocks discussed with reference to Figures 20 and 21 can take place collectively for luminance and chrominance components or can take place separately for (i) luminance and 00 chrominance components. These processes will be discussed briefly with reference to Figures 22 and 23 below.

In Figure 22, a block splitting controller 2200 undertakes the process described with reference to Figure 21. The steps 2110 and 2120 can be performed with reference to a detection of a so-called cost function. An example of a suitable cost function is a detection of the amount of data predicted to be required to encode the image based upon a particular partitioning arrangement under test. The partitioning can continue according to the tree structure shown schematically in Figure 20 in order to arrive at a low value such as a minimum value of the cost function. The cost function can be assessed by, for example, a trial encoding process or by other analytical processes which arrive at an estimation of the amount of data which would result from an encoding process.

In some examples, the block splitting process carried out by the controller 2200 is carried out before an encoding mode is selected for each block. In other examples, the two processes may be carried out together so that the derivation of the cost function by the block splitting controller takes into account a most-suitable (lowest cost) encoding mode, for example for the luminance component, for the block under test. But referring back to Figure 22, once the block partitioning has been performed by the block splitting controller 2200, a luminance mode selector 2210 and a chrominance mode selector 2220 can be configured to select a encoding mode for the luminance and chrominance components based on the block size and shape selected (as a common selection for luminance and chrominance) by the block splitting controller 2200. The selection of an encoding mode for the chrominance components by the chrominance mode selector 2220 has, as an option, a linear model (LM) mode as discussed above. More detail of a mode selection process will be given below.

An encoder 2230 encodes data defining the selected encoding mode. In the case of the selection of an LM mode, examples of the form of this encoded data will be discussed further below.

A similar arrangement is shown in Figure 23, except that the block partitioning process is carried out independently for luminance and chrominance, by a luminance block splitting controller 2308 chrominance block splitting controller 2310. There can optionally be a dependency between them so that, for example, a chrominance block is constrained so as to be no larger than a co-sited luminance block, or the processes could be independent.

In each case, an encoding mode is selected by a luminance mode selector 2320 and a chrominance mode selector 2330 respectively and details of the block partitioning and selected modes are encoded by an encoder 2340.

Figure 24 schematically illustrates a mode selector such as the units 2210, 2220, 2320, 2330 discussed above comprising a controller 2400 which controls the operation of a mode tester 2410 and a detector 2420. The mode selector of Figure 24 is responsive to sample data relating to the block or region under test and generates an output indication 2430 of a selected mode. The encoding mode selected by the arrangement of Figure 24 may be an inter-prediction mode, and intra-prediction mode such as a directional intra-image prediction mode, a linear model (LM) mode of the type discussed above, a DC mode or the like.

In each case, as shown in a schematic flowchart of Figure 25, the detector 2420 is operable at a step 2500 to detect a cost function associated with a mode under test by the mode tester 2410. At an optional step 2510 a weighting is applied; this will be discussed further below. If, at a step 2520, there is a further mode to be tested, then control returns to the step 2500 before (at the end of the set of notes to be tested) control is passed to a step 2530 at which the mode, of those tested, giving the lowest cost function, optionally taking into account the weighting applied at the step 2510, is selected as the mode to be used in encoding the current block or region.

In each case, the cost function takes into account the data requirements to encode any ancillary data associated with the mode in question. In examples such as that shown in Figure 14 and described above, where parameters alpha and beta are transmitted in association with the encoded image data, the quantity of data required to transmit the alpha and beta parameters is taken into account in deriving the cost function for the use of the linear model. In examples such as that shown in Figure 15 and discussed above, where the parameter alpha is transmitted in association with the encoded image data, then the quantity of data required to transmit the alpha parameter is taken into account in the derivation of the cost function. Figure 26 schematically represents an arrangement which can reduce the amount of data required to transmit parameters associated with the use of the linear model. A data store 2600 stores a value "prey" which is a previously-used set of linear model parameters (alpha and beta, or alternatively just alpha in the examples discussed above) for a previous block or region of samples. In the present examples, the previous block is actually the most recently encoded block for which the linear model (the component model) was used, in the encoding order of blocks in the image. A difference detector 2610 detects whether a difference exists between the value(s) "prey" and the newly generated alpha and/or beta values 2620 in respect of a current block. An encoder 2630 encodes alpha and/or beta for association with the encoded data stream according to the difference detected by the difference detector 2610. Examples of this operation will be discussed with reference to schematic flowcharts of Figures 27 and 28.

Therefore, in examples, the previously encoded image region is a most recently encoded image region in the ordered succession of image regions, for which the intra-image predictor used the component model.

In other examples, more than one "prey" store 2600 may be used so as to save a predetermined set of data relating to recently encoded regions (for example, the last six encoded in that way, or three nearest neighbours, or the like, as long as the same predetermined set is saved at the encoder and the decoder), and the model generator at the encoder can be configured to select model data for a previously encoded region according to a best fit or cost function, and to encode data identifying the previously encoded image region.

Referring to Figure 27, in some examples the difference detector 2610 is configured to detect a zero difference, which is to say that the new parameters 2620 identical to the previously used parameters stored in the storage 2600. In the event that these parameters are identical, then at a step 2700 the difference detector 2610 detects that the previously used parameters should be employed and control passes to a step 2710 at which the parameters are encoded by a simple flag (potentially a one-bit flag) indicating "match with previously used parameters" or "re-use previous parameters". At the decoder, the previously used parameters are also stored and the receipt of the match flag indicates to the decoder that the previously used parameters should be employed in respect of the decoding of the current block or region. On the other hand, if (in the example of Figure 27) the difference detector 2610 detects a difference at the step 2700 between the previously used values and the new values 2620 then control passes to a step 2720 at which the parameters are encoded as a "no match" flag (again, for example, a one-bit flag) along with the value(s) of the parameter(s) to be transmitted. So, the arrangement of Figure 27 can provide a net reduction in the data quantity requires to implement the linear model in the form discussed above with reference to Figures 14 and 15 by reducing the amount of data (for example to just one bit) required to transmit the alpha and/or beta parameters when they match the previously used parameters, but it must be taken into account as well that an extra bit is used for the "no match" flag when the values differ from the previously used values.

Therefore, in examples, the model data selectively comprises a difference value between the model data for the current image region and the model data for the previously encoded image region. Alternatively, the model data selectively comprises a flag indicating reuse, for the current image region, of the model data for the previously encoded image region.

The match/no match detection could be carried out first for one of the components, for example Cb (for the previous respective Cb model data), and then in the derivation of Cr the mode selection process could be weighted towards a match with the previous respective Cr model data, for example by applying a weighting at the step 2510. The reason for such a weighting is that the potential gain in using a "match" situation for both components is relatively large and if a match has been found for one component a similar outcome is encouraged by the weighting for the other component.

In another example, in Figure 28, the difference detector 2610 is arranged to detect, at a step 2800, whether the new values 2620 are near to the previously used values, which is to say within a threshold difference of the previously used values. The threshold may vary in dependence upon block size, in a manner to be discussed below with reference to Figure 30.

But in general terms, if the outcome of the step 2800 is that the new values are near to the previously used values then control passes to a step 2810 at which the parameters are encoded as a "match" flag along with a difference value representing the difference between the new values and the previously used values. If not, then control passes to a step 2820 as which the parameters are encoded as a "no match" flag along with the value of the alpha and/or beta parameters.

In other words, if the difference between the model data for the current image region and the model data for the previously encoded image region is less than a threshold difference, the model function generator is configured to encode a difference value.

In other examples, if the difference between the model data for the current image region and the model data for the previously encoded image region is zero, the model function generator is configured to encode a flag indicating re-use of the model data for the previously encoded image region.

Figure 29 schematically represents another example of this type of operation in which, at a step 2900, a detection is made as to whether the current block or region is a "small" block, for example using a definition of "small" to be discussed with reference to the top row of Figure 30 below. If the answer is yes then control passes to a step 2910 at which a decision is made as to whether to "reset" the current arrangements applicable to small blocks. The reset function will be discussed further below.

Assuming that the arrangement should not be reset, then at a step 2930 the encoding of the alpha and/or beta parameters is forced to be performed using the match flag, which takes place at a step 2940.

The reason for the reset step at the decision point 2910 is that it can be useful not to allow an indefinite string of small blocks to continually reuse the same parameters. A reset operation (in which the parameter values are actually encoded) could be performed, for example, once every n small blocks (where n might equal eight for example), or once every CTU or slice, or the like. Therefore, if a reset operation is triggered by one of these criteria at the step 2910, control passes to a step 2950 at which the alpha and/or beta parameters are encoded as actual values rather than as a reuse of previously used values.

Therefore, in examples, for image regions at a set of predetermined image locations (such as the first in a CTU) or after encoding a predetermined number of image regions using the flag indicating re-use of the model data for the previously encoded image region, the mode selector is configured to allow the selection of the component model mode independently of whether when the model data for the current region can be encoded by the flag indicating reuse of the model data for the previously encoded image region.

Returning to the step 2920 which applies in the case where the current block is not a "small block", a detection is made at the step in a similar manner to the steps 2700 or 2800 discussed above, as to whether the current parameters should be encoded as a reuse of previously used parameters (with or without a difference value as discussed above), in which case control passes to the step 2940, or not, in which case control passes to the step 2950.

Therefore, in at least some of these examples, for a current region size below a threshold size, the mode selector is configured to select use of the component model mode only when the model data for the current region can be encoded by the flag indicating re-use of the model data for the previously encoded image region.

In other examples, however, for a current region size below a threshold size (for example, a small block as discussed), the mode selector is in fact configured to inhibit re-use of the model data for the previously encoded image region.

Figure 30 schematically represents a technique for expressing or encoding the encoded alpha values (for example in the example arrangement of Figure 15 above, but also applicable to the example arrangement of Figure 14 above) using a resolution which varies according to one or both of the block or region size and shape for the current block or region.

In Figure 30, the model function generator is configured to select one of multiple encoding resolutions in dependence upon whether the current image region has one or both of the region size and the region shape in a respective range.

At the left side of Figure 30, the block or region "area" is expressed. This refers to a product of the block height and the block width, in samples, and refers to the "derived" or target block rather than the source block used as source data in the linear model. So, in at least some examples, the target block is a chrominance block and the height and width measurements relates to the size, in samples, of the relevant chrominance block. The area is divided, in these examples, into three area ranges, namely an area of less than 32 (an example definition of "small block" relevant to the step 2900 discussed above), an area of at least 32 and less than 256, or an area of at least 256. In each of these area ranges, the alpha parameter is expressed using a different respective number of bits. In other words, the alpha parameter is quantised more heavily for smaller blocks and less heavily for larger blocks.

In other examples the block shape could determine the encoding resolution (for example, a lower number of bits for non-square blocks and a higher number of bits for square blocks. In other examples, a block length or a block width (or the larger, or the smaller, of the length and width) could determine the encoding resolution. A feature of using the block area is that it is agnostic to block shape or to any individual dimension of the block.

The values shown in Figure 30 represent the alpha value multiplied by 16, so that the actual alpha value used by the linear model processing at the decoder is the received value (one of the options shown in Figure 30) divided by 16, so as to express an alpha value magnitude of between 0 and 2. In all cases except for the values of 0, a sign bit or other indication of sign is provided, so that the overall range of alpha values which can be expressed by this arrangement is as follows: -2 <= alpha a +2 In order to express the top row of alpha values, two bits plus a sign bit are required. For the second row, three bits plus a sign bit that are required and for the third (bottom) row, four bits plus a sign bit are required. In a practical system, however, the alpha values may be encoded using respective CABAC contexts, so that the exact mapping between the number of bits required to express a particular alpha value and the resolution of the respective row of alpha values may not be as simple as just provided, in that, for example for the bottom row of Figure 30, more commonly occurring alpha values may require fewer than four net bits to be encoded and less commonly occurring alpha values may require more than four net bits.

In some examples, a sign bit is not provided when an alpha value of 0 is being encoded. Using these techniques, information provided in the data stream from the encoded to the decoder may comprise (amongst other things): i. [specification of current block size and shape]; ii. [indication that linear model is in use]; [alpha value]; iv. [beta value in Figure 14 -but not in the example of Figure 15] In the case of data (iii), the decoder reads from the encoded data stream a number of bits appropriate to the block size and shape specified by the previously-received data (i). So, in the example case of the middle row of Figure 30 (a block area of at least 32 but less than 256), the decoder reads from the data stream (for example at the output of a CABAC decoder) sufficient bits either to define a value of 0 or to define three bits of alpha value and a sign bit.

If the beta value is also transmitted, a similar technique of variable resolution in dependence upon the block size and/or shape may be used, or a fixed resolution independent of block size and/or shape may be used for the transmission of the beta value.

In an example in which both chrominance components (for example, Cb and Cr) are derived by the linear model, two respective alpha values would normally be transmitted, one for each chrominance component. However, in alternative examples, the two chrominance components could share an alpha value.

In the examples discussed above (and some examples discussed below) the linear model technique is used so as to derive chrominance components from decoded luminance samples at the decoder. However, it is not a strict requirement that this particular dependency is used. In general terms, any component could be derived by a linear model from any other component; it is just generally considered more convenient to treat the luminance component as the primary or source component in linear model processing. In order to illustrate this potential generalisation, Figures 31 and 32 are schematic flowcharts illustrating example instances of linear model processing is performed at the decoder side. It will of course be assumed that corresponding encoding has taken place at the encoder in order to provide for this decoding process.

In Figures 31 and 32, the three colour components are referred to as components (A, B, C). These could represent (Y, Cb, Cr), (R, G, B) or any other expression of colour components, potentially in any order. Purely by way of one example, the mapping between (A, B, C) and (Y, Cb, Cr) may be as follows: A Luminance Y B Blue colour difference Cb C Red colour difference Cr Referring to Figure 31, at a step 3100, the decoder decodes samples of the colour component A. Then, at a step 3110, the decoder generates predicted samples of the colour component B using data 3120 (received at least in part from the encoder as part of the encoded data stream) defining a linear model between the component B and the component A. At a step 3130 the decoder decodes samples of the colour component B using the predicted samples generated at the step 3110.

Then, at a step 3140, the decoder generates predicted samples of the colour component C using data 3150 (received at least in part from the encoder as part of the encoded data stream) defining a linear model between the component C and the component B. Finally, at a step 3160, the decoder decodes samples of the colour component C using the predicted samples generated at the step 3140.

An alternative arrangement is shown in Figure 32, in which, at a step 3200 the decoder decodes samples of the colour component A. Then, serially or in parallel, steps 3210 and 3240 are performed by the decoder.

At the step 3210, predicted samples of the colour component B are generated from the decoded samples of the colour component A using data 3220 (received at least in part from the encoder as part of the encoded data stream) defining a linear model between the component B and the component A. At a step 3230, the decoder decodes samples of the colour component B using the predicted samples generated at the step 3210.

Similarly, at the step 3240, predicted samples of the colour component C are generated from the decoded samples of the colour component A using data 3250 (received at least in part from the encoder as part of the encoded data stream) defining a linear model between the component C and the component A. At a step 3260, the decoder decodes samples of the colour component C using the predicted samples generated at the step 3240.

Therefore, in some examples, the model function generator is configured to generate model data in respect of two or more of the colour components and to encode the model data for each of the two or more colour components in dependence upon a difference between the model data for that colour component for the current image region and model data for that colour component for a previously encoded image region.

The intra-image predictor may be responsive to model data in respect of two or more of the colour components. In some but not necessarily all examples, the set of colour components comprises luminance and two chrominance components; the source set of at least one of the colour components comprises the luminance component; and the target set of at least one of the colour components comprises one or both of the chrominance components. For example the chrominance components may comprise first, second and third colour components, and the intra-image predictor is operable in the component model mode (Figure 31) to generate: predicted samples of the second colour component from decoded samples of the first colour component using a first component model; and predicted samples of the third colour component from decoded samples of the second colour component using a second component model. Foer example, the components may comprise a blue-difference component and a red-difference component; the second colour component is the blue-difference component; and the third colour component is the red-difference component. Inferred meaning of the encoded alpha value 0 Figures 33-36 provide an indication of possible meanings attached to the encoded alpha value of 0 as shown in Figure 30 for example.

In Figures 33 and 34, which represent operations at the encoder and decoder side respectively, the encoded alpha value of 0 is used to represent a zero slope in the linear model, which is to say an indication of no linear dependence between the target colour component values and the source colour component values.

The component model (linear model) function may represent a linear relationship between predicted target sample values and decoded source samples.

In Figure 33, at a step 3300 the encoder detects such an instance of no dependence and at a step 3310 encodes an alpha value of 0. In Figure 34, the decoder detects the encoded alpha value of zero at a step 3400 and, at a step 3410 users a zero slope in the linear model. The question may then arise as to why the linear model is selected (at the mode selection stage) and encoded when there is a zero slope encoded, indicating no useful dependence between the target and source colour components for that block. One reason could be that the linear model is selectable either for both (target) colour components collectively, for example the linear model is selectable for both of Cb, Cr or not at all, such that if a useful linear model is derivable for one colour component but not for the other it can still provide a net benefit (as indicated by the cost function) to employ the linear model mode but with a zero slope encoded for one of the colour components.

Therefore, in examples, a mode selector is configured to select whether or not to use the component model mode in dependence upon a cost function dependent at least in part on a quantity of data required to encode the model data at an encoding resolution applicable to the region size and/or region shape of the current image region.

Another potential meaning for an encoded alpha value of 0 is shown schematically in Figures 35 and 36, which again represent operations at the encoder and decoder side respectively. In Figure 35, a detection is made at a step 3500 during the mode selection process that for a particular target colour component (for example either Cb or Cr), it is in fact of benefit in terms of the cost function to use the same prediction mode as that used for the luminance Y component. To represent this, the encoder encodes an alpha value of 0 at a step 3510. At the decoder side (Figure 36) a detection is made at a step 3600 that the alpha value encoded in the data stream is 0 and at a step 3610 the decoder is constrained to use the same prediction mode as that employed for the corresponding luminance block.

Figure 37 is a schematic flowchart illustrating an image encoding method comprising: predicting (at a step 3700) samples for respective components of a set of colour 35 components; encoding (at a step 3710) respective component residual data representing a difference between input samples for the current image region and the respective predicted samples for respective ones of the set of colour components; decoding (at a step 3720) the residual data and to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the predicting step being selectively operable in a component model mode in which the predicting step comprises generating predicted samples of a target set of at least one of the colour components from decoded samples of a source set of the colour components by applying a component model function to the decoded samples of the source set of the colour components; and generating (at a step 3730) model data at least partially defining the component model function from input samples of the source set of the colour components and input samples of the target set of the colour components.

Figure 38 is a schematic flowchart illustrating an image decoding method responsive to an input encoded data stream, the method comprising: predicting (at a step 3800) samples for respective components of a set of colour components; decoding (at a step 3810) respective residual data representing a difference between output samples for the current image region and the respective predicted samples for respective ones of the set of colour components; generating (at a step 3830) decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the predicting step being selectively operable in a component model mode in which the predicting step comprises generating the predicted samples of a target set of at least one of the colour components from the decoded samples of a source set of at least one of the colour components by applying a component model function to the decoded samples of the source set of at least one of the colour components, the component model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.

Figure 39 is a schematic flowchart illustrating an image encoding method comprising: encoding (at a step 3900) samples of one or more colour components for respective image regions of an ordered succession of image regions so as to generate encoded image data; and decoding (at a step 3910) the encoded image data so as to generate decoded samples; in which the encoding step comprises: deriving (at a step 3920) predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted target samples for a colour component in dependence upon decoded source samples of another colour component by applying a component model function to the decoded source samples; and selectively generating and encoding (at a step 3930) model data at least partially defining the component model function for the current image region in dependence upon a difference between the model data for the current image region and model data for a previously encoded image region.

Figure 40 is a schematic flowchart illustrating an image decoding method responsive to an input encoded data stream comprising encoded image data and operable in respect of samples of one or more colour components for respective image regions of an ordered succession of image regions, the method comprising: deriving (at a step 4000) predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted target samples for a colour component in dependence upon decoded source samples of another colour component by applying a component model function to the decoded source samples; and decoding (at a step 4010) the encoded image data so as to generate decoded samples; in which the component model for at least some of the image regions is defined in the input encoded data stream at least in part by model data encoded as a difference between the model data for the current image region and model data for a previously encoded image region.

Figure 41is a schematic flowchart illustrating an image encoding method comprising: encoding (at a step 4100) samples of one or more colour components for respective image regions so as to generate encoded image data; and decoding (at a step 4110) the encoded image data so as to generate decoded samples; in which the encoding step comprises: selecting (at a step 4120) one or both of a region size and a region shape for a current image region; deriving (at a step 4130) predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples for a target set of one of more of the colour components in dependence upon decoded samples of a source set of one or more of the colour components by applying a component model function to the decoded source samples; and encoding (at a step 4140) model data at least partially defining the component model function for the current image region, according to an encoding resolution dependent upon one or both of the region size and the region shape for the current image region.

Figure 42 is a schematic flowchart illustrating an image decoding method responsive to an input encoded data stream comprising encoded data representing respective image regions having a region size and a region shape, the method comprising: deriving (at a step 4200) predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples for a target set of one of more of the colour components in dependence upon decoded samples of a source set of one or more of the colour components by applying a component model function to the decoded source samples; and decoding (at a step 4210) the encoded image data so as to generate decoded samples; in which the encoded data stream comprises model data at least partially defining the component model function for the current image region, the model data being encoded in the encoded data stream according to an encoding resolution dependent upon one or both of the region size and the region shape for the current image region.

Figure 43 is a summary schematic diagram of an encoding apparatus illustrating the various techniques discussed above (all of which may be embodied individually or in combination, unless the technical context specifically prohibits such a combination). An actual embodiment may use some or all of the features shown depending on the technical context of the particular aspect of the disclosure.

The encoding apparatus comprises an intra-image predictor 4300 configured to derive predicted samples for respective components of a set of colour components. the intra-image predictor may be selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples of a target set of at least one of the colour components from decoded samples of a source set of the colour components by applying a component model function to the decoded samples of the source set of the colour components. As discussed above, although in some examples the source set of colour components may be the Y (luminance) component, this is not a requirement, and indeed it is not a requirement that a Y Cb Cr component system is used. IN other words, the intra-image predictor is configured to derive predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples for a target set of one of more of the colour components in dependence upon decoded samples of a source set of one or more of the colour components by applying a component model function to the decoded source samples.

A decoding stage 4310 (equivalent to at least a part of the return path of Figure 7) is configured to decode the residual data and to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component. In other words, the decoding stage is configured to decode the encoded image data so as to generate decoded samples.

A model function generator 4320 is configured to generate model data at least partially defining the component model function from input samples of the source set of the colour components and input samples of the target set of the colour components, using the techniques discussed above.

Optionally, a residual data encoder 4330 is configured to encode respective component residual data representing a difference between input samples for the current image region and the respective predicted samples for respective ones of the set of colour components.

A mode selector 4340 is configured to select the encoding mode for a particular image region, the region size and/or shape for a current image region being set by a region controller 4350. Note that it is not essential to at least some examples to employ a region controller. In some examples, however, this provides an example of the use of a region controller configured to select one or both of a region size and a region shape for a current image region; and a mode selector configured to select whether the current region should be encoded using the component model mode.

Overall, the apparatus receives input image data 4360 and generates an encoded data stream 4370 of encoded image data. The stream 4370 can include at least a representation of the model data (for example, the alpha and/or beta parameters, or a "re-use" flag, or difference data, or identification data or the like). Therefore, the model function generator may be configured to provide the model data in association with an encoded data stream generated by the apparatus.

In at least some examples, the model function generator 4320 is configured to encode model data at least partially defining the component model function for the current image region, in which the model function generator is configured to encode the model data according to an encoding resolution dependent upon one or both of the region size and the region shape for the current image region.

In at least some examples, the model function generator 4320 is configured to selectively generate and encode model data at least partially defining the component model function for the current image region in dependence upon a difference between the model data for the current image region and model data for a previously encoded image region.

Figure 44 is a summary schematic diagram of a decoding apparatus illustrating the various techniques discussed above (all of which may be embodied individually or in combination, unless the technical context specifically prohibits such a combination). An actual embodiment may use some or all of the features shown depending on the technical context of the particular aspect of the disclosure.

The image decoding apparatus of Figure 44 is responsive to an input encoded data stream comprising encoded image data, the apparatus being operable in respect of samples of one or more colour components, which may be for respective image regions of an ordered succession of image regions.

An intra-image predictor 4400 is configured to derive predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted target samples for a colour component in dependence upon decoded source samples of another colour component by applying a component model function to the decoded source samples.

A decoding stage 4410 is configured to decode the encoded image data so as to generate decoded samples.

In at least some examples, the component model for at least some of the image regions is defined in the input encoded data stream at least in part by model data encoded as a difference between the model data for the current image region and model data for a previously encoded image region.

In at least some other (overlapping or different) examples, the encoded data stream comprises model data at least partially defining the component model function for the current image region, the model data being encoded in the encoded data stream according to an encoding resolution dependent upon one or both of the region size and the region shape for the current image region.

In at least some other (overlapping or different) examples, the intra-image predictor 4400 is selectively operable in a component model mode in which the intra-image predictor is configured to generate the predicted samples of a target set of at least one of the colour components from the decoded samples of a source set of at least one of the colour components by applying a component model function to the decoded samples of the source set of the colour components, the component model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.

In order to retrieve or decode or otherwise obtain the component model data from the input encoded data stream, a model data decoder 4420 may be employed. This can operate according to the relevant format of data defined above. For example, in the case of re-use of data, the decoder 4420 can provide the store 2600 (and select amongst multiple such stores where applicable). In the case of difference data, the decoder 4420 can apply the relevant difference. IN the case of different resolution or quantisation data, the decoder 4420 can dequantise the model data as appropriate.

The decoding stage 4410 is configured to decode samples for a colour component by combining the decoded residual data and the predicted samples for that colour component.

In some (though not necessarily all) examples the decoding stage 4410 may provide a residual data decoder 4412 configured to decode respective component residual data representing a difference between output samples for the current image region and the respective predicted samples for respective ones of the set of colour components and a combiner 4414 configured to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component.

In at least some examples, the received model data (as part of an input encoded data stream 4405 defines one or both of a gradient and an intercept of a linear relationship between samples of the target set of colour components and samples of the source set of colour components.

The apparatus generates decoded image data samples 4415.

In examples, the predictor 4400 may comprise a model function generator is configured to generate intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded samples of the source set of one or more colour components and decoded samples of the target set of one or more colour components, at least the decoded samples of the target set of one or more colour components being from outside the current image region.

With regard to the dependency upon decoded samples, where the intra-image predictor uses the component model to generate predicted samples, this can be in dependence upon model data generated from input samples of the source and target components for the current image region 4500 (Figure 45) under consideration, and/or from reference sample positions 4510, 4520 such as previously encoded and decoded samples positions around the current image region 4500.

In so far as embodiments of the disclosure have been described as being implemented, at least in part, by software-controlled data processing apparatus, it will be appreciated that a non-transitory machine-readable medium carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure. Similarly, a data signal comprising coded data generated according to the methods discussed above (whether or not embodied on a non-transitory machine-readable medium) is also considered to represent an embodiment of the present disclosure.

It will be apparent that numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended clauses, the technology may be practised otherwise than as specifically described herein.

It will be appreciated that the above description for clarity has described embodiments with reference to different functional units, circuitry and/or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuitry and/or processors may be used without detracting from the embodiments.

Described embodiments may be implemented in any suitable form including hardware, software, firmware or any combination of these. Described embodiments may optionally be implemented at least partly as computer software running on one or more data processors and/or digital signal processors. The elements and components of any embodiment may be physically, functionally and logically implemented in any suitable way. Indeed the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the disclosed embodiments may be implemented in a single unit or may be physically and functionally distributed between different units, circuitry and/or processors.

Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to the specific form set forth herein. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in any manner suitable to implement the technique.

Respective aspects and features are defined by the following numbered clauses: 1. An image encoding apparatus comprising: an intra-image predictor configured to derive predicted luminance and chrominance samples; a residual data encoder configured to encode respective luminance and chrominance residual data representing a difference between input luminance and chrominance samples for the current image region and the respective predicted samples; a decoding stage configured to decode the residual data and to generate decoded luminance samples by combining the decoded residual data and the predicted luminance samples; the intra-image predictor being selectively operable in a chrominance model mode in which the intra-image predictor is configured to generate the predicted chrominance samples from the decoded luminance samples by applying a chrominance model function to the decoded luminance samples; and a model function generator configured to generate model data at least partially defining the chrominance model function from the input luminance samples and the input chrominance samples.

2. Apparatus according to clause 1, in which the chrominance model function represents a linear relationship between predicted chrominance sample values and the decoded luminance sample values.

3. Apparatus according to clause 2, in which the model function generator is configured to detect a linear regression between the input luminance samples and the input chrominance samples so as to detect gradient data defining at least a gradient of the linear relationship.

4. Apparatus according to clause 3, in which the model function generator is configured to generate intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded luminance samples and the decoded chrominance samples.

5. Apparatus according to clause 4 in which the model function generator is configured to generate the intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded luminance samples and decoded chrominance samples, at least the decoded chrominance samples being from outside the current image region.

6. Apparatus according to any one of clauses 2 to 5, in which the model function generator is configured to detect a linear regression between the input luminance samples and the input chrominance samples so as to detect data defining a gradient and an intercept of the linear relationship.

7. Apparatus according to any one of the preceding clauses, in which the model function generator is configured to provide the model data in association with an encoded data stream generated by the apparatus.

8. An image decoding apparatus responsive to an input encoded data stream, the apparatus comprising: an intra-image predictor configured to derive predicted luminance and chrominance samples; a residual data decoder configured to decode respective luminance and chrominance residual data representing a difference between input luminance and chrominance samples for the current image region and the respective predicted samples; a combiner configured to generate decoded luminance samples by combining the decoded residual data and the predicted luminance samples; the intra-image predictor being selectively operable in a chrominance model mode in which the intra-image predictor is configured to generate the predicted chrominance samples from the decoded luminance samples by applying a chrominance model function to the decoded luminance samples, the chrominance model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.

9. Apparatus according to clause 8, in which the chrominance model function represents a linear relationship between predicted chrominance sample values and the decoded luminance sample values.

10. Apparatus according to clause 9, in which the received model data comprises gradient data defining at least a gradient of the linear relationship; the apparatus comprising: a model function generator configured to detect intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded luminance samples and the decoded chrominance samples.

11. Apparatus according to clause 10, in which the model function generator is configured to generate the intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded luminance samples and decoded chrominance samples, at least the decoded chrominance samples being from outside the current image region.

12. Apparatus according to any one of clauses 9 to 11, in which the received model data defines a gradient and an intercept of the linear relationship.

13. Video storage, capture, transmission or reception apparatus comprising apparatus according to any one of clauses 1 to 7.

14. Video storage, capture, transmission or reception apparatus comprising apparatus according to any one of clauses 8 to 12.

15. An image encoding method comprising: predicting luminance and chrominance samples; encoding respective luminance and chrominance residual data representing a difference between input luminance and chrominance samples for the current image region and the respective predicted samples; decoding the residual data and to generate decoded luminance samples by combining the decoded residual data and the predicted luminance samples; the predicting step being selectively operable in a chrominance model mode in which the predicting step comprises generating the predicted chrominance samples from the decoded luminance samples by applying a chrominance model function to the decoded luminance samples; and generating model data at least partially defining the chrominance model function from the input luminance samples and the input chrominance samples.

16. Computer software which, when executed by a computer, causes the computer to carry out a method according to clause 15.

17. A machine-readable non-transitory storage medium which stores software according to clause 16.

18. A data signal comprising coded data generated according to the method of clause 15.

19. An image decoding method responsive to an input encoded data stream, the method comprising: predicting luminance and chrominance samples; decoding respective luminance and chrominance residual data representing a difference between input luminance and chrominance samples for the current image region and the respective predicted samples; generating decoded luminance samples by combining the decoded residual data and the predicted luminance samples; the predicting step being selectively operable in a chrominance model mode in which the predicting step comprises generating the predicted chrominance samples from the decoded luminance samples by applying a chrominance model function to the decoded luminance samples, the chrominance model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.

20. Computer software which, when executed by a computer, causes the computer to carry out a method according to clause 19.

21. A machine-readable non-transitory storage medium which stores software according to clause 20.

Further respective aspects and features are defined by the following numbered clauses: 1. An image encoding apparatus comprising: an intra-image predictor configured to derive predicted samples for respective components of a set of colour components; a residual data encoder configured to encode respective component residual data representing a difference between input samples for the current image region and the respective predicted samples for respective ones of the set of colour components; a decoding stage configured to decode the residual data and to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples of a target set of at least one of the colour components from decoded samples of a source set of the colour components by applying a component model function to the decoded samples of the source set of the colour components; and a model function generator configured to generate model data at least partially defining the component model function from input samples of the source set of the colour components and input samples of the target set of the colour components.

2. Apparatus according to clause 1, in which the set of colour components comprises first, second and third colour components, and the intra-image predictor is operable in the component model mode to generate: predicted samples of the second colour component from decoded samples of the first colour component using a first component model; and predicted samples of the third colour component from decoded samples of the second colour component using a second component model.

3. Apparatus according to clause 2, in which the first colour component is a luminance component and the second and third colour components are respective chrominance components.

4. Apparatus according to clause 3, in which: the chrominance components comprise a blue-difference component and a red-difference component; the second colour component is the blue-difference component; and the third colour component is the red-difference component.

5. Apparatus according to any one of clauses 1 to 4, in which: the set of colour components comprises luminance and two chrominance components; the source set of at least one of the colour components comprises the luminance component; and the target set of at least one of the colour components comprises the chrominance components.

6. Apparatus according to any one of the preceding clauses, in which the component model function represents a linear relationship between predicted sample values of the target set of one or more colour components and decoded sample values of the source set of one of more colour components.

7. Apparatus according to clause 6, in which the model function generator is configured to detect a linear regression between the input samples of the target set of one or more colour components and the input samples of the source set of one of more colour components so as to detect gradient data defining at least a gradient of the linear relationship.

8. Apparatus according to clause 7, in which the model function generator is configured to generate intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded samples of the source set of one or more colour components and the decoded samples of the target set of one or more colour components.

9. Apparatus according to clause 8, in which the model function generator is configured to generate the intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded samples of the source set of one or more colour components and decoded samples of the target set of one or more colour components, at least the decoded samples of the target set of one or more colour components being from outside the current image region.

10. Apparatus according to clause 2, in which the model function generator is configured to detect a linear regression between the input samples of the source set of one or more colour components and the input samples of the target set of one or more colour components so as to detect data defining a gradient and an intercept of the linear relationship.

11. Apparatus according to any one of the preceding clauses, in which the model function generator is configured to provide the model data in association with an encoded data stream generated by the apparatus.

12. An image decoding apparatus responsive to an input encoded data stream, the apparatus comprising: an intra-image predictor configured to derive predicted samples for respective components of a set of colour components; a residual data decoder configured to decode respective component residual data representing a difference between output samples for the current image region and the respective predicted samples for respective ones of the set of colour components; a decoding stage configured to decode samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; a combiner configured to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate the predicted samples of a target set of at least one of the colour components from the decoded samples of a source set of at least one of the colour components by applying a component model function to the decoded samples of the source set of the colour components, the component model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.

13. Apparatus according to clause 12, in which the set of colour components comprises first, second and third colour components, and the intra-image predictor is operable in the component model mode to generate: predicted samples of the second colour component from decoded samples of the first colour component using a first component model; and predicted samples of the third colour component from decoded samples of the second colour component using a second component model.

14. Apparatus according to clause 13, in which the first colour component is a luminance component and the second and third colour components are respective chrominance components.

15. Apparatus according to clause 14, in which: the chrominance components comprise a blue-difference component and a red-difference component; the second colour component is the blue-difference component; and the third colour component is the red-difference component.

16. Apparatus according to clause 12, in which: the set of colour components comprises luminance and two chrominance components; the source set of at least one of the colour components comprises the luminance component; and the target set of at least one of the colour components comprises the chrominance components.

17. Apparatus according to clause 16, in which the chrominance model function represents a linear relationship between predicted sample values of the target set of at least one of the colour components and the decoded sample values of the source set of at least one of the colour components.

18. Apparatus according to clause 17, in which the received model data comprises gradient data defining at least a gradient of the linear relationship; the apparatus comprising: a model function generator configured to detect intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded samples of the source set of at least one of the colour components and the decoded chrominance samples of the target set of at least one of the colour components.

19. Apparatus according to clause 18, in which the model function generator is configured to generate the intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded samples of the source set of at least one of the colour components and decoded samples of the target set of at least one of the colour components, at least the decoded samples of the target set of at least one of the colour components being from outside the current image region.

20. Apparatus according to clause 17, in which the received model data defines a gradient and an intercept of the linear relationship.

21. Video storage, capture, transmission or reception apparatus comprising apparatus according to any one of clauses 1 to 11.

22. Video storage, capture, transmission or reception apparatus comprising apparatus according to any one of clauses 12 to 20.

23. An image encoding method comprising: predicting samples for respective components of a set of colour components; encoding respective component residual data representing a difference between input samples for the current image region and the respective predicted samples for respective ones of the set of colour components; decoding the residual data and to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the predicting step being selectively operable in a component model mode in which the predicting step comprises generating predicted samples of a target set of at least one of the colour components from decoded samples of a source set of the colour components by applying a component model function to the decoded samples of the source set of the colour components; and generating model data at least partially defining the component model function from input samples of the source set of the colour components and input samples of the target set of the colour components.

24. Computer software which, when executed by a computer, causes the computer to carry out a method according to clause 23.

25. A machine-readable non-transitory storage medium which stores software according to clause 24.

26. A data signal comprising coded data generated according to the method of clause 23.

27. An image decoding method responsive to an input encoded data stream, the method comprising: predicting samples for respective components of a set of colour components; decoding respective residual data representing a difference between output samples for the current image region and the respective predicted samples for respective ones of the set of colour components; generating decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the predicting step being selectively operable in a component model mode in which the predicting step comprises generating the predicted samples of a target set of at least one of the colour components from the decoded samples of a source set of at least one of the colour components by applying a component model function to the decoded samples of the source set of at least one of the colour components, the component model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.

28. Computer software which, when executed by a computer, causes the computer to carry out a method according to clause 27.

29. A machine-readable non-transitory storage medium which stores software according to clause 28.

Further respective aspects and features are defined by the following numbered clauses: 1. An image encoding apparatus comprising: an image data encoder configured to encode samples of one or more colour components for respective image regions so as to generate encoded image data; and a decoding stage configured to decode the encoded image data so as to generate decoded samples; in which the image data encoder comprises: a region controller configured to select one or both of a region size and a region shape for a current image region; an intra-image predictor configured to derive predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples for a target set of one of more of the colour components in dependence upon decoded samples of a source set of one or more of the colour components by applying a component model function to the decoded source samples; and a model function generator configured to encode model data at least partially defining the component model function for the current image region, in which the model function generator is configured to encode the model data according to an encoding resolution dependent upon one or both of the region size and the region shape for the current image region.

2. Apparatus according to clause 1, in which the component model function represents a linear relationship between predicted target sample values and decoded source samples.

3. Apparatus according to clause 2, in which the model function generator is configured to generate the model data from input samples of the source set of the colour components and input samples of the target set of the colour components.

4. Apparatus according to clause 3, in which the model function generator is configured to detect a linear regression between the input samples of the target set of colour components and the input samples of the source set of colour components so as to detect gradient data defining at least a gradient of the linear relationship.

5. Apparatus according to clause 4, in which the model function generator is configured to generate intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded samples of the source set of one or more colour components and the decoded samples of the target set of one or more colour components.

6. Apparatus according to clause 5, in which the model function generator is configured to generate the intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded samples of the source set of one or more colour components and decoded samples of the target set of one or more colour components, at least the decoded samples of the target set of one or more colour components being from outside the current image region.

7. Apparatus according to clause 3, in which the model function generator is configured to detect a linear regression between the input samples of the source set of one or more colour components and the input samples of the target set of one or more colour components so as to detect data defining a gradient and an intercept of the linear relationship.

8. Apparatus according to any one of the preceding clauses, in which the model function generator is configured to select one of multiple encoding resolutions in dependence upon whether the current image region has one or both of the region size and the region shape in a respective range.

9. Apparatus according to any one of the preceding clauses, comprising a mode selector configured to select whether or not to use the component model mode in dependence upon a cost function dependent at least in part on a quantity of data required to encode the model data at an encoding resolution applicable to the region size and/or region shape of the current image region.

10. Apparatus according to any one of the preceding clauses, in which: the set of colour components comprises luminance and two chrominance components; the source set of at least one of the colour components comprises the luminance component; and the target set of at least one of the colour components comprises one or both of the chrominance components.

12. An image decoding apparatus responsive to an input encoded data stream comprising encoded data representing respective image regions having a region size and a region shape, the apparatus comprising: an intra-image predictor configured to derive predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples for a target set of one of more of the colour components in dependence upon decoded samples of a source set of one or more of the colour components by applying a component model function to the decoded source samples; and a decoding stage configured to decode the encoded image data so as to generate decoded samples; in which the encoded data stream comprises model data at least partially defining the component model function for the current image region, the model data being encoded in the encoded data stream according to an encoding resolution dependent upon one or both of the region size and the region shape for the current image region.

13. Apparatus according to clause 12, in which the received model data defines one or both of a gradient and an intercept of a linear relationship between samples of the target set of colour components and samples of the source set of colour components.

14. Apparatus according to clause 13, in which the model function generator is configured to generate intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded samples of the source set of one or more colour components and decoded samples of the target set of one or more colour components, at least the decoded samples of the target set of one or more colour components being from outside the current image region.

15. Apparatus according to any one of clauses 12 to 14, in which the model data is encoded in the encoded data stream according to one of multiple encoding resolutions in dependence upon whether the current image region has one or both of the region size and the region shape in a respective range.

16. Apparatus according to any one of clauses 12 to 15, in which: the set of colour components comprises luminance and two chrominance components; the source set of at least one of the colour components comprises the luminance component; and the target set of at least one of the colour components comprises one or both of the chrominance components.

17. Video storage, capture, transmission or reception apparatus comprising apparatus according to any one of clauses 1 to 13.

18. Video storage, capture, transmission or reception apparatus comprising apparatus according to any one of clauses 12 to 16.

19. An image encoding method comprising: encoding samples of one or more colour components for respective image regions so as to generate encoded image data; and decoding the encoded image data so as to generate decoded samples; in which the encoding step comprises: selecting one or both of a region size and a region shape for a current image region; deriving predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples for a target set of 30 one of more of the colour components in dependence upon decoded samples of a source set of one or more of the colour components by applying a component model function to the decoded source samples; and encoding model data at least partially defining the component model function for the current image region, according to an encoding resolution dependent upon one or both of the region size and the region shape for the current image region.

22. A data signal comprising coded data generated according to the method of clause 12.

23. An image decoding method responsive to an input encoded data stream comprising encoded data representing respective image regions having a region size and a region shape the method comprising: deriving predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples for a target set of one of more of the colour components in dependence upon decoded samples of a source set of one or more of the colour components by applying a component model function to the decoded source samples; and decoding the encoded image data so as to generate decoded samples; in which the encoded data stream comprises model data at least partially defining the component model function for the current image region, the model data being encoded in the encoded data stream according to an encoding resolution dependent upon one or both of the region size and the region shape for the current image region.

Further respective aspects and features are defined by the following numbered clauses: 1. An image encoding apparatus comprising: an image data encoder configured to encode samples of one or more colour components for respective image regions of an ordered succession of image regions so as to generate encoded image data; and a decoding stage configured to decode the encoded image data so as to generate decoded samples; in which the image data encoder comprises: an intra-image predictor configured to derive predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted target samples for a colour component in dependence upon decoded source samples of another colour component by applying a component model function to the decoded source samples; and a model function generator configured to selectively generate and encode model data at least partially defining the component model function for the current image region in dependence upon a difference between the model data for the current image region and model data for a previously encoded image region.

2. Apparatus according to clause 1, in which the previously encoded image region is a most recently encoded image region in the ordered succession of image regions, for which the intra-image predictor used the component model.

3. Apparatus according to clause 1 or clause 2, in which the model function generator is configured to encode data identifying the previously encoded image region.

4. Apparatus according to any one of the preceding clauses, in which, if the difference between the model data for the current image region and the model data for the previously encoded image region is less than a threshold difference, the model function generator is configured to encode a difference value.

5. Apparatus according to any one of the preceding clauses, in which, if the difference between the model data for the current image region and the model data for the previously encoded image region is zero, the model function generator is configured to encode a flag indicating re-use of the model data for the previously encoded image region.

6. Apparatus according to any one of the preceding clauses, in which the model function generator is configured to generate model data in respect of two or more of the colour components and to encode the model data for each of the two or more colour components in dependence upon a difference between the model data for that colour component for the current image region and model data for that colour component for a previously encoded image region 7. Apparatus according to clause 5, comprising: a region controller configured to select one or both of a region size and a region shape for a current image region; and a mode selector configured to select whether the current region should be encoded using the component model mode.

8. Apparatus according to clause 7, in which, for a current region size below a threshold size, the mode selector is configured to select use of the component model mode only when the model data for the current region can be encoded by the flag indicating re-use of the model data for the previously encoded image region.

9. Apparatus according to clause 7, in which, for a current region size below a threshold size, the mode selector is configured to inhibit re-use of the model data for the previously encoded image region.

10. Apparatus according to clause 8, in which, for image regions at a set of predetermined image locations or after encoding a predetermined number of image regions using the flag indicating re-use of the model data for the previously encoded image region, the mode selector is configured to allow the selection of the component model mode independently of whether when the model data for the current region can be encoded by the flag indicating re-use of the model data for the previously encoded image region.

11. An image decoding apparatus responsive to an input encoded data stream comprising encoded image data, the apparatus being operable in respect of samples of one or more colour components for respective image regions of an ordered succession of image regions and comprising: an intra-image predictor configured to derive predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted target samples for a colour component in dependence upon decoded source samples of another colour component by applying a component model function to the decoded source samples; and a decoding stage configured to decode the encoded image data so as to generate decoded samples; in which the component model for at least some of the image regions is defined in the input encoded data stream at least in part by model data encoded as a difference between the model data for the current image region and model data for a previously encoded image region.

12. Apparatus according to clause 11, in which the previously encoded image region is a most recently encoded image region in the ordered succession of image regions, for which the intra-image predictor used the component model.

13. Apparatus according to clause 11 or clause 12, in which the model function generator is configured to encode data identifying the previously encoded image region.

14. Apparatus according to clause 12, in which the model data selectively comprises a difference value between the model data for the current image region and the model data for the previously encoded image region.

15. Apparatus according to clause 12, in which the model data selectively comprises a flag indicating re-use, for the current image region, of the model data for the previously encoded image region.

16. Apparatus according to clause 11, in which the intra-image predictor is responsive to model data in respect of two or more of the colour components.

17. Video storage, capture, transmission or reception apparatus comprising apparatus according to any one of clauses 1 to 10.

18. Video storage, capture, transmission or reception apparatus comprising apparatus according to any one of clauses 11 to 16.

19. An image encoding method comprising: encoding samples of one or more colour components for respective image regions of an ordered succession of image regions so as to generate encoded image data; and decoding the encoded image data so as to generate decoded samples; in which the encoding step comprises: deriving predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted target samples for a colour component in dependence upon decoded source samples of another colour component by applying a component model function to the decoded source samples; and selectively generating and encoding model data at least partially defining the component model function for the current image region in dependence upon a difference between the model data for the current image region and model data for a previously encoded image region.

22. A data signal comprising coded data generated according to the method of clause 19.

23. An image decoding method responsive to an input encoded data stream comprising encoded image data and operable in respect of samples of one or more colour components for respective image regions of an ordered succession of image regions, the method comprising: deriving predicted blocks of samples of one or more colour components for the current image region, the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted target samples for a colour component in dependence upon decoded source samples of another colour component by applying a component model function to the decoded source samples; and decoding the encoded image data so as to generate decoded samples; in which the component model for at least some of the image regions is defined in the input encoded data stream at least in part by model data encoded as a difference between the model data for the current image region and model data for a previously encoded image region.

Claims

CLAIMS1. An image encoding apparatus comprising: an intra-image predictor configured to derive predicted samples for respective components of a set of colour components; a residual data encoder configured to encode respective component residual data representing a difference between input samples for the current image region and the respective predicted samples for respective ones of the set of colour components; a decoding stage configured to decode the residual data and to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate predicted samples of a target set of at least one of the colour components from decoded samples of a source set of the colour components by applying a component model function to the decoded samples of the source set of the colour components; and a model function generator configured to generate model data at least partially defining the component model function from input samples of the source set of the colour components and input samples of the target set of the colour components.
2. Apparatus according to claim 1, in which the set of colour components comprises first, second and third colour components, and the intra-image predictor is operable in the component model mode to generate: predicted samples of the second colour component from decoded samples of the first colour component using a first component model; and predicted samples of the third colour component from decoded samples of the second colour component using a second component model.
3. Apparatus according to claim 2, in which the first colour component is a luminance component and the second and third colour components are respective chrominance components.
4. Apparatus according to claim 3, in which: the chrominance components comprise a blue-difference component and a red-difference component; the second colour component is the blue-difference component; and the third colour component is the red-difference component.
5. Apparatus according to claim 1, in which: the set of colour components comprises luminance and two chrominance components; the source set of at least one of the colour components comprises the luminance component; and the target set of at least one of the colour components comprises the chrominance components.
6. Apparatus according to claim 1, in which the component model function represents a linear relationship between predicted sample values of the target set of one or more colour components and decoded sample values of the source set of one of more colour components.
7. Apparatus according to claim 6, in which the model function generator is configured to detect a linear regression between the input samples of the target set of one or more colour components and the input samples of the source set of one of more colour components so as to detect gradient data defining at least a gradient of the linear relationship.
8. Apparatus according to claim 7, in which the model function generator is configured to generate intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded samples of the source set of one or more colour components and the decoded samples of the target set of one or more colour components.
9. Apparatus according to claim 8, in which the model function generator is configured to generate the intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded samples of the source set of one or more colour components and decoded samples of the target set of one or more colour components, at least the decoded samples of the target set of one or more colour components being from outside the current image region.
10. Apparatus according to claim 2, in which the model function generator is configured to detect a linear regression between the input samples of the source set of one or more colour components and the input samples of the target set of one or more colour components so as to detect data defining a gradient and an intercept of the linear relationship.
11. Apparatus according to claim 1, in which the model function generator is configured to provide the model data in association with an encoded data stream generated by the apparatus.
12. An image decoding apparatus responsive to an input encoded data stream, the apparatus comprising: an intra-image predictor configured to derive predicted samples for respective components of a set of colour components; a residual data decoder configured to decode respective component residual data representing a difference between output samples for the current image region and the respective predicted samples for respective ones of the set of colour components; a decoding stage configured to decode samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; a combiner configured to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the intra-image predictor being selectively operable in a component model mode in which the intra-image predictor is configured to generate the predicted samples of a target set of at least one of the colour components from the decoded samples of a source set of at least one of the colour components by applying a component model function to the decoded samples of the source set of the colour components, the component model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.
13. Apparatus according to claim 12, in which the set of colour components comprises first, second and third colour components, and the intra-image predictor is operable in the component model mode to generate: predicted samples of the second colour component from decoded samples of the first colour component using a first component model; and predicted samples of the third colour component from decoded samples of the second colour component using a second component model.
14. Apparatus according to claim 13, in which the first colour component is a luminance component and the second and third colour components are respective chrominance 30 components.
15. Apparatus according to claim 14, in which: the chrominance components comprise a blue-difference component and a red-difference component; the second colour component is the blue-difference component; and the third colour component is the red-difference component.
16. Apparatus according to claim 12, in which: the set of colour components comprises luminance and two chrominance components; the source set of at least one of the colour components comprises the luminance component; and the target set of at least one of the colour components comprises the chrominance components.
17. Apparatus according to claim 16, in which the chrominance model function represents a linear relationship between predicted sample values of the target set of at least one of the colour components and the decoded sample values of the source set of at least one of the colour components.
18. Apparatus according to claim 17, in which the received model data comprises gradient data defining at least a gradient of the linear relationship; the apparatus comprising: a model function generator configured to detect intercept data defining an intercept of the linear relationship by applying the gradient defined by the gradient data to the decoded samples of the source set of at least one of the colour components and the decoded chrominance samples of the target set of at least one of the colour components.
19. Apparatus according to claim 18, in which the model function generator is configured to generate the intercept data in respect of a current image region by applying the gradient defined by the gradient data to decoded samples of the source set of at least one of the colour components and decoded samples of the target set of at least one of the colour components, at least the decoded samples of the target set of at least one of the colour components being from outside the current image region.
20. Apparatus according to claim 17, in which the received model data defines a gradient and an intercept of the linear relationship.
21. Video storage, capture, transmission or reception apparatus comprising apparatus according to claim 1.
22. Video storage, capture, transmission or reception apparatus comprising apparatus according to claim 12.
23. An image encoding method comprising: predicting samples for respective components of a set of colour components; encoding respective component residual data representing a difference between input samples for the current image region and the respective predicted samples for respective ones of the set of colour components; decoding the residual data and to generate decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the predicting step being selectively operable in a component model mode in which the predicting step comprises generating predicted samples of a target set of at least one of the colour components from decoded samples of a source set of the colour components by applying a component model function to the decoded samples of the source set of the colour components; and generating model data at least partially defining the component model function from input samples of the source set of the colour components and input samples of the target set of the colour components.
24. Computer software which, when executed by a computer, causes the computer to carry out a method according to claim 23.
25. A machine-readable non-transitory storage medium which stores software according to claim 24.
26. A data signal comprising coded data generated according to the method of claim 23.
27. An image decoding method responsive to an input encoded data stream, the method comprising: predicting samples for respective components of a set of colour components; decoding respective residual data representing a difference between output samples for the current image region and the respective predicted samples for respective ones of the set of colour components; generating decoded samples for a colour component by combining the decoded residual data and the predicted samples for that colour component; the predicting step being selectively operable in a component model mode in which the predicting step comprises generating the predicted samples of a target set of at least one of the colour components from the decoded samples of a source set of at least one of the colour components by applying a component model function to the decoded samples of the source set of at least one of the colour components, the component model function being defined at least partially by model data received by the apparatus in association with the input encoded data stream.
28. Computer software which, when executed by a computer, causes the computer to carry out a method according to claim 27.
29. A machine-readable non-transitory storage medium which stores software according to claim 28.