GB2597859A

GB2597859A - Data processing apparatuses, methods, computer programs and computer-readable media

Info

Publication number: GB2597859A
Application number: GB2114525.5A
Authority: GB
Inventors: Damnjanovic Ivan
Original assignee: V Nova International Ltd
Current assignee: V Nova International Ltd
Priority date: 2016-09-08
Filing date: 2016-09-08
Publication date: 2022-02-09
Anticipated expiration: 2036-09-08
Also published as: GB202114525D0; GB2597859B

Abstract

First and second pluralities of residual elements 208 are obtained. The residual elements are useable to reconstruct first and second parts of a first representation of a signal 202 from corresponding first and second parts of a second representation 206 of the signal the second representation derivable from elements in a lower quality representation 204 of the signal. The first and second parts of the second representation are derived from different sets of elements in the downscaled representation. A transformation operation uses at least one residual element in the first plurality and at least one residual element in the second plurality to generate multiple correlation elements dependent on an extent of correlation between the residual element in the first plurality and the residual element in the second plurality. The transformation operation is performed prior to the correlation elements being encoded. Data is also generated which indicates that at least some of the plurality of correlation elements are not to be encoded. The transformation may comprise at least one directional decomposition operation.

Description

DATA PROCESSING APPARATUSES, METHODS, COMPUTER PROGRAMS AND COMPUTER-READABLE MEDIA

Technical Field

The present invention relates to data processing apparatuses, methods, computer programs and computer-readable media.

Background

Compression and decompression of signals is an important consideration in 10 many known systems.

Many types of signal, for example video, audio or volumetric signals, may be compressed and encoded for transmission, for example over a data communications network. Other signals may be stored in a compressed form, for example on a storage medium such as a Digital Versatile Disc (DVD) When such a signal is decoded, it may be desired to increase a level of quality of the signal and/or recover as much of the information contained in the original signal as possible Some known systems exploit scalable encoding techniques. Scalable encoding involves encoding a signal along with information to allow the reconstruction of the signal at different levels of quality, depending on the capabilities of the decoder and the available bandwidth. However, relatively large amounts of information may need to be stored and/or transmitted, particularly as the usage of higher quality, higher definition video becomes more widespread

Summary

According to a first aspect of the present invention, there is provided apparatus configured to: obtain a first plurality of residual elements useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the first part of the second representation being derivable from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; obtain a second plurality of residual elements useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal, the second part of the second representation being derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and perform at least one transformation operation involving at least one residual element in the first plurality of residual elements and at least one residual element in the second plurality of residual elements to generate at least one correlation element, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, wherein the apparatus is configured to perform the at least one transformation operation prior to the at least one correlation element being encoded.

According to a second aspect of the present invention, there is provided apparatus configured to: perform at least one transformation operation involving at least one correlation element to generate at least one residual element in a first plurality of residual elements and at least one residual element in a second plurality of residual elements, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, the first plurality of residual elements being useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the second plurality of residual elements being useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal; derive the first part of the second representation from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; derive the second part of the second representation from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and use at least the first plurality of residual elements, the second plurality of residual elements, the first part of the second representation and the second part of the second representation to generate output data, wherein the apparatus is configured to perform the at least one transformation operation following the at least one correlation element being decoded.

According to a third aspect of the present invention, there is provided a method comprising: obtaining a first plurality of residual elements useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the first part of the second representation being derivable from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; obtaining a second plurality of residual elements useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal, the second part of the second representation being derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and performing at least one transformation operation involving at least one residual element in the first plurality of residual elements and at least one residual element in the second plurality of residual elements to generate at least one correlation element, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, wherein the at least one transformation operation is performed prior to the at least one correlation element being encoded.

According to a fourth aspect of the present invention, there is provided a method comprising: performing at least one transformation operation involving at least one correlation element to generate at least one residual element in a first plurality of residual elements and at least one residual element in a second plurality of residual elements, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, the first plurality of residual elements being useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the second plurality of residual elements being useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal; deriving the first part of the second representation from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; deriving the second part of the second representation from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and using at least the first plurality of residual elements, the second plurality of residual elements, the first part of the second representation and the second part of the second representation to generate output data, wherein the at least one transformation operation is performed following the at least one correlation element being decoded.

According to a fifth aspect of the present invention, there is provided a computer program comprising instructions which, when executed, cause an apparatus to perform a method comprising: obtaining a first plurality of residual elements useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the first part of the second representation being derivable from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; obtaining a second plurality of residual elements useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal, the second part of the second representation being derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and performing at least one transformation operation involving at least one residual element in the first plurality of residual elements and at least one residual element in the second plurality of residual elements to generate at least one correlation element, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, wherein the at least one transformation operation is performed prior to the at least one correlation element being encoded According to a sixth aspect of the present invention, there is provided a computer program comprising instructions which, when executed, cause an apparatus to perform a method comprising: performing at least one transformation operation involving at least one correlation element to generate at least one residual element in a first plurality of residual elements and at least one residual element in a second plurality of residual elements, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, the first plurality of residual elements being useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the second plurality of residual elements being useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal; deriving the first part of the second representation from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality, deriving the second part of the second representation from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and using at least the first plurality of residual elements, the second plurality of residual elements, the first part of the second representation and the second part of the second representation to generate output data, wherein the at least one transformation operation is performed following the at least one correlation element being decoded.

Brief Description of the Drawings

Figure 1 shows a schematic block diagram of an example of a signal processing system in accordance with an embodiment of the present invention; Figure 2 shows a schematic diagram of an example of a signal processing technique in accordance with an embodiment of the present invention; Figure 3 shows a schematic diagram of another example of a signal processing technique in accordance with an embodiment of the present invention; and Figure 4 shows a schematic block diagram of an example of an apparatus in accordance with an embodiment of the present invention.

Detailed Description

Referring to Figure 1, there is shown an example of a signal processing system 100. The signal processing system 100 is used to process signals. Examples of types of signal include, but are not limited to, video signals, image signals, audio signals, volumetric signals such as those used in medical, scientific or holographic imaging, or other multidimensional signals.

The signal processing system 100 includes a first apparatus 102 and a second apparatus 104. The first apparatus 102 and second apparatus 104 may have a client-server relationship, with the first apparatus 102 performing the functions of a server device and the second apparatus 104 performing the functions of a client device. The signal processing system 100 may include at least one additional apparatus. The first apparatus 102 and/or second apparatus 104 may comprise one or more components. The components may be implemented in hardware and/or software. The one or more components may be co-located or may be located remotely from each other in the signal processing system 100. Examples of types of apparatus include, but are not limited to, computerised devices, routers, workstations, handheld or laptop computers, tablets, mobile devices, games consoles, smart televisions, set-top boxes, etc..

The first apparatus 102 is communicatively coupled to the second apparatus 104 via a data communications network 106. Examples of the data communications network 106 include, but are not limited to, the Internet, a Local Area Network (LAN) and a Wide Area Network (WAN). The first and/or second apparatus 102, 104 may have a wired and/or wireless connection to the data communications network 106.

The first apparatus 102 comprises an encoder device 108. The encoder device 108 is configured to encode signal data. The encoder device 108 may perform one or more further functions in addition to encoding signal data. The encoder device 108 may be embodied in various different ways. For example, the encoder device 108 may be embodied in hardware and/or software.

The second apparatus 104 comprises a decoder device 110. The decoder device 110 is configured to decode signal data. The decoder device 110 may perform one or more further functions in addition to decoding signal data. The decoder device 110 may be embodied in various different ways. For example, the decoder device 110 may be embodied in hardware and/or software.

The encoder device 108 encodes signal data and transmits the encoded signal data to the decoder device 110 via the data communications network 106. The decoder device 110 decodes the received, encoded signal data and generates decoded signal data. The decoder device 110 may output the decoded signal data, or data derived using the decoded signal data. For example, the decoder device 110 may output such data for display on one or more display devices associated with the second apparatus 104. In some examples described herein, the encoder device 108 transmits to the decoder device 110 a representation of a signal at a given level of quality and information the decoder device 110 can use to reconstruct a representation of the signal at one or more higher levels of quality.

Compared to some known techniques, the examples described herein involve transmitting a relatively small amount of information to be used for such reconstmction. This reduces the amount of data transmitted via the data communications network 106. The savings may be particularly relevant where the signal data corresponds to high quality video data, where the amount of information transmitted in known systems can be especially high.

Examples described herein facilitate more selective encoding of the information the decoder device 110 uses to reconstruct the representation of the signal at the one or more higher levels of quality. In particular, examples described herein convert such information into a form that allows fine-grained control over encoding of individual data elements. This facilitates more efficient and effective encoding, for example where the encoding takes into account the relative importance of the data elements being encoded.

Referring to Figure 2, there is shown schematically an example of a signal processing technique 200. The signal processing technique 200 may be performed by the first apparatus 102.

The first apparatus 102 obtains input data 202. For example, the first apparatus 102 may receive the input data 202 from one or more other entities.

The input data 202 comprises a representation of a signal at a relatively high level of quality. The relatively high level of quality is high relative to one or more other levels of quality in a tiered hierarchy having multiple different levels of quality. The relatively high level of quality may be the highest level of quality in the tiered hierarchy or may be an intermediate level of quality.

In this example, the input data 202 is arranged as a 4x4 array of signal elements comprising four rows and four columns of signal elements. The input data 202 may for example comprise an image and each of the signal elements may represent a pixel comprised in the image.

In this example, the input data 202 comprises four sets of signal elements. A signal element h corresponds to the it signal element in the ith set of signal elements in the input data 202. The different sets of signal elements may correspond to different regions of some or all of an image, for example where the input data 202 comprises image or video data.

The input data 202 is processed to generate data 204 based on the input data 202. In this example, the data 204 is generated by downsampling the input data 202 and is therefore referred to as "downsampled data".

In this example, the downsampled data 204 is arranged as a 2x2 array comprising four signal elements. A downsampled signal element Di corresponds to a downsampled version of the ith set of signal elements in the input data 202.

The downsampled data 204 is a representation of the signal at a relatively low level of quality. The relatively low level of quality is low relative to one or more other levels of quality in the tiered hierarchy having multiple different levels of quality. The relatively low level of quality may be the lowest level of quality in the tiered hierarchy or may be an intermediate level of quality.

The down sampled data 204 is processed to generate data 206. In this example, the data 206 is obtained by upsampling the downsampled data 204 and is therefore referred to as "upsampled data".

In this example, the upsampled data 206 is arranged as a 4x4 array of signal elements comprising four rows and four columns of signal elements. The upsampled data 206 comprises four sets of signal elements. In this example, each of the sets of signal elements is a 2x2 array of signal elements. A signal element I.Jj corresponds to the jth signal element in the set of signal elements in the upsampled data 206. The upsampled data 206 is based on a representation of the signal at a relatively low level of quality, namely the downsampled data 204.

The input data 202 and the upsampled data 206 are used to obtain a set of residual elements 208. The set of residual elements 208 is arranged as a 4x4 array of signal elements comprising four rows and four columns of signal elements. In this example, the set of residual elements 208 comprises four groups of residual elements.

Each of the groups of residual elements comprises a plurality of residual elements. In this example, each group of residual elements is a 2x2 array of residual elements. A residual element Rij corresponds to the ith residual element in the ith group of residual elements in the set of residual elements 208.

In this example, a given residual element in the set of residual elements 208 is obtained by comparing a value of a signal element in the upsampled data 206 with a value of a corresponding signal element in the input data 202.

As such, the first apparatus 102 obtains a first plurality of residual elements, for example residual elements MI, R12, Rii, Ria. In this example, obtaining the first plurality of residual elements involves the first apparatus 102 generating or deriving the first plurality of residual elements, for example using the input data 202 and the upsampled data 206. In other examples, obtaining the first plurality of residual elements may involve the first apparatus 102 receiving the first plurality of residual elements or data useable to derive the first plurality of residual elements.

The first apparatus 102 also obtains a second plurality of residual elements, for example residual elements R2I, R22, R23, R24. In this example, obtaining the second plurality of residual elements involves the first apparatus 102 generating or deriving the second plurality of residual elements, for example using the input data 202 and the upsampled data 206. In other examples, obtaining the second plurality of residual elements may involve the first apparatus 102 receiving the second plurality of residual elements or data useable to derive the second plurality of residual elements.

The set of residual elements 208 is useable in combination with the upsampled data 206 to reconstruct the input data 202. For example, the comparison between the input data 202 and the upsampled data 206 that is used to determine the set of residual elements 208 may be reversed such that the input data 202 can be obtained by comparing the upsamped data 206 with the set of residual elements 208.

As such, the first plurality of residual elements is useable to reconstruct a first part of a first representation of a signal at the relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality. For example, residual elements R11, R12, Rli, R14 are usable to reconstruct the part of the input data 202 consisting of signal elements In, 112, 113, 114 using the part of the upsampled data 206 consisting of signal elements Un, U12, U13, U14. The input data 202 and the upsampled data 206 are both at the relatively high level of quality. The reconstruction may take place at the first apparatus 102, at the second apparatus 104 or elsewhere. The first part of the second representation is derivable from a first set of one or more signal elements in a representation of the signal at the relatively low level of quality. For example, the part of the upsampled data 206 consisting of signal elements Un, U12, U13, U14 is derivable from signal element Di in the downsampled data 204. The downsampled data 204 is a representation of the signal at the relatively low level of quality. The first set of one or more signal elements in the representation of the signal at the relatively low level of quality would, in this example, consist solely of signal element Di.

The second plurality of residual elements is useable to reconstruct a second part of the first representation of the signal at the relatively high level of quality using a second, different part of the second representation of the signal at the relatively high level of quality. For example, residual elements R21, R22, R23, R24 are usable to reconstruct the part of the input data 202 consisting of signal elements 121, 122, 123, 124 using the part of the upsampled data 206 consisting of signal elements Un, U22, U23, U24. The reconstruction may take place at the first apparatus 102, at the second apparatus 104 or elsewhere. The second part of the second representation is derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality. For example, the part of the upsampled data 206 consisting of signal elements U21, U22, U23, U24 is derivable from signal element D2 in the downsampled data 204. The downsampled data 204 is a representation of the signal at the relatively low level of quality. The second set of one or more signal elements in the representation of the signal at the relatively low level of quality would, in this example, consist solely of signal element D2.

The first and second parts of the second representation of the signal at the relatively high level of quality may be different in that at least one of the first and second parts comprises at least one signal element that is not in the other of the first and second parts. The first and second parts of the second representation of the signal at the relatively high level of quality may be different in that the first and second parts do not comprise any signal elements in common.

The first and second sets of one or more signal elements in the representation of the signal at the relatively low level of quality may be different in that at least one of the first and second sets comprises at least one signal element that is not in the other of the first and second sets. The first and second sets of one or more signal elements in the representation of the signal at the relatively low level of quality may be different in that the first and second sets do not comprise any signal elements in common.

The set of residual elements 208 may be transmitted to the second apparatus 104 to allow the second apparatus 104 to reconstruct the input data 202. The set of residual elements 208 may be encoded prior to transmission to reduce the amount of data involved in transmitting the set of residual elements 208 to the second apparatus 104.

In examples described herein, a set of correlation elements 210 is derived by performing at least one transformation operation involving at least some residual elements in the set of residual elements 208. The reader is referred to international patent application no. PCT/EP2013/059847, which published as W02013/171173. PCT/EP2013/059847 describes decomposition of residual data in a tiered hierarchy. The entire contents of PCT/EP2013/059847 are incorporated herein by reference.

As such, the first apparatus 102 performs at least one transformation operation.

Multiple transformation operations may be performed, for example in series and/or in parallel.

The at least one transformation operation involves at least one residual element in the first plurality of residual elements, Ru, R12, R13, R14, and at least one residual element in the second plurality of residual elements, R21, R22, R23, R24. The at least one transformation operation may involve multiple residual elements in the first plurality of residual elements, Ri 1, R12, R13, R14, and/or multiple residual elements in the second plurality of residual elements, R21, R22, R23, R24. The at least one residual element in the first plurality of residual elements, Ri Ri2, Ri 3, R14, and the at least one residual element in the second plurality of residual elements, R21, R22, R23, R24, may be inputs to the at least one transformation operation.

The at least one transformation operation may comprise at least one directional decomposition operation. A directional decomposition operation may exploit correlation. For example, a directional decomposition operation may exploit directional (or 'spatial') correlation. This can lead to more efficient encoding, especially when there is a high degree of such correlation. A directional decomposition operation may result in a set of directional components. Different directional components may be encoded using different encoding parameters, for example according to the relative importance of different directional components in terms of perceived quality. Some directional components may therefore be encoded differently compared to other directional components without a noticeable difference in visual quality. This facilitates more selective encoding, for example to take account of the importance of the individual directional components. In some examples, the precision of encoding of some directional components is increased compared to the precision of encoding of other directional components, for example according to differences in perceptual importance. In some examples, some directional components undergo more compression compared to other directional components. In some examples, some directional components are allocated more bitrate compared to other directional components, for example according to differences in perceptual importance between different directional components. The reader is referred to international patent application no. PCT/EP2013/059847 for examples of how directional decomposition components may be prioritised and encoded.

As is described in more detail below, multiple directional decomposition operations may be performed to further exploit correlation between different sets and types of data elements. This may also lead to more efficient encoding by providing more flexibility in terms of how individual data elements are encoded by allowing groups of data elements that may otherwise all be encoded using the same encoder settings to be decoupled from each other.

In this example, the set of correlation elements 210 is arranged as a 4x4 array of correlation elements comprising four rows and four columns of signal elements. In this example, the set of correlation elements 210 comprises four groups of correlation elements. Each of the groups of correlation elements comprises a plurality of correlation elements. In this example, each group of correlation elements is a 2x2 array of correlation elements. A correlation element au corresponds to the jth correlation element in the ith group of correlation elements in the set of correlation elements 210. The set of correlation elements 210 may be derived by pre-multiplying the set of residual elements 208 with a first transformation (or 'transform') matrix (or 'kernel '), KI. This may be written as cr= KJ. * R, where a represents the set of correlation elements 210 and R represents the set of residual elements 208.

Instead of deriving the set of correlation elements 210 by pre-multiplying the entire set of residual elements 208 with a single first transformation matrix, smaller groups of residual elements in the set of residual elements 208 could be pre-multiplied by respective smaller first transformation matrices. Using smaller matrices may result in more efficient processing than using a single, larger matrix since fewer calculations may be performed.

As such, the at least one transformation operation may comprise at least one mathematical operation. The at least one transformation operation may comprise at least one matrix multiplication operation. The at least one transformation operation may comprise at least one linear combination operation. For example, the at least one linear combination operation may be used instead of the at least one matrix multiplication operation. As such, the data to be encoded may be arranged in a form that can lead to encoding in a more efficient and well-defined manner In some examples, some or all of the correlation elements au in the set of correlation elements 210 are derived based on residual elements from only one of the pluralities of residual elements in the set of residual elements 208. Such correlation elements aij may be indicative of an extent of correlation between the residual elements on which they are based.

The set of correlation elements 210 may exploit correlation between residual elements in the set of residual elements 208. Less data may be used to transmit the set of correlation elements 210 than the set of residual elements 208, particularly where there is strong correlation in the set of residual elements 208. The set of correlation elements 210 may exploit directional correlation. The set of correlation elements 210 may exploit average, horizontal, vertical and/or diagonal correlation. In some examples, the set of correlation elements 210 includes at least one value derived based on an average of at least some of the values in the set of residual elements 208 and an average of at least some values in the downsampled data 204. Such a derived value may correspond to a predicted average value or a delta average value, AA, which is described in international patent application no. PCT/EP2013/059847.

In this example, the set of correlation elements 210 is transformed into a transformed set of correlation elements 212. In this example, the transformed set of correlation elements 212 is derived by performing at least one transformation operation involving some or all of the correlation elements in the set of correlation elements 210. The transformed set of correlation elements 212 may be derived by pre-multiplying the set of correlation elements 210 with a second transformation matrix (or 'kernel'), K2.

This may be written as 4? = K2 * a, where 13 represents the transformed set of correlation elements 212 and a represents the set of correlation elements 210.

Instead of deriving the transformed set of correlation elements 212 by pre-multiplying the entire set of correlation elements 210 with a single second transformation matrix, smaller groups of correlation elements in the set of correlation elements 210 could be pre-multiplied by respective smaller second transformation matrices.

In this example, the transformed set of correlation elements 212 is arranged as a 4x4 array of correlation elements comprising four rows and four columns of signal elements. In this example, the transformed set of correlation elements 212 comprises four groups of correlation elements. Each of the groups of correlation elements comprises a plurality of correlation elements. In this example, each group of correlation elements is a 2x2 array of correlation elements. A correlation element Oij corresponds to the j'h correlation element in the ill' group of correlation elements in the set of correlation elements 212.

The transformed set of correlation elements 212 may facilitate more efficient encoding compared to the set of correlation elements 210 by providing a larger number of different types of correlation element whose associated encoding parameters may be set individually.

As such, the least one transformation operation may be performed by performing a transformation (or 'transform') involving the first plurality of residual elements, for example R11, R12, R13, R14, to generate a first set of correlation elements, for example consisting of an, an, an, am. Performing the least one transformation operation may also involve performing a transformation involving the second plurality of residual elements, for example R21, R22, R21, R24 to generate a second set of correlation elements, for example consisting of an, an, an, am. At least one correlation element in the first set of correlation elements, for example an, an, an, am, is dependent on an extent of correlation between at least some of the first plurality of residual elements, R11, R12, R13, R14. Each of the correlation elements in the first set of correlation elements, for example an, an, an, am, may be dependent on an extent of correlation between each of the first plurality of residual elements, R11, R12, R13, R14. At least one correlation element in the second set of correlation elements, for example an, an, an, am, is dependent on an extent of correlation between at least some of the second plurality of residual elements, for example R21, R22, Rni, R24. Each of the correlation elements in the second set of correlation elements, for example an, an, an, an, may be dependent on an extent of correlation between each of the second plurality of residual elements, for example R21, R22, R23, R24. As such, in a first stage, residual elements are transformed into one or more intermediate sets of correlation elements, for example the set of correlation elements 210. Related correlation elements may be grouped together. For example, correlation elements may be grouped based on the residual elements with which they are associated or based on correlation element type.

The at least one transformation operation may be performed by performing a transformation involving at least one correlation element in a first subset of correlation elements in the set of correlation elements 210, for example all,an, an, am, and at least one correlation element in a second subset of correlation elements in the set of correlation elements 210, for example an, an, an, a24, to generate at least one correlation element in a third set of correlation elements, for example 13i I in the transformed set of correlation elements 212. For example, correlation elements an and an may be involved in a transformation operation to generate correlation element pu.

One of more further correlation elements, such as a3i and cut!, may also be involved in a transformation operation to generate correlation element Pn. The at least one correlation element, for example pu, in the third set of correlation elements, is dependent on an extent of correlation between the at least one correlation element, for example an, in the first set of correlation elements and the at least one correlation element, for example an, in the second set of correlation elements. As such, in a second stage, correlation elements in one or more intermediate sets of correlation elements, for example the set of correlation elements 210, are involved in a transformation operation into one or more final sets of correlation elements, for example the transformed set of correlation elements 212. The correlation elements in the one or more final sets of correlation elements may exploit two levels of correlation, namely one between residual elements in the set of residual elements 208 and another between correlation elements in the set of correlation elements 210. This may lead to more efficient encoding where there is strong correlation at both levels and also allowing a fine-grained configuration of encoder settings.

The at least one transformation operation may be performed by performing a different transformation involving the at least one correlation element, for example an, in the first set of correlation elements and the at least one correlation element, for example an, in the second set of correlation elements to generate at least one other correlation element in the third set of correlation elements, for example 1321. For example, correlation elements an and an may be involved in a transformation operation using another type of transformation to generate correlation element 1321. One or more further correlation elements, such as all and au, may also be used to generate correlation element 1321. The at least one other correlation element, for example 1321, in the third set of correlation elements is dependent on an extent of correlation between the at least one correlation element, for example al i, in the first set of correlation elements and the at least one correlation element, for example az', in the second set of correlation elements. Multiple different types of correlation may be exploited by performing different transformations involving the intermediate correlation elements aii. This may facilitate efficient encoding where multiple different types of correlation show strong con-elation characteristics.

In this example, deriving the transformed set of con-elation elements 212 may comprise performing one or more transformation operations involving at least one residual element, for example Au, in the first plurality of residual elements, RH, R12, R13, R14, and at least one residual element, for example R21, in the second plurality of residual elements, R21, R22, R23, R24. A residual element, for example Rii, in the first plurality of residual elements, Rii, R12, R13, R14, may be used to generate a first correlation element, for example all, in the set of correlation elements 210 and a residual element, for example R21, in the second plurality of residual elements may be used to generate a second con-elation element, for example an, in the set of correlation elements 210. The first and second con-elation elements, for example an and an, in the set of correlation elements 210 may be used to generate at least one correlation element, for example f311, in the transformed set of correlation elements 212.

In contrast to the con-elation elements in the transformed set of correlation elements 212, in this example the correlation elements in the set of correlation elements 210 are not generated by performing one or more transformation operations involving at least one residual in the first plurality of residual elements, Ril, R12, R13, R14, and at least one residual element in the second plurality of residual elements, R21, R22, R23, R24. Instead, in this example, each of the correlation elements in the set of correlation elements 210 is generated by performing one or more transformation operations involving at least one residual from a single plurality of residual elements only, for example involving only residual elements in the first plurality of residual elements, Rit, R12, R13, R14.

In this example, the at least one transformation operation generates at least one con-elation element, for example con-elation element [3ii in the transformed set of con-elation elements 212. The at least one correlation element is dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements. The value of the at least one correlation element may be inversely proportional to the extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements. In other words, the stronger the correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements the closer the value of the at least one correlation element is to zero. The extent of correlation may be related to the amount or strength of correlation.

In some examples, each of the correlation elements in the transformed set of correlation elements 212 exploits correlation between each of the residual elements in the set of residual elements 208. As such the at least one transformation operation may involve all of the residual elements in the first plurality of residual elements, Rit, R12, Rli, R14, and all of the residual elements in the second plurality of residual elements, R21, R22, R23, RN. Correlation between a relatively large number of residual elements may therefore be exploited. This may lead to efficient encoding where there is a strong correlation between a relatively large number of correlation elements.

A plurality of correlation elements may be generated in performing the at least one transformation operation. By generating multiple correlation elements, different types of correlation may be exploited. As such, more efficient encoding can be obtained where correlation is strong in different directions. Furthermore, different encoder settings may be used for different correlation elements. This facilitates more selective encoding, for example to take account of the importance of the individual correlation elements.

In some examples, the transformed set of correlation elements 212 is used to obtain one or more further transformed sets of correlation elements (not shown). The one or more further transformed sets of correlation elements may be obtained by performing one or more transform operations on the transformed set of correlation elements 212, for example in series. The one or more transform operations may involve one or more directional decomposition operations, as described above. As such, the one or more further transformed sets of correlation elements may be indicative of an extent of correlation, for example a directional or spatial correlation, between different correlation elements in the transformed set of correlation elements 212, or in sets of data elements derived therefrom. This can lead to more efficient encoding, especially when there is a high degree of such correlation.

The first apparatus 102 performs the at least one transformation operation prior to the at least one correlation element being encoded. The first apparatus 102 may encode the at least one correlation element. The at least one correlation element may be encoded by one or more entities other than the first apparatus 102.

The at least one correlation element being encoded may comprise quantisation being performed on the at least one correlation element. Quantisation facilitates reducing the amount of data used to represent the at least one correlation element. The first apparatus 102 may quantise the at least one correlation element. The at least one correlation element may be quantised by one or more entities other than the first apparatus 102.

Individual encoding parameters may be used for encoding individual correlation elements in the transformed set of correlation elements 212. For example, correlation elements that are relatively unimportant (relative to other correlation elements in the transformed set of correlation elements 212) may be subject to a higher level of encoding, for example quantisation, than correlation elements that are relatively important (relative to other correlation elements in the transformed set of correlation elements 212).

In this example, the encoder device 108 obtains a set of encoding parameters. Each of the correlation elements in the plurality of correlation elements may be associated with one or more encoding parameters. This allows for fine-grained specification of encoding parameters, for example to achieve an overall bit rate allowance for the transformed set of correlation elements 212. The individual encoding parameters may also be indicative of the importance of the associated correlation element. This may facilitate efficient encoding by prioritising the most important correlation elements. A correlation element may be associated with multiple encoding parameters. An encoding parameter may be associated with multiple correlation elements.

In some examples, an encoding parameter or other data indicates that an associated correlation element is not to be encoded. For example, the associated correlation element may be discarded and not encoded at all. In some examples, there is a predefined importance threshold level below which a correlation element is not encoded. Instead, a default value for that correlation element may be used. Data may therefore be generated to indicate that at least some of the plurality of correlation elements are not to be encoded. Encoding may be made more efficient by not encoding some of the plurality of correlation elements. Encoding may be more efficient as less data is encoded.

The encoding parameter for each of the correlation elements in the plurality of correlation elements may be determined. Determining the encoding parameters may involve looking up some or all of the encoding parameters, calculating the some or all of the encoding parameters or obtaining some or all of the encoding parameters in another way.

In some examples, the value of an encoding parameter is determined based at least in part on the importance of the correlation element with which it is associated, relative to the importance other correlation elements in the transformed set of correlation elements 212. The value of the encoding parameter may be indicative of the importance of the associated correlation element in terms of perceived quality.

In some examples, at least some of the correlation elements in the transformed set of correlation elements 212 are associated with the same encoding parameter and/or encoding parameter value as each other. This may indicate that those correlation elements are all equally important in terms of perceived quality. Encoding may be enhanced by designating the same level of encoding for different ones of the correlation elements, for example where any differences in importance of associated correlation elements to not exceed a threshold level, so the encoder can use the same encoder settings for multiple ones of the correlation elements. The threshold level may be zero or non-zero.

In some examples, at least some of the correlation elements in the transformed set of correlation elements 212 are associated with different encoding parameters and/or encoding parameter values. This may facilitate encoding by determining the relative importance of those correlation elements in terms of perceived quality. For example, different amounts of quanti sati on may be used dependent upon the relative importance of those correlation elements in terms of perceived quality.

In some examples, all of the correlation elements in the transformed set of correlation elements are associated with different encoding parameters and/or encoding parameter values.

Various different factors may be taken into account in determining the values of the encoding parameters in the set of encoding parameters. Examples of such factors include, but are not limited to, frequency analysis, angular frequency analysis, viewing distance analysis and pixel size analysis. Taking such factors into account may facilitate ranking of the correlation elements in terms of perceptual importance. This, in turn, may be used to determine the extent to which, for example, quantisation should be applied to the correlation elements.

At least some of the encoding parameters may be representative of respective measures of perceptual importance amongst the correlation elements. This may enhance encoding by allowing encoding to be weighted based on a measure of importance of the correlation elements.

At least some of the encoding parameters may be the same for different ones of the correlation elements in the plurality of correlation elements.

The plurality of encoding parameters may comprise at least one quantisation coefficient. As such, different levels of quantisation may be applied to different correlation elements, for example based on their importance. This may provide encoding techniques in which reducing an amount of data to be transmitted is balanced against the perception of quality associated with quantised data.

In some examples, the first apparatus 102 outputs the set of encoding parameters to at least one further apparatus, for example the second apparatus 104. The second apparatus 104 may use the set of encoding parameters to facilitate decoding of the encoded correlation elements. As such, data comprising the encoding parameters may be output. This also facilitates encoding where encoding is performed by another entity.

The plurality of correlation elements may be encoded based on the set of encoding parameters. As such, a reliable mechanism is provided for encoding the plurality of correlation elements, for example to reflect their relative importance.

In this example, the first apparatus 102 performs the at least one transformation operation, to generate the at least one correlation element in the transformed set of correlation elements, prior to the at least one correlation element being encoded.

In this example, the first apparatus 102 transmits the downsampled data 204 and the transformed set of correlation elements 212 to the second apparatus 104, possibly in an encoded form.

Referring to Figure 3, there is shown schematically an example of a signal processing technique 300. The signal processing technique 300 may be performed by the second apparatus 104. Some items depicted in Figure 3 are similar to items shown in Figure 2. Corresponding reference signs, incremented by 100, have therefore been used for similar items.

In this example, the second apparatus 104 receives downsampled data 304 and the transformed set of correlation elements 312 from the first apparatus 102. In other examples, the second apparatus 104 receives data usable to obtain the downsampled data 304 and/or the transformed set of correlation elements 312, rather than the downsampled data 304 and/or the transformed set of correlation elements 312 itself For example, the second apparatus 104 may derive the downsampled data 304 and/or the transformed set of correlation elements 312 based on the received data. Pre-multiplying both sides of the above equation fl = K2 * a by the inverse of K2, namely K24. gives K2-1 * [3 = K2-1 * K2 * a. Since pre-multiplying a matrix by its inverse gives the identity matrix, K2-1 * 13 = a. As such, the second apparatus 104 can derive the set of correlation elements 310 by pre-multiplying the transformed set of correlation elements 312 with the inverse of the second transformation matrix, K21 Pre-multiplying both sides of the above equation a = Kl* R by the inverse of Ki, namely Ki1. gives K1-1 * a = kyl * Ki * R. Since pre-multiplying a matrix by its inverse gives the identity matrix, K1-1 * a = R. As such, the second apparatus 104 can derive the set of residual elements 308 by pre-multiplying the set of correlation elements 310 with the inverse of the first transformation matrix, Krl.

As such, the second apparatus 104 recovers the set of residual elements 308. The downsampled data 304 is processed to generate data 306. In this example, the data 306 is obtained by upsampling the downsampled data 304 in the same manner as the first apparatus 102 upsampled the data 204. In this specific example, the data 306 is referred to as "upsampled data", it being understood that the data could be processed in a different way in other examples.

The second apparatus 104 can therefore recover the input data 302 using the upsampled data 306 and the set of residual elements 308 since all of the values 1/;/* and Rki are known to the second apparatus 104.

The second apparatus 104 performs at least one transformation operation involving at least one correlation element, for example pi I in the transformed set of correlation elements 312. The at least one transformation operation may involve at least one transformation that is the inverse of a transformation performed by the first apparatus 102. The at least one transformation operation generates at least one residual element, for example Rn, in a first plurality of residual elements, for example R11, R12, R13, R14, and at least one residual element, for example Rn, in a second plurality of residual elements, for example R21, R22, R23, R24. The at least one transformation operation may generate one or more further residual elements. The at least one transformation operation may involve one or more additional correlation elements.

The at least one correlation element, for example f3u, is dependent on an extent of correlation between the at least one residual element, for example Rtt, in the first plurality of residual elements and the at least one residual element, for example R21, in the second plurality of residual elements. The first plurality of residual elements, for example Rii, R12, R13, R14, is useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality, for example the part of the input data 302 consisting of signal elements Ill, 112, 113, 114, using a first part of a second representation of the signal at the relatively high level of quality, for example the part of the upsampled data 306 consisting of signal elements Ui U12, U13, U14. The second plurality of residual elements, for example R21, R22, R23, R24, is useable to reconstruct a second, different part of the first representation of the signal, for example the part of the input data 302 consisting of signal elements 121, 122, 123,124, using a second, different part of the second representation of the signal, for example the part of the upsampled data 306 consisting of signal elements U21, U22, U23, U24.

The second apparatus 104 derives the first part of the second representation, for example the part of the upsampled data 306 consisting of signal elements Ut 1, U12, Ul 3, U14, from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality, for example signal element Di in the downsampled data 304.

The second apparatus 104 derives the second, different part of the second representation, for example the part of the upsampled data 306 consisting of signal elements U21, U22, U23, U24, from a second, different set of one or more signal elements in a representation of the signal at a relatively low level of quality, for example signal element D2 in the downsampled data 304.

The second apparatus 104 uses at least the first plurality of residual elements, for example RH, R12, R113, Am, the second plurality of residual elements, for example R2 I, R22, R23, R24, the first part of the second representation, for example Ui, U12, U13, U14, and the second part of the second representation, for example U21, U22, U23, U24, to generate output data. The output data may comprise input data 302.

The second apparatus 104 is configured to perform the at least one transformation operation following the at least one correlation element, for example I311, having been decoded. The at least one correlation element may be decoded by the second apparatus 104 or by one or more other entities.

The second apparatus 104 may have at least some other features described herein, for example features described in relation to the first apparatus 102. The second apparatus 104 may be configured to perform at least some other techniques described herein, for example techniques described herein in relation to the first apparatus 102.

Referring to Figure 4, there is shown a schematic block diagram of an example of an apparatus 400.

In an example, the apparatus 400 comprises a decoder device. In another example, the apparatus 400 comprises an encoder device.

Other examples of apparatus 400 include, but are not limited to, a mobile computer, a personal computer system, a wireless device, base station, phone device, desktop computer, laptop, notebook, netbook computer, mainframe computer system, handheld computer, workstation, network computer, application server, storage device, a consumer electronics device such as a camera, camcorder, mobile device, video game console, handheld video game device, a peripheral device such as a switch, modem, router, etc., or in general any type of computing or electronic device In this example, the apparatus 400 comprises one or more processors 401 configured to process information and/or instructions. The one or more processors 401 may comprise a central processing unit (CPU). The one or more processors 401 are coupled with a bus 402. Operations performed by the one or more processors 401 may be carried out by hardware and/or software. The one or more processors 401 may comprise multiple co-located processors or multiple disparately located processors.

In this example, the apparatus 400 comprises computer-useable volatile memory 403 configured to store information and/or instructions for the one or more processors 401. The computer-useable volatile memory 403 is coupled with the bus 402. The computer-useable volatile memory 403 may comprise random access memory (RAM).

In this example, the apparatus 400 comprises computer-useable non-volatile memory 404 configured to store information and/or instructions for the one or more processors 401. The computer-useable non-volatile memory 404 is coupled with the bus 402. The computer-useable non-volatile memory 404 may comprise read-only memory (ROM).

In this example, the apparatus 400 comprises one or more data-storage units 405 configured to store information and/or instructions. The one or more data-storage units 405 are coupled with the bus 402. The one or more data-storage units 405 may for example comprise a magnetic or optical disk and disk drive or a solid-state drive (SSD).

In this example, the apparatus 400 comprises one or more input/output (I/0) devices 406 configured to communicate information to and/or from the one or more processors 401. The one or more I/O devices 406 are coupled with the bus 402. The one or more I/O devices 406 may comprise at least one network interface. The at least one network interface may enable the apparatus 400 to communicate via one or more data communications networks. Examples of data communications networks include, but are not limited to, the Internet and a Local Area Network (LAN). The one or more I/O devices 406 may enable a user to provide input to the apparatus 400 via one or more input devices (not shown). The one or more input devices may include for example a remote control, one or more physical buttons etc. The one or more I/O devices 406 may enable information to be provided to a user via one or more output devices (not shown).

The one or more output devices may for example include a display screen.

Various other entities are depicted for the apparatus 400. For example, when present, an operating system 407, signal processing module 408, one or more further modules 409, and data 410 are shown as residing in one, or a combination, of the computer-usable volatile memory 403, computer-usable non-volatile memory 404 and the one or more data-storage units 405. The signal processing module 408 may be implemented by way of computer program code stored in memory locations within the computer-usable non-volatile memory 404, computer-readable storage media within the one or more data-storage units 405 and/or other tangible computer-readable storage media. Examples of tangible computer-readable storage media include, but are not limited to, an optical medium (e.g., CD-ROM, DVD-ROM or Blu-ray), flash memory card, floppy or hard disk or any other medium capable of storing computer-readable instructions such as firmware or microcode in at least one ROM or RAM or Programmable RONI (PROM) chips or as an Application Specific Integrated Circuit (ASIC).

The apparatus 400 may therefore comprise a signal processing module 408 which can be executed by the one or more processors 401. The signal processing module 408 can be configured to include instructions to implement at least some of the operations described herein. During operation, the one or more processors 401 launch, run, execute, interpret or otherwise perform the instructions in the signal processing module 408.

Although at least some aspects of the examples described herein with reference to the drawings comprise computer processes performed in processing systems or processors, examples described herein also extend to computer programs, for example computer programs on or in a carrier, adapted for putting the examples into practice. The carrier may be any entity or device capable of carrying the program.

It will be appreciated that the apparatus 400 may comprise more, fewer and/or different components from those depicted in Figure 4.

The apparatus 400 may be located in a single location or may be distributed in multiple locations. Such locations may be local or remote.

The techniques described herein may be implemented in software or hardware, or may be implemented using a combination of software and hardware. They may include configuring an apparatus to carry out and/or support any or all of techniques described herein.

Various measures (for example apparatuses, methods, computer programs and computer-readable media) are provided in which a first plurality of residual elements is obtained. The first plurality of residual elements is useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality. The first part of the second representation is derivable from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality. A second plurality of residual elements is obtained. The second plurality of residuals is useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal The second part of the second representation is derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality. At least one transformation operation involving at least one residual element in the first plurality of residual elements and at least one residual element in the second plurality of residual elements is performed to generate at least one correlation element. The at least one correlation element is dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements. The apparatus is configured to perform the at least one transformation operation prior to the at least one correlation element being encoded.

In examples described above, the at least one transformation operation comprises at least one directional decomposition operation.

In examples described above, the at least one transformation operation comprises at least one mathematical operation.

In examples described above, the at least one transformation operation comprises at least one linear combination operation.

In examples described above, a plurality of correlation elements is generated in performing the at least one transformation operation.

In examples described above, data is generated to indicate that at least some of the plurality of correlation elements are not to be encoded.

In examples described above, each of the correlation elements in the plurality of correlation elements is associated with an encoding parameter.

In examples described above, the encoding parameter for each of the correlation elements in the plurality of correlation elements is determined.

In examples described above, the encoding parameter for each of the correlation elements in the plurality of correlation elements is determined based on a position of the correlation elements in the plurality of correlation elements In examples described above, a value of an encoding parameter in the plurality of correlation elements is independent of a value of a correlation element associated with the encoding parameter.

In examples described above, at least some of the encoding parameters are different for different ones of the correlation elements in the plurality of correlation elements.

In examples described above, at least some of the encoding parameters are the same for different ones of the correlation elements in the plurality of correlation elements.

In examples described above, at least some of the encoding parameters are representative of respective measures of perceptual importance amongst the correlation elements In examples described above, the plurality of encoding parameters comprises at least one quanti sati on coefficient.

In examples described above, the plurality of correlation elements is encoded based on the encoding parameters.

In examples described above, data comprising the encoding parameters is output In examples described above, the at least one correlation element being encoded comprises quantisation being performed on the at least one correlation element.

In examples described above, at least one transformation operation is performed by performing a transformation involving the first plurality of residual elements to generate a first set of correlation elements and performing a transformation involving the second plurality of residual elements to generate a second set of correlation elements. At least one correlation element in the first set of correlation elements is dependent on an extent of correlation between at least some of the first plurality of residual elements. At least one correlation element in the second set of correlation elements is dependent on an extent of correlation between at least some of the second plurality of residual elements.

In examples described above, different correlation elements in the first and second sets of correlation elements represent different types of correlation. Correlation elements of the same type are grouped together within the first and second sets of correlation elements.

In examples described above, the at least one transformation operation is performed by performing a transformation involving at least one correlation element in the first set of correlation elements and at least one correlation element in the second set of correlation elements to generate at least one correlation element in a third set of correlation elements. The at least one correlation element in the third set of correlation elements is dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements.

In examples described above, the at least one transformation operation is performed by performing a different transformation involving the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements to generate at least one other correlation element in the third set of correlation elements. The at least one other correlation element in the third set of correlation elements is dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements.

In examples described above, at least one transformation operation involves all of the residual elements in the first plurality of residual elements and all of the residual elements in the second plurality of residual elements.

Various measures (for example apparatuses, methods, computer programs and computer-readable media) are provided in which at least one transformation operation involving at least one correlation element is performed to generate at least one residual element in a first plurality of residual elements and at least one residual element in a second plurality of residual elements. The at least one correlation element is dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements. The first plurality of residual elements is useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality. The second plurality of residual elements is useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal. The first part of the second representation is derived from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality. The second part of the second representation is derived from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality. At least the first plurality of residual elements, the second plurality of residual elements, the first part of the second representation and the second part of the second representation are used to generate output data. The at least one transformation operation is performed following the at least one correlation element being decoded.

Various measures (for example apparatuses, methods, computer programs and computer-readable media) are provided to process signal data. A set of residual elements is obtained. A first plurality of residual elements in the set of residual elements is useable to generate a first set of correlation elements. A second plurality of residual elements in the set of residual elements is useable to generate a second set of correlation elements. At least one correlation element in the first set of correlation elements is indicative of an extent of correlation, for example spatial or directional correlation, between at least some of the first plurality of residual elements. At least one correlation element in the second set of correlation elements is indicative of an extent of correlation, for example spatial or directional correlation, between at least some of the second plurality of residual elements. The first and second sets of correlation elements are transformable into a third set of correlation elements. At least one correlation element in the third set of correlation elements is indicative of an extent of correlation, for example spatial or directional correlation, between one or more of the at least one correlation elements in the first set of correlation elements and one or more of the at least one correlation elements in the second set of correlation elements.

A first correlation element in the third set of correlation elements is generated by performing a first transformation operation involving at least one of the first plurality of residual elements and at least one of the second plurality of residual elements. A second correlation element in the third set of correlation elements is generated by performing a second transformation operation on the at least one of the first plurality of residual elements and on the at least one of the second plurality of residual elements. The above embodiments are to be understood as illustrative examples. Further embodiments are envisaged.

In examples described above, the second apparatus 104 is able to recover or reconstruct fully the input data obtained by the first apparatus 102. In other examples, the second apparatus 104 may not be able to recover fully the input data obtained by the first apparatus 102. For example, some information that may be needed to recover the original input data may be lost during quantisation and may not be recoverable by the second apparatus 104.

In examples described above, the transformed set of correlation elements 212 is obtained indirectly from the set of residual elements 208 by two separate matrix multiplications involving the first and second transformation matrices K1 and K2. In other examples, the transformed set of correlation elements 212 is obtained directly from the set of residual elements 208 by combining the two matrix multiplications involving the first and second transformation matrices Ki and K2 described above. In such other examples, rather than carrying out two separate matrix multiplications, one involving Ki and the other involving K2, a single matrix multiplication operation merging both matrix transformations is used. Pre-multiplying both sides of the above equation a.= K1* R by 1(2 gives K2 * a = 112 * KJ. * R. Using the above equation, #1 = K2 * a, it can be seen that # = K2 * a = K2 * K1 * R. A merged matrix, 1(3, may be obtained by K3 = K2 * KJ. and the transform performed as # = K3 * R. Merging two matrix multiplications may be beneficial in that memory is not used to store an intermediate matrix. Processing time may also be reduced in performing one matrix multiplication compared to performing two matrix multiplications.

It is to be understood that any feature described in relation to any one embodiment may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments, or any combination of any other of the embodiments. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.

Clauses The invention may be further understood with reference to the following clauses 1. Apparatus configured to: obtain a first plurality of residual elements useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the first part of the second representation being derivable from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; obtain a second plurality of residual elements useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal, the second part of the second representation being derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and perform at least one transformation operation involving at least one residual element in the first plurality of residual elements and at least one residual element in the second plurality of residual elements to generate at least one correlation element, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, wherein the apparatus is configured to perform the at least one transformation operation prior to the at least one correlation element being encoded.

2. Apparatus according to clause 1, wherein the at least one transformation operation comprises at least one directional decomposition operation.

3. Apparatus according to clause 1 or 2, wherein the at least one transformation operation comprises at least one mathematical operation.

4. Apparatus according to any of clauses 1 to 3, wherein the at least one transformation operation comprises at least one matrix multiplication operation.

5. Apparatus according to any of clauses 1 to 4, wherein the at least one transformation operation comprises at least one linear combination operation 6. Apparatus according to any of clauses 1 to 5, the apparatus being configured to generate a plurality of correlation elements in performing the at least one transformation operation.

7. Apparatus according to clause 6, the apparatus being configured to generate data to indicate that at least some of the plurality of correlation elements are not to be encoded 8. Apparatus according to clause 6 or 7, wherein each of the correlation elements in the plurality of correlation elements is associated with an encoding parameter.

9 Apparatus according to clause 8, the apparatus being configured to determine the encoding parameter for each of the correlation elements in the plurality of correlation elements 10. Apparatus according to clause 8 or 9, the apparatus being configured to determine the encoding parameter for each of the correlation elements in the plurality of correlation elements based on a position of the correlation elements in the plurality of correlation elements.

11. Apparatus according to any of clauses 8 to 10, wherein a value of an encoding parameter in the plurality of correlation elements is independent of a value of a correlation element associated with the encoding parameter.

12. Apparatus according to any of clauses 8 to 11, wherein at least some of the encoding parameters are different for different ones of the correlation elements in the plurality of correlation elements.

13. Apparatus according to any of clauses 8 to 12, wherein at least some of the encoding parameters are the same for different ones of the correlation elements in the plurality of correlation elements.

14. Apparatus according to any of clauses 8 to 13, wherein at least some of the encoding parameters are representative of respective measures of perceptual importance amongst the correlation elements.

15. Apparatus according to any of clauses 8 to 14, wherein the plurality of encoding parameters comprises at least one quantisation coefficient.

16. Apparatus according to any of clauses 8 to 15, the apparatus being configured to encode the plurality of correlation elements based on the encoding parameters.

17. Apparatus according to any of clauses 8 to 16, the apparatus being configured to output data comprising the encoding parameters.

18 Apparatus according to any of clauses 1 to 17, wherein the at least one correlation element being encoded comprises quantisation being performed on the at least one correlation element.

19. Apparatus according to any of clauses 1 to 18, the apparatus being configured to perform the at least one transformation operation by: performing a transformation involving the first plurality of residual elements to generate a first set of correlation elements, at least one correlation element in the first set of correlation elements being dependent on an extent of correlation between at least some of the first plurality of residual elements; and performing a transformation involving the second plurality of residual elements to generate a second set of correlation elements, at least one correlation element in the second set of correlation elements being dependent on an extent of correlation between at least some of the second plurality of residual elements.

20. Apparatus according to clause 19, wherein different correlation elements in the first and second sets of correlation elements represent different types of correlation, the apparatus being configured to group correlation elements of the same type together in the first and second sets of correlation elements 21 Apparatus according to clause 19 or 20, the apparatus being configured to perform the at least one transformation operation by performing a transformation involving at least one correlation element in the first set of correlation elements and at least one correlation element in the second set of correlation elements to generate at least one correlation element in a third set of correlation elements, the at least one correlation element in the third set of correlation elements being dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements 22 Apparatus according to clause 21, the apparatus being configured to perform the at least one transformation operation by performing a different transformation involving the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements to generate at least one other correlation element in the third set of correlation elements, the at least one other correlation element in the third set of correlation elements being dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements.

23. Apparatus according to any of clauses 1 to 22, wherein the at least one transformation operation involves all of the residual elements in the first plurality of residual elements and all of the residual elements in the second plurality of residual elements 24. Apparatus configured to: perform at least one transformation operation involving at least one correlation element to generate at least one residual element in a first plurality of residual elements and at least one residual element in a second plurality of residual elements, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, the first plurality of residual elements being useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the second plurality of residual elements being useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal, derive the first part of the second representation from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; derive the second part of the second representation from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and use at least the first plurality of residual elements, the second plurality of residual elements, the first part of the second representation and the second part of the second representation to generate output data, wherein the apparatus is configured to perform the at least one transformation operation following the at least one correlation element being decoded.

25. Apparatus according to clause 24, wherein the at least one transformation operation comprises at least one inverse directional composition operation.

26. Apparatus according to clause 24 or 25, wherein the at least one transformation operation comprises at least one mathematical operation.

27 Apparatus according to any of clauses 24 to 26, wherein the at least one transformation operation comprises at least one matrix multiplication operation.

28. Apparatus according to any of clauses 24 to 27, wherein the at least one transformation operation comprises at least one linear combination operation 29. A method comprising: obtaining a first plurality of residual elements useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the first part of the second representation being derivable from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; obtaining a second plurality of residual elements useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal, the second part of the second representation being derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and performing at least one transformation operation involving at least one residual element in the first plurality of residual elements and at least one residual element in the second plurality of residual elements to generate at least one correlation element, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, wherein the at least one transformation operation is performed prior to the at least one correlation element being encoded.

30. A method according to clause 29, wherein the at least one transformation operation comprises at least one directional decomposition operation.

31. A method according to clause 29 or 30, wherein the at least one transformation operation comprises at least one mathematical operation 32. A method according to any of clauses 29 to 31, wherein the at least one transformation operation comprises at least one matrix multiplication operation.

33. A method according to any of clauses 29 to 32, wherein the at least one transformation operation comprises at least one linear combination operation 34. A method according to any of clauses 29 to 33, wherein performing the at least one transformation operation comprises generating a plurality of correlation elements.

35. A method according to clause 34, the apparatus being configured to generate data to indicate that at least some of the plurality of correlation elements are not to be encoded 36. A method according to clause 34 or 35, wherein each of the correlation elements in the plurality of correlation elements is associated with an encoding parameter.

37 A method according to clause 36, the method comprising determining the encoding parameter for each of the correlation elements in the plurality of correlation elements.

38 A method according to clause 36 or 37, comprising determining the encoding parameter for each of the correlation elements in the plurality of correlation elements based on a position of the correlation elements in the plurality of correlation elements.

39. A method according to any of clauses 36 to 38, wherein a value of an encoding parameter in the plurality of correlation elements is independent of a value of a correlation element associated with the encoding parameter.

40. A method according to any of clauses 36 to 39, wherein at least some of the encoding parameters are different for different ones of the correlation elements in the plurality of correlation elements 41. A method according to any of clauses 36 to 40, wherein at least some of the encoding parameters are the same for different ones of the correlation elements in the plurality of correlation elements 42. A method according to any of clauses 36 to 41, wherein at least some of the encoding parameters are representative of respective measures of perceptual importance amongst the correlation elements 43. A method according to any of clauses 36 to 42, wherein the plurality of encoding parameters comprises at least one quantisation coefficient.

44. A method according to any of clauses 36 to 43, comprising encoding the plurality of correlation elements based on the encoding parameters.

45. A method according to any of clauses 36 to 44, comprising outputting data comprising the encoding parameters.

46 A method according to any of clauses 29 to 45, wherein the at least one correlation element being encoded comprises quantisation being performed on the at least one correlation element.

47. A method according to any of clauses 29 to 46, wherein performing the at least one transformation operation comprises: performing a transformation involving the first plurality of residual elements to generate a first set of correlation elements, at least one correlation element in the first set of correlation elements being dependent on an extent of correlation between at least some of the first plurality of residual elements; and performing a transformation involving the second plurality of residual elements to generate a second set of correlation elements, at least one correlation element in the second set of correlation elements being dependent on an extent of correlation between at least some of the second plurality of residual elements 48 A method according to clause 47, wherein different correlation elements in the first and second sets of correlation elements represent different types of correlation, the method comprising grouping correlation elements of the same type together in the first and second sets of correlation elements.

49 A method according to clause 47 or 48, wherein performing the at least one transformation operation comprises performing a transformation involving at least one correlation element in the first set of correlation elements and at least one correlation element in the second set of correlation elements to generate at least one correlation element in a third set of correlation elements, the at least one correlation element in the third set of correlation elements being dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements.

50. A method according to clause 49, wherein performing the at least one transformation operation comprises performing a different transformation involving the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements to generate at least one other correlation element in the third set of correlation elements, the at least one other correlation element in the third set of correlation elements being dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements 51. A method according to any of clauses 29 to 50, wherein the at least one transformation operation involves all of the residual elements in the first plurality of residual elements and all of the residual elements in the second plurality of residual elements.

52. A method comprising: performing at least one transformation operation involving at least one correlation element to generate at least one residual element in a first plurality of residual elements and at least one residual element in a second plurality of residual elements, the at least one correlation element being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, the first plurality of residual elements being useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the second plurality of residual elements being useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal; deriving the first part of the second representation from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; deriving the second part of the second representation from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and using at least the first plurality of residual elements, the second plurality of residual elements, the first part of the second representation and the second part of the second representation to generate output data, wherein the at least one transformation operation is performed following the at least one correlation element being decoded 53. A method according to clause 52, wherein the at least one transformation operation comprises at least one inverse directional composition operation 54. A method according to clause 52 or 53, wherein the at least one transformation operation comprises at least one mathematical operation.

55. A method according to any of clauses 52 to 54, wherein the at least one transformation operation comprises at least one matrix multiplication operation.

56. A method according to any of clauses 52 to 55, wherein the at least one transformation operation comprises at least one linear combination operation.

57. A computer program comprising instructions which, when executed, cause an apparatus to perform a method according to any of clauses 29 to 56.

58. A computer-readable medium comprising a computer program according to clause 57.

Claims

CLAIMSI. Apparatus configured to: obtain a first plurality of residual elements useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the first part of the second representation being derivable from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; obtain a second plurality of residual elements useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal, the second part of the second representation being derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and perform at least one transformation operation involving at least one residual element in the first plurality of residual elements and at least one residual element in the second plurality of residual elements to generate a plurality of correlation elements, the plurality of correlation elements being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, wherein the apparatus is configured to perform the at least one transformation operation prior to the plurality of correlation elements being encoded, and to generate data to indicate that at least some of the plurality of correlation elements are not to be encoded.
2. Apparatus according to claim 1, wherein the at least one transformation operation comprises at least one directional decomposition operation.
3. Apparatus according to claim 1 or 2, wherein the at least one transformation operation comprises at least one mathematical operation
4. Apparatus according to any of claims 1 to 3, wherein the at least one transformation operation comprises at least one matrix multiplication operation.
5. Apparatus according to any of claims 1 to 4, wherein the at least one transformation operation comprises at least one linear combination operation
6. Apparatus according to any of claims 1 to 5, wherein each of the correlation elements in the plurality of correlation elements is associated with an encoding parameter.
7. Apparatus according to claim 6, the apparatus being configured to determine the encoding parameter for each of the correlation elements in the plurality of correlation elements.
8. Apparatus according to claim 6 or 7, the apparatus being configured to determine the encoding parameter for each of the correlation elements in the plurality of correlation elements based on a position of the correlation elements in the plurality of correlation elements 9.
Apparatus according to any of claims 6 to 8, wherein a value of an encoding parameter in the plurality of correlation elements is independent of a value of a correlation element associated with the encoding parameter.
Apparatus according to any of claims 6 to 9, wherein at least some of the encoding parameters are different for different ones of the correlation elements in the plurality of correlation elements.
11. Apparatus according to any of claims 6 to 10, wherein at least some of the encoding parameters are the same for different ones of the correlation elements in the plurality of correlation elements
12. Apparatus according to any of claims 6 to 11, wherein at least some of the encoding parameters are representative of respective measures of perceptual importance amongst the correlation elements.
13. Apparatus according to any of claims 6 to 12, wherein the plurality of encoding parameters comprises at least one quantisati on coefficient
14. Apparatus according to any of claims 6 to 13, the apparatus being configured to encode the plurality of correlation elements based on the encoding parameters
15. Apparatus according to any of claims 6 to 14, the apparatus being configured to output data comprising the encoding parameters.
16. Apparatus according to any of claims Ito 15, wherein the at least one correlation element being encoded comprises quantisation being performed on the at least one correl ati on element.
17. Apparatus according to any of claims 1 to 16, the apparatus being configured to perform the at least one transformation operation by: performing a transformation involving the first plurality of residual elements to generate a first set of correlation elements, at least one correlation element in the first set of correlation elements being dependent on an extent of correlation between at least some of the first plurality of residual elements; and performing a transformation involving the second plurality of residual elements to generate a second set of correlation elements, at least one correlation element in the second set of correlation elements being dependent on an extent of correlation between at least some of the second plurality of residual elements.
18. Apparatus according to claim 17, wherein different correlation elements in the first and second sets of correlation elements represent different types of correlation, the apparatus being configured to group correlation elements of the same type together in the first and second sets of correlation elements
19. Apparatus according to claim 17 or 18, the apparatus being configured to perform the at least one transformation operation by performing a transformation involving at least one correlation element in the first set of correlation elements and at least one correlation element in the second set of correlation elements to generate at least one correlation element in a third set of correlation elements, the at least one correlation element in the third set of correlation elements being dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements.
20. Apparatus according to claim 19, the apparatus being configured to perform the at least one transformation operation by performing a different transformation involving the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements to generate at least one other correlation element in the third set of correlation elements, the at least one other correlation element in the third set of correlation elements being dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements.
21. Apparatus according to any of claims 1 to 20, wherein the at least one transformation operation involves all of the residual elements in the first plurality of residual elements and all of the residual elements in the second plurality of residual elements
22. Apparatus configured to: perform at least one transformation operation involving a plurality of correlation elements to generate at least one residual element in a first plurality of residual elements and at least one residual element in a second plurality of residual elements, the plurality of correlation elements being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, the first plurality of residual elements being useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the second plurality of residual elements being useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal; derive the first part of the second representation from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; derive the second part of the second representation from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and use at least the first plurality of residual elements, the second plurality of residual elements, the first part of the second representation and the second part of the second representation to generate output data, wherein the apparatus is configured to perform the at least one transformation operation following the plurality of correlation elements being decoded, and to generate data to indicate that at least some of the plurality of correlation elements are not to be 15 decoded
23. Apparatus according to claim 22, wherein the at least one transformation operation comprises at least one inverse directional composition operation
24. Apparatus according to claim 22 or 23, wherein the at least one transformation operation comprises at least one mathematical operation.
25. Apparatus according to any of claims 22 to 24, wherein the at least one transformation operation comprises at least one matrix multiplication operation.
26. Apparatus according to any of claims 22 to 25, wherein the at least one transformation operation comprises at least one linear combination operation.
27. A method comprising: obtaining a first plurality of residual elements useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the first part of the second representation being derivable from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; obtaining a second plurality of residual elements useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal, the second part of the second representation being derivable from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and performing at least one transformation operation involving at least one residual element in the first plurality of residual elements and at least one residual element in the second plurality of residual elements to generate a plurality of correlation elements, the plurality of correlation elements being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, wherein the at least one transformation operation is performed prior to the plurality of correlation elements being encoded, wherein the method further comprises generating data to indicate that at least some of the plurality of correlation elements are not to be encoded.
28. A method according to claim 27, wherein the at least one transformation operation comprises at least one directional decomposition operation
29. A method according to claim 27 or 28, wherein the at least one transformation operation comprises at least one mathematical operation.
30 A method according to any of claims 27 to 29, wherein the at least one transformation operation comprises at least one matrix multiplication operation.
31. A method according to any of claims 27 to 30, wherein the at least one transformation operation comprises at least one linear combination operation.
32. A method according to any of claims 27 to 31, wherein each of the correlation elements in the plurality of correlation elements is associated with an encoding parameter.
33. A method according to claim 32, the method comprising determining the encoding parameter for each of the correlation elements in the plurality of correlation elements.
34. A method according to claim 32 or 33, comprising determining the encoding parameter for each of the correlation elements in the plurality of correlation elements based on a position of the correlation elements in the plurality of correlation elements.
35. A method according to any of claims 32 to 34, wherein a value of an encoding parameter in the plurality of correlation elements is independent of a value of a correlation element associated with the encoding parameter.
36. A method according to any of claims 32 to 35, wherein at least some of the encoding parameters are different for different ones of the correlation elements in the plurality of correlation elements.
37 A method according to any of claims 32 to 36, wherein at least some of the encoding parameters are the same for different ones of the correlation elements in the plurality of correlation elements.
38 A method according to any of claims 32 to 37, wherein at least some of the encoding parameters are representative of respective measures of perceptual importance amongst the correlation elements.
39. A method according to any of claims 32 to 38, wherein the plurality of encoding parameters comprises at least one quantisation coefficient.
40. A method according to any of claims 32 to 39, comprising encoding the plurality of correlation elements based on the encoding parameters
41. A method according to any of claims 32 to 40, comprising outputting data comprising the encoding parameters.
42. A method according to any of claims 27 to 41, wherein the at least one correlation element being encoded comprises quantisation being performed on the at least one correlation element.
43. A method according to any of claims 27 to 42, wherein performing the at least one transformation operation comprises: performing a transformation involving the first plurality of residual elements to generate a first set of correlation elements, at least one correlation element in the first set of correlation elements being dependent on an extent of correlation between at least some of the first plurality of residual elements; and performing a transformation involving the second plurality of residual elements to generate a second set of correlation elements, at least one correlation element in the second set of correlation elements being dependent on an extent of correlation between at least some of the second plurality of residual elements.
44 A method according to claim 43, wherein different correlation elements in the first and second sets of correlation elements represent different types of correlation, the method comprising grouping correlation elements of the same type together in the first and second sets of correlation elements
45. A method according to claim 43 or 44, wherein performing the at least one transformation operation comprises performing a transformation involving at least one correlation element in the first set of correlation elements and at least one correlation element in the second set of correlation elements to generate at least one correlation element in a third set of correlation elements, the at least one correlation element in the third set of correlation elements being dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements
46 A method according to claim 45, wherein performing the at least one transformation operation comprises performing a different transformation involving the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements to generate at least one other correlation element in the third set of correlation elements, the at least one other correlation element in the third set of correlation elements being dependent on an extent of correlation between the at least one correlation element in the first set of correlation elements and the at least one correlation element in the second set of correlation elements.
47 A method according to any of claims 27 to 46, wherein the at least one transformation operation involves all of the residual elements in the first plurality of residual elements and all of the residual elements in the second plurality of residual elements.
48. A method comprising: performing at least one transformation operation involving a plurality of correlation elements to generate at least one residual element in a first plurality of residual elements and at least one residual element in a second plurality of residual elements, the plurality of correlation elements being dependent on an extent of correlation between the at least one residual element in the first plurality of residual elements and the at least one residual element in the second plurality of residual elements, the first plurality of residual elements being useable to reconstruct a first part of a first representation of a signal at a relatively high level of quality using a first part of a second representation of the signal at the relatively high level of quality, the second plurality of residual elements being useable to reconstruct a second, different part of the first representation of the signal using a second, different part of the second representation of the signal; deriving the first part of the second representation from a first set of one or more signal elements in a representation of the signal at a relatively low level of quality; deriving the second part of the second representation from a second, different set of one or more signal elements in the representation of the signal at the relatively low level of quality; and using at least the first plurality of residual elements, the second plurality of residual elements, the first part of the second representation and the second part of the second representation to generate output data, wherein the at least one transformation operation is performed following the plurality of correlation elements being decoded, wherein the method further comprises generating data to indicate that at least some of the plurality of correlation elements are not to be decoded.
49 A method according to claim 48, wherein the at least one transformation operation comprises at least one inverse directional composition operation
50. A method according to claim 48 or 49, wherein the at least one transformation operation comprises at least one mathematical operation.
51. A method according to any of claims 48 to 50, wherein the at least one transformation operation comprises at least one matrix multiplication operation
52. A method according to any of claims 48 to 51, wherein the at least one transformation operation comprises at least one linear combination operation.
53. A computer program comprising instructions which, when executed, cause an apparatus to perform a method according to any of claims 27 to 52.
54. A computer-readable medium comprising a computer program according to claim 53.