GB2268655A

GB2268655A - Image data processing

Info

Publication number: GB2268655A
Application number: GB9213790A
Authority: GB
Inventors: James Hedley Wilkinson; Clive Henry Gillard; Terence Ralph Hurley; Michael John Ludgate; Jonathan Mark Soloff; Michael David Cunningham
Original assignee: Sony Broadcast and Communications Ltd; Sony United Kingdom Ltd
Current assignee: Sony Broadcast and Communications Ltd; Sony Europe BV United Kingdom Branch
Priority date: 1992-06-29
Filing date: 1992-06-29
Publication date: 1994-01-12
Anticipated expiration: 2012-06-29
Also published as: GB2268655B; GB9213790D0

Abstract

The range data processing system includes both a DCT transformation circuit (6) and a sub-band transformation circuit (8), which are selected by switches 4, 14. Data sequencers 10, 12 allow the rearrangement of the particular frequency coefficiente of the transformed image by writing the transformed data into 2 frame store (34 see fig.6) under the control of a first address pattern and by reading the data from the store under the control of a second address pattern. The sequence matched DCT and sub-band transformations can be used in similar data processing techniques, thus reducing hardware costs. <IMAGE>

Description

IMAGE DATA PROCESSING This invention relates to the field of image data processing. More particularly, this invention relates to image data. processing that involves a transformation from the spatial domain to the spatial frequency domain.

The use of domain transformations within image data processing systems is known. There are many different types of domain transformation that may be applied, e.g. discrete fourier transformation, discrete cosine transformation, Hadamard transformation, sub band transformation etc. The form and properties of the transformed data produced by these differing transforms vary considerably. As a result, each transform tends to have an associated field of use, e.g. DCT in image data compression.

It will be appreciated that once an image has been transformed using a particular transformation technique, a specific inverse transformation must be used to return the image to its original domain.

Thus, if an image is transformed with DCT, then it must be returned to its original domain with a matched inverse discrete transformation (IDCT). Another factor that distinguishes how these different techniques may be used is the way in which subsequent processing performed upon the transformed data is matched to the particular sequence of that transformed data. Given the typically high data rates in image data processing systems, matching of the subsequent processing to the sequence of the data being handled is used to improve performance.

The decision as to which transform is to be performed within a system has a large effect on how that system is implemented. The differences between the forms of the transformed data are such that commonality between systems using different transformations would is slight. No image data processing system is to be expected to operate with anything other than data transformed using the particular transform for which that system was designed.

This invention provides apparatus for processing input image data, said apparatus comprising: a DCT transformation unit for performing discrete cosine transformation upon said input image data to generate DCT data; a sub band transformation unit for performing sub band transformation upon said input image data to generate sub band data; a data sequencer for sequencing at least one of said DCT data and said sub band data such that said DCT data and-said-sub:band--data have a matching data sequence; and means for processing a selected one of said DCT data and said sub band data to form output image data.

The invention both recognises and exploits a similarity between DCT data and sub band data. This similarity is sufficient that DCT data and sub band data can be placed into a matching sequence such that the image data processing system can use the same hardware and techniques upon either form of data once it has been placed into the matching sequence. Thus, the need to develop completely different systems for the different transforms can be avoided and a single system capable of processing data associated with either transform can be more readily achieved.

It will be appreciated that the use of domain transformation is particularly suited to systems in which said input image data comprises arrays of pixel values. The arrays of pixel values may be interlaced fields of full frames.

A preferred way in which the sub band data and the DCT data may be made to have a matching sequence is that said data sequencer is operable to sequence at least one of said DCT data and said sub band data such a DCT data value from said DCT data occurs at a position within said DCT data corresponding to that sub band data value from said sub band data most closely correlated to said DCT data value.

In this way, the data values that most closely relate in each of the sub band data and DCT data appear at the same point and will be subject to matching processing operations.

The relationship between the sub band data and the DCT data can be simplified in systems in which said DCT transformation unit processes each array of pixel values as an array of N horizontal by M vertical blocks of pixel values and transforms each block of pixel values into an array of X horizontal by Y vertical cosine coefficients, and said sub band transformation unit transforms each frame of pixel values into X horizontal by Y vertical sub bands, each sub band containing frequency component samples from an array of N horizontal by M vertical sample regions corresponding to said blocks of pixel values.

The need to place the sub band data and the DCT data into a matching sequence, and consequently the usefulness of the-invention, -is increased in systems in which said matching data sequence gathers together data representing similar spatial frequencies and said means for processing performs at least one processing operation that varies in dependence upon position within said matching data sequence.

Commonly used systems in which such variation is strong are when said means for processing includes a quantiser that applies a quantisation step width that varies in dependence upon position within said matching data sequence and when said means for processing includes a data coding unit that applies differing coding in dependence upon position within said matching data sequence.

A complementary aspect of the invention provides apparatus for processing input image data that is either DCT data generated by discrete cosine transformation or sub band data generated by sub band transformation, said DCT data and said sub band data having a matching data sequence, said apparatus comprising:: means for processing said input image data in said matching data sequence; a data sequencer for receiving image data from said means for processing and for sequencing at least one of image data that is DCT data and image data that is sub band data such that said image data that is DCT data and said image data that is sub band data have nonmatching data sequences; a DCT transformation unit for performing discrete cosine transformation upon said DCT data to form output image data; and a sub band transformation unit for performing sub band transformation upon said sub band data to form output image data.

In this aspect the input data is either sub band data or DCT data in a matching sequence which is subject to processing by the same circuitry before being sequenced into non-matching sequences suited to their respective transformations of the sub band domain and DCT domain.

An embodiment of the invention will now be described by way of example, with reference to the accompanying drawings in which: Figure 1 illustrates a data transformation, quantisation, coding and recording apparatus; Figure 2 schematically illustrates sub band transformation; Figure 3 schematically illustrates DCT transformation;- Figure 4 illustrates the relationship between pixel values of DCT data and sub band data; Figure 5 illustrates scanning patterns for the DCT-data and sub band data of Figure 4; Figure 6 illustrates a data sequencer; Figure 7 illustrates the spatial frequency characteristics of the "DO" coefficient in DCT data;; Figure 8 illustrates the spatial frequency characteristics of the "DO" sub band in sub band data; Figure 9 illustrates a quantisation matrix; Figure 10 illustrates a data coder; and Figure 11 illustrates an apparatus for reproducing, decoding, dequantizing and transforming image data.

In Figure 1 input image data is applied to input node 2. A multiplexer 4 passes this input image data to either a DCT transformation unit 6 or a sub band transformation unit 8. Each of the DCT transformation unit 6 and the sub band transformation unit 8 have a respective data sequencer 10, 12 at their output which serves to place the transformed data into an appropriate sequence for subsequent processing. The operation of the sequencers 10, 12 is arranged so that they produce data in a matching sequence. The output from the data sequencers 10, 12 is supplied to a multiplexer 14.

The DCT transformation unit 6 and data sequencer 10 provide one branch of a parallel path and the sub band transformation unit 8 and the data sequencer 12 provide the other branch. The combined action of the two multiplexers 4, 14 serves to select either the DCT transformation technique or the sub band transformation technique for application to the input image data at the input node 2.

The output from the multiplexer 14 is passed through a quantiser 16 and data coder 18 and is recorded on a magnetic tape 20 by recording circuits (not illustrated). The processing applied by the quantiser 16 and the data coder 18 is varied depending upon the position within the sequence. This variation is controlled via data stored in a quantisation matrix 22 for the quantiser 16 and coding tables 24 for the data coder 18.

The overall operation of the apparatus of Figure 1 allows either DCT or sub band transformation to be selected under user~control prior to compression being applied to the data by the quantiser 16 and data coder 18. The transformed and compressed data -is then--recorded-on the magnetic tape 20. The provision of the data sequencers , 12 all-ows common circuitry to be used for the data compression processing whilst providing the flexibility of supporting either transformation technique.

Figure 2 schematically illustrates sub band transformation. An input image 26 comprising an array of pixel values is subject to sub band transformation to produce an array of sub band components 28. The input image 26 is passed through a branching hierarchy of respective horizontal and vertical filters to separate it into its sub band components.

Moving downwards in the array of sub band components 28 represents an increase in vertical spatial frequency and moving rightwards represents an increase in horizontal spatial frequency. Y vertical frequency bands are sampled and X horizontal frequency bands are sampled. The outputs from the array of filters are sub sampled and, as illustrated, the entire input image 26 makes a contribution to each of the sub band components. The sub band component 30 represents the lowest horizontal and vertical frequency and is termed the "DC" sub band. Each sub band is sub sampled so as contain sub band components values from an array of N horizontal by M vertical regions within the input image 26.The nature of input image data is such that most of the information is of a low frequency nature as can be seen from the decrease in content of the sub band components with increasing spatial frequency.

Figure 3 schematically illustrates the operation of DCT transformation. The same input image 26 is split into an array of N horizontal by M vertical blocks of pixel values. Each of these blocks of pixel values is then subject to the DCT process to form a respective block of cosine coefficients. Each block of cosine coefficients contains X horizontal by Y vertical cosine coefficients; Within each block, the vertical frequency represented by the. cosine coefficient increases on moving downwards and the horizontal frequency increases on moving rightwards. It will be seen that the majority of the information within each block of cosine coefficients is contained towards the lower frequency cosine coefficients.Since each block of cosine coefficients relates to a discrete block of pixels within the input image 26, no impression of the overall image can-be ebtained from one block of cosine coefficients.

Figure 4 illustrates the correlation between DCT data values and sub band data values. In the sub band data all of the "DC" data values are present in one sub band component. In contrast, in the DCT data each block of cosine coefficients contains one cosine coefficient at the upper left hand corner which is a "DC" value for the block of pixels it represents. In a similar way, each of the sub band components contains data values with corresponding cosine coefficient data values spread amongst each of the blocks of cosine coefficients.

Whilst the similarity between respective DCT data values and sub band data values is good enough that common subsequent processing may be applied when they are placed into a matching sequence, the correlation between individual values is not exact. One reason for this is that the DCT transform operates on a finite block of pixels, but the sub band transformation filters the entire image. Each sub band data value is predominantly influenced by a small area comparable to the block of pixels used in the DCT transformation but there is at least some contribution from further afield.

Figure 5 illustrates scanning patterns that can be applied to the data in the forms shown in Figure 4 so as to achieve a matching data sequence. The desired sequence is that all of the data relating to a particular horizontal and vertical frequency for the entire image is gathered together. In this way, subsequent processing may be varied in dependence upon what spatial frequencies are being handled.

In the case of sub band data in the form illustrated in Figure 4, a zig-zag scan through each sub band component is performed. In the case of the DCT data in the sequence shown in Figure 4, a zig-zag scan through the entire array of blocks of DCT cosine coefficients is performed picking up the cosine coefficients for the same spatial frequency from each block of cosine coefficients.- I-n-the illustration, the "DC" cosine coefficients are being picked out the by zig-zag scan.

It will be appreciated that different scanning patterns could be used providing the scanning pattern for the sub band data is complementary to that for the DCT data.

Figure 6 illustrates a data sequencer. Data from either the DCT transformation unit 6 or the sub band transformation unit 8 is input to node 32. This data is then written into a frame store 34 under control of a write address W. The write address W is generated by. a counter 40 whose incrementing value is mapped to the write address W via a PROM 42. In this way, the data values output by the transforms can be directed to appropriate positions within the frame store so as to generate data of the form illustrated in Figures 2, 3 and 4.

When a frame of data has been transformed and stored in the frame store 34, it may be read from the frame store under control of a read address R. Again, the read address R is generated by the combined action of a counter 44 and a PROM 46. The PROM 46 stores a mapping to effect one of the scanning patterns illustrated in Figure 5.

Figures 7 and 8 illustrate the similarity, but not complete equivalence, of the DCT and sub band transform techniques that is exploited by this invention. Figure 7 illustrates the contribution a spatial frequency makes to the "DC" cosine coefficient for a given block of pixels. In the case of a 8*8 block of pixels within an array of 8*8 such blocks, the major weighting is below fas/8 (f, = sampling frequency).

In the case of the sub band transformation and the "DC" sub band illustrated in Figure 8, the major part of the spatial frequency contribution to this sub band occurs below Fs/8. Those contributions above fas/8 in both cases produce alias effects 48, 50. It will be appreciated from Figures 7 and 8 that the two approaches are substantially the same with the sub band approach providing better frequency partitioning whereas the DCT approach provides an easier implementation with only 8*8 pixel values having to be considered at any one time.

As illustrated in Figure 1, the quantiser 16 varies the processing it performs in dependence upon a quantisation matrix. Such a matrix is illustrated in Figure 9. The matrix of Figure 9 is intended to be used in systems with 8*8 sub bands.or 8*8 blocks of cosine coefficients. The low frequency and "DC" data is subjected to less quantisation than the high frequency data. Quantisation is a lossy process and it has been found that a loss of resolution in the low frequency components results in a greater subjective effect that a loss of resolution in the high frequency components. Thus, the high frequency components are more heavily quantized.

Similarly, the data cdder 18 illustrated in Figure 1- applies different coding to the data depending upon what frequency content is being represented. The data coder 18 is illustrated in Figure 10. The transformed, quantized and sequenced data is input to a multiplexer 52 which directs the data to either a run length coder 54 or a single symbol coder 56 in dependence upon a scan number. The scan number indicates which of the sub band components or collection of cosine coefficients of a particular spatial frequency is being received. The "DO" sub band or DCT coefficients are fed to the single symbol coder 56. All the other data is fed to the run length coder 54. The single symbol coder 56 is used for the "DC" data since this data rarely contains long runs of constant values.The outputs from the run length coder 54 and the single symbol coder 56 are fed to a Huffman coder 58 which applies one of a collection of Huffman coding tables in dependence upon the indicated scan number. The use of different Huffman coding tables increases compression efficiency since the characteristics of the data representing different spatial frequencies vary considerably and so a single Huffman coding table that suited one part of the data would not suit another.

Figure 11 illustrates a reproducing, decoding, dequantizing and transforming apparatus. Data is read from the magnetic tape 60 and fed via playback and amplification circuits (not illustrated) to a decoder 62 and then a dequantiser 64. The decoder 62 applies a varying decoding table 66 in dependence upon which part of the transformed image data. Similarly, the dequantisation applied by the dequantiser 64 is the inverse of that applied during quantisation and is controlled by a dequantisation matrix 68. The output from the dequantiser 64 is fed to a multiplexer 70 which feeds it to either data sequencer 72 and DCT transformation unit 74 (in fact IDCT) or data sequencer 76 and sub band transformation unit 78.

The data sequencers 72, 76 serve to change the sequence of the data from the matching sequence chosen for the coding and quantisation processes to non-matching sequences required by the different transformation processes. These data sequencers 72, 76 have the same form as illustrated in Figure 6 with different mappings being stored in the PROMs 42, 46. The data stored in respective ones of the data sequencers 72, 76 have the forms illustrated in Figure 4.

One of . the DCT transformation unit and the - sub band transformation unit 78 transforms the data back into the spatial domain. The output from the currently active one of the DCT transformation unit 74 and the sub band transformation unit 78 is selected by a multiplexer 80 and fed to an output node 82.

The apparatus of Figure 11 is able to cope with reproducing either DCT or sub band data from the magnetic tape 60 whilst maintaining a large portion of common processing circuitry.

Claims

1. Apparatus for processing input image data, said apparatus comprising: a DCT transformation unit for performing discrete cosine transformation upon said input image data to generate: DCT data; a sub band transformation unit for performing sub band transformation upon said input image data to generate sub band data; a data sequencer for sequencing at least one of said DCT data and said sub band data such that said DCT data and said sub band data have a matching data sequence; and means for processing a selected one of said DCT data and said sub band data to form output image data.

2. Apparatus as claimed in claim 1, wherein said input image data comprises arrays of pixel values.

3. Apparatus as claimed in any one of claims 1 and 2, wherein said data sequencer is operable to sequence at least one of said DCT data and said sub band data such a DCT data value from said DCT data occurs at a position within said DCT data corresponding to that sub band data value from said sub band data most closely correlated to said DCT data value.

4. Apparatus as claimed in claim 3, wherein said DCT transformation unit processes each array of pixel values as an array of N horizontal by M vertical blocks of pixel values and transforms each block of pixel values into an array of X horizontal by Y vertical cosine coefficients, and said sub band transformation unit transforms each frame of pixel values into X horizontal by Y vertical sub bands, each sub band containing frequency component samples from an array of N horizontal by M vertical sample regions corresponding to said blocks of pixel values.

5. Apparatus as claimed in any one of claims 2, 3 and 4, wherein said means for processing performs at least one processing operation that varies in dependence upon position within said matching data sequence.

6. Apparatus as claimed in claim 5, wherein said means for processing includes a quantiser that applies a quantisation step width that varies in dependence upon position within said matching -data sequence.

7. Apparatus as claimed in any one of claims 5 and 6, wherein said means for processing includes a data coding unit that applies differing coding in dependence upon position within said matching data sequence.

8. Apparatus for processing input image data that is either DCT data generated by discrete cosine transformation or sub band data generated by sub band transformation, said DCT data and said sub band data having a matching data sequence, said apparatus comprising:: means for processing said input image data in said matching data sequence; a data sequencer for receiving image data from said means for processing and for sequencing at least one of image data that is DCT data and image data that is sub band data such that said image data that is DCT data and said image data that is sub band data have nonmatching data sequences; a DCT transformation unit for performing discrete cosine transformation upon said DCT data to form output image data; and a sub band transformation unit for performing sub band transformation upon said sub band data to form output image data.

9. Apparatus as claimed in claim 8,- wherein said output image data comprises frames of pixel values.

10. Apparatus as claimed in any one of claims 8 and 9, wherein said data sequencer is operable to sequence at least one of said DCT data and said sub band data such a DCT data value from said DCT data occurs at a position within said DCT data corresponding to that sub band data value from said sub band data most closely. correlated -to said DCT data value.

11. Apparatus as claimed in claim 10, wherein said DCT transformation unit processes said DOT data as an array of N horizontal by M vertical blocks of cosine coefficients, each block of cosine coefficients comprising an array of X horizontal by Y vertical cosine coefficients, and said sub band transformation unit processes said sub band data as X horizontal by Y vertical sub bands, each sub band containing frequency component samples from an array of N horizontal by M vertical sample regions corresponding to said blocks of cosine coefficients.

12. Apparatus as claimed in any one of claims 9, 10 and 11, wherein said matching data sequence gathers together data representing similar spatial frequencies and said means for processing performs at least one processing operation that varies in dependence upon position within said matching data sequence.

13. Apparatus as claimed in claim 12, wherein said means for processing includes a dequantiser that applies a dequantisation step width that varies in dependence upon position within said matching data sequence.

14. Apparatus as claimed in any one of claims 12 and 13, wherein said means for processing includes a data decoding unit that applies differing decoding in dependence upon position within said matching data sequence.

15. Apparatus substantially as hereinbefore described with reference to Figures 1 to 10 of the accompanying drawings.

16. Apparatus substantially as hereinbefore described with reference to Figures 2 to 11 of the accompanying drawings.