GB2556115A

GB2556115A - Data processing systems

Info

Publication number: GB2556115A
Application number: GB1619708.9A
Authority: GB
Inventors: Karol Bogusz Michal; Tadeusz Chrobak Piotr
Original assignee: ARM Ltd; Advanced Risc Machines Ltd
Current assignee: ARM Ltd
Priority date: 2016-11-22
Filing date: 2016-11-22
Publication date: 2018-05-23
Anticipated expiration: 2036-11-22
Also published as: GB2556115B; CN108093192B; US10659723B2; CN108093192A; GB201619708D0; US20180146158A1

Abstract

A data processing system includes a scaler operable to scale a received input data array (30) (e.g. even or odd frame of an image) to provide a scaled output version (31) of the input array. When it is desired to produce a de-interlaced and scaled (e.g. horizontally or vertically) output version of an input array, the scaler scales the input array so as to simultaneously de-interlace and scale the input array and to produce a de-interlaced and scaled output version of the input array at the same time. This may be achieved by the selection of scaling parameters which may be based on the ratio of the size of the output array to the input array, e.g. initial vertical or horizontal phase offset (which may be negative), being different for odd and even fields. The input array may comprise a rotated and/or flipped version of a data array (figs. 19-22).

Description

(54) Title of the Invention: Data processing systems

Abstract Title: Simultaneously Scaling and De-interlacing an Input Array (57) A data processing system includes a scaler operable to scale a received input data array (30) (e.g. even or odd frame of an image) to provide a scaled output version (31) of the input array. When it is desired to produce a deinterlaced and scaled (e.g. horizontally or vertically) output version of an input array, the scaler scales the input array so as to simultaneously de-interlace and scale the input array and to produce a de-interlaced and scaled output version of the input array at the same time. This may be achieved by the selection of scaling parameters which may be based on the ratio of the size of the output array to the input array, e.g. initial vertical or horizontal phase offset (which may be negative), being different for odd and even fields. The input array may comprise a rotated and/or flipped version of a data array (figs. 19-22).

**•*^30 ^-31

FIG. 11

At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy.

/18

4

I * DISPLAY

1202 18

OFF-CHIP

MEMORY

2/18

1202 18

3/18

1202 18

18c

4/18

1202 18

external data interface, e.g, HDMI, DPI.

MIPI

DDR SDRAM i^-7

5/18 even frame •22

CM

O

CM odd frame original frame

0	.,0¾..............	0
		1
2	-	2
		3
4	-	4
		5
6	-a-	6
		7
8	-	8
		9
10	-	10

original content video stream

-21a '22a '21b '22b '21c -22c '21 d '22d

6/18 o

co co

CO

co • '’srt·' a> CZ3 as

-ίΣΣ “O eg .CM ^:ll as co •CO jS

CD

CD co

CO cd co co €O co wo co wo

1202 18

Ο a

Φ

JM <D i

re i

JO

WO

CO

CM

cmco .CM •CO

“θ ^!.CM -:

•co

-Ό;cm~ •co o

ω co re r“ co co wo co rco wo co

CM co wo *53“ co

CM wo ’st' co co

CM

CM re <o co

CM <O

CM co

CM

CO

CM CO WO

CX. Ok Ok Ok CX ex <ϋ re ¢0 CO Π3 Π3 —-8*¹

WO

CO

CM

;;;.

CO

CD

CO

r-

co

wo

co

CM

^v~

O’

CO

Φ rM 'co.

'S

o.

Ο

CO •co .ις|“ —cs?

CN ’v— .CO _

co r-» co

7/18

CD cd oo co

CO ™CO .CN •CO

CD oo

C-.

co

LO

1202 18 θ

J φ

ρν

CZ3 ί

CZ

I χχ

LO ’SCf”

CO

CN

Φ co co r“ fa

CO

-r—<

Φ

O •’nT' sg co “O .CN

CO —co

CN • CO o

CN CO CO _£Χ

CO S2l 03

Φ “03

GO .CN •CO ’-53-Ό .CN •CO

	Φ
	LO co
	JSSS
“θ	CSX
	co
.CN	A¹
•v—	Φ
	“O
CO

~”O> „CN_ •CO

Φ

CD

CO XX CL — CS>^..

.CN

CO cs

CD co

CO

LO co co w>

CO

CN co wo ’st' co

CN co

LO co

CN

LO

CO

CN co co

CN

CD

CN

CO

-S* ex

Φ

LO

CO f

fa cs <o

CN CO -s3“ LO iSSL {SSL Ok CSX CSX

CO Φ CO co co co

S3 o

t

Φ

PN ’co, o,

8/18

1202 18

β	4;BA-7
αΑΒ·<	JbS	/Sx
\|jAGS	ΐ·'Β·ί	\|;ec\|

	/•/MA.

’/ab/·'	βί	;-/SK

		\|Β;\|

		UBArf	\|cg\|
Iwl	B	•ABAy	\|O\|
TAfiY'	ijABji	ffiB
TAB4	•Bl/i		•IB
\|i§S\|	\|W%	zBCA:	\|§g\|
		ABGA	IQ

9/18 in first location

0	1	2	3	4
_a delta	delta f ’	delta	delta	delta '	delta r !	_a delta _E f V

-^-30 out

0

1

2

3

4

5

Λ 0

F—

1202 18

10/18 out frame in frame

Q j -88*-	0
	1
j -a-b*	2
	3
2 -β-88*-	4
	5
3 —s-	6
	7

1202 18

in frame out frame

0		0
		1
1		2
		3
2		4
		5
3	-a-x-	6
		7

vertical direction in

4-4-+-+-+4-+

4-+-+ phase(n) 8 101214 q 2 4 δ 8 101214 +4-+4-+4-+

4 δ 8 101214

1-4-42 4 6 8 10 12 14 della phase della phase +-++ out aelia phase se pnase pnase

	I 1 i	f 1		F 1
0	1	2	3	4	5	6	7

^—•31 /18 in frame out frame

0	—	——gs*»	0
			1
1	—	-	2
			3
2	—	-	4
			5
3	-S-		6

1202 18 vertical direction

in frame out frame

vertical direction

12/18

CM

Ο

CM

φ

E co

Ό

Φ

O

Φ -+-j

E 'φ

Ό

θ	.....	CM	CO		GO	<D	[·*·»,

CO

CM

13/18

CM

Ο

CM ο

co φ

E co

Z5

O

Φ

E co

O

Φ o

JTO ω

Φ

O

Φ

E co

CM

14/18

HORIZONTAL FLIP

HORIZONTAL FLIP + VERTICAL FLIP

1202 18

NATURAL

ARM*
ROTATION 90°
	z O£ <

ROTATION 180°

WW

MRA

HORIZONTAL FLIP + ROTATION 90° <

Z

VERTICAL FLIP

VKW.
VERTICAL FLIP ROTATION 90^s	+
	2 >

ww

VERTICAL FLIP + HORIZONTAL FLIP + ROTATION 90°

HORIZONTAL FLIP + ROTATION 180°

VERTICAL FLIP + ROTATION 180°

VERTICAL FLIP + HORIZONTAL FLIP + ROTATION 180°

WW ’MIA

ARM*

HORIZONTAL FLIP + ROTATION 270°

VERTICAL FLIP + ROTATION 270°

VERTICAL FLIP + HORIZONTAL FLIP + ROTATION 270°

ROTATION 270°

<		z
		oc
Σ		<

>- £= co Lr

15/18

1202 18

□7 O LU LU

I— 3S ϋ a

NOI133dia

30V1d31NI

_{u u u}	o
	or
Q or <	LU O <c
Q	or
	LU
<c	1—
1—
co	LLJ
	Q

O l!

or o LU LU i— — a or o LU LU IEC a

O	>—
or	or
Ll_	O
1—
=)	LU
Q_	*>

NOI133dia

30V1H31NI

DEINTERLACING

DIRECTION

O >7 07 07 LU O

U0IP9JIP 6upe|J9)UI9p

1202 18

Ο !

'-S— s

o

*O uoipgjip Bupepgjupp jgpjo S9ui| Ae|dsip co

c:

Φ o

o

Ό

O

JT>

k„

17/18

CM

Ο

CM

co

CN

18/11

CM

Ο

CM c

'o jo

Φ

0*5 o

1± co σ5

CM

LO >

	cz
εζ	_o

O> c	o aa
o	Ό
co
k__	CO
-S	c:
cz	O
0>	jm
Ό	Luu, o

Φ o

o>

c

	/ O
to	/ ©
	/ θ
jg	/
Φ	g
£—*»	c -°
co	\
u_ CO	\ °
Cl	\ 2

	\ o

- 1 Data Processing Systems

The present invention relates to data processing systems, and in particular to data processing systems that support and use interlaced video.

Interlacing is a commonly used technique in data processing systems for video data. Each video frame of the video data is effectively divided into two frames or “fields”, namely an odd frame (odd field) and an even frame (even field). Each odd frame is made up of the odd lines of the corresponding original video frame and each even frame is made up of the even lines of the corresponding original video frame. Each video frame of the video data is propagated through the data processing system in the form of these two separate (odd and even) frames.

In order to display interlaced video, each of the odd and even frames is used to produce its own “full” frame for display. This is done by de-interlacing each of the odd and even frames, whereby the frame in question is interpolated to produce a “full” de-interlaced frame for display. Each de-interlaced frame is then provided to a display for display.

In this way, the frame rate at which the video data is displayed is doubled relative to the frame rate at which the video data is generated or decoded, without consuming extra bandwidth in the data processing system.

In data processing systems, it is often necessary to scale (upscale or downscale) data arrays, e.g. video frames, prior to the data arrays being displayed.

The Applicants believe that there remains scope for improvements to data processing systems.

According to a first aspect of the present invention, there is provided a method of operating a data processing system that includes a scaler operable to scale a received input data array to provide a scaled output version of the input data array, the method comprising:

when it is desired to produce a de-interlaced and scaled output version of an input data array:

providing the input data array to the scaler; and the scaler scaling the input data array so as to simultaneously de-interlace and scale the input data array and to thereby produce a de-interlaced and scaled output version of the input data array.

-2According to a second aspect of the present invention, there is provided a data processing system, the data processing system comprising:

a scaler operable to scale a received input data array to provide a scaled output version of the input data array;

wherein the data processing system is configured, when it is desired to produce a de-interlaced and scaled output version of an input data array, to provide the input data array to the scaler; and wherein the scaler is operable to scale a received input data array so as to simultaneously de-interlace and scale the input data array, to thereby produce a deinterlaced and scaled output version of the input data array.

The present invention relates to the de-interlacing and scaling of input data arrays. The Applicants have recognised that when it is desired to both de-interlace and scale an input data array, these two operations can be (and in a preferred embodiment are) performed simultaneously (i.e. in a single pass) by appropriate scaling using a single scaler.

This is in contrast with known methods wherein two separate stages are provided in the data processing system and used for these two operations, e.g. a de-interlacing stage and a scaling stage. Thus the operation in the manner of the present invention can reduce the hardware requirements (chip area) of the data processing system.

This is also in contrast with known methods wherein de-interlacing and scaling of an input data array is performed as a two-pass operation, e.g. where a de-interlaced version of the input data array is written out to memory, and then read in from memory for scaling. Thus the operation in the manner of the present invention can also reduce the bandwidth and power consumption requirements of the data processing system.

It will be appreciated, therefore, that the present invention provides an improved data processing system.

The scaler of the data processing system can be any suitable and desired scaler that is operable to scale an input data array to provide a scaled output data array. Thus it can be any suitable and desired scaling unit, scaling engine, scaling pipeline, etc.

In a particularly preferred embodiment, the scaler that is used to scale the input data array comprises a scaler of a given processing unit (processor) of the overall data processing system. Thus, in a preferred embodiment, the data

-3processing system includes a processor that includes a scaler, and the input data array to be de-interlaced is provided to the scaler of that processor for scaling.

Such a processor can be any suitable and desired processor (processing unit) of a data processing system that includes a scaler. In a preferred embodiment it is a display controller, a graphics processor (graphics processing unit (GPU)), a video processor (video processing unit (VPU)), or an image signal processor (ISP). In a particularly preferred embodiment it is a display controller (for a display of the data processing system).

In a preferred embodiment, the mode of operation of the processing unit (e.g. the display controller) is controlled by an application, e.g. running on a host processor, preferably by the application generating instructions which are interpreted by a driver for the processing unit (that is running on the host processor) to generate appropriate commands to the processing unit to operate as required by the application. Preferably, where the application requires an input data array to de-interlaced and scaled, then the application generates appropriate instructions which are interpreted by the driver to generate appropriate commands to the processing unit (e.g. display controller) to operate in the manner of the present invention, i.e. to perform simultaneous de-interlacing and scaling using the scaler.

The scaler may be implemented as desired, e.g. and preferably, as appropriate processing circuitry (either that is programmable to perform the desired scaling operation, and/or that comprises fixed function scaling circuitry operable to perform the scaling operation).

The input data array to be de-interlaced can be any desired and suitable input data array that may need to be de-interlaced and scaled by the data processing system.

The input data array can represent any suitable and desired data array, but in a particularly preferred embodiment comprises an image, e.g., and preferably, a frame to be displayed. In a particularly preferred embodiment, the input data array is an even or an odd (video) frame (or “field”) to be de-interlaced (e.g., and preferably by a display controller) for display.

The input data array is preferably one such data array (frame) from a sequence of data arrays (frames), e.g. and preferably from a sequence of interleaved even and odd (video) frames (fields) to be de-interlaced for display.

The input data array should (and preferably does) comprise an array of data positions for which respective data values are stored, and should (and preferably

-4does) have a given horizontal and vertical size (in terms of the number of data positions in the horizontal and vertical directions for the input data array). Thus, the input data array preferably comprises plural columns of data positions and plural rows (lines) of data positions.

Correspondingly, the data values for the data positions of the data array can be any suitable and desired data values. In a preferred embodiment, the data values represent colour values such as RGB or YUV colour values (where the data array is an image, e.g. for display), but they could be other data values for other types of data array, as and if desired.

The input data array can be generated as desired. For example the data array(s) may be generated by a processing unit of the overall data processing system, such as by being appropriately rendered and stored into a memory (e.g. frame buffer) by a graphics processor of the data processing system. Additionally or alternatively, the data array(s) may be generated by being appropriately decoded and stored into a memory (e.g. frame buffer) by a video codec (video processor). Additionally or alternatively, one or more data arrays may be generated by a digital camera image signal processor (ISP), or other image processor. The data array(s) may be, e.g., for a game, a demo, a graphical user interface (GUI), a GUI with video data (e.g. a video frame with graphics “playback” and “pause” icons), etc.

Where the input data array is one of a pair of even and odd (video) frames (fields) to be de-interlaced for display, then each of the even and an odd (video) frames (fields) may be stored in a respective frame buffer in memory, e.g. main memory, of the data processing system.

The input data array to be de-interlaced preferably corresponds to and/or is generated from a corresponding initial data array, e.g. an initial frame. The initial data array (frame) is preferably a “full” data array (frame) for display, e.g. where the input data array preferably comprises every other row (line) (e.g. each of the odd or even rows (lines)) of the initial data array (frame). The initial data array (frame) is preferably one such data array (frame) from a sequence of data arrays (frames) for display.

Thus, in a preferred embodiment, there is a sequence of initial data arrays, e.g. a sequence of (video) frames for display, which is generated in the form of and/or which is used to generate a sequence of data arrays (frames) to be deinterlaced, e.g. a sequence of interleaved even and odd (video) frames (fields) to be de-interlaced. One or more or each of the sequence of data arrays (frames) to be

-5de-interlaced may be de-interlaced and scaled in the manner of the present invention.

In the present invention, the scaler scales the input data array so as to simultaneously de-interlace and scale the input data array and to produce a deinterlaced and scaled output version of the input data array. That is, the scaler performs a single scaling operation in order to both de-interlace and scale the input data array. Thus, the de-interlaced and scaled output version of the input data array (the output data array) is not merely (is other than) a de-interlaced version of the input data array, but is a scaled version of a de-interlaced version of the input data array.

The de-interlaced and scaled output data array that is produced from the input data array will comprise an appropriately de-interlaced and scaled version of the input data array. Thus, where the input data array is an even or an odd video frame (or “field”) to be de-interlaced for display, the output data array will comprise a de-interlaced and scaled frame for display.

The scaling operation that is performed on the input data array to provide the de-interlaced and scaled output data array can comprise any suitable scaling operation.

In a preferred embodiment, the scaling operation uses interpolation (or “filtering”) in order to de-interlace and scale the input data array. Any suitable type of interpolation may be used, such as for example, nearest neighbour interpolation, bilinear interpolation, bicubic interpolation, polyphase interpolation, and the like. In a preferred embodiment, the scaler uses polyphase interpolation (polyphase filtering) to scale the input data array.

In a particularly preferred embodiment, the scaling operation scales the input data array in one direction (dimension), i.e. in either the horizontal or vertical direction. However, the present invention is equally applicable to arrangements in which an input data array is being scaled in both directions (dimensions) (e.g. both horizontally and vertically). Thus the scaling operation that is performed on the input data array may be to scale the input data array in one or both directions (horizontal and vertical).

In a preferred embodiment, the scaler is operable to scale the input data array by (separately) scaling each of plural sets (sub-sets) of data positions of the input data array, e.g. and preferably, so as to produce a scaled output version of each set (sub-set) of data positions. Accordingly, the de-interlaced and scaled

-6output version of the input data array preferably comprises scaled output versions of each of plural input sets (sub-sets) of data positions of the input data array.

Each of the plural input sets of data positions of the input data array may have any suitable size and shape. However, in a preferred embodiment, when scaling the input data array in a particular (horizontal or vertical) direction, the scaling operation is performed on a column-by-column or row-by-row (line-by-line) basis. That is, each input set (sub-set) of data positions preferably comprises a column or a row of data positions (i.e. a set of data positions that is one data position high and many data positions long or vice versa), and each input column or each input row of data positions of the input data array is preferably (separately) scaled so as to produce a scaled output version of that column or row of data positions (where each scaled output column or row preferably comprises a set of data positions that is one data position high and many data positions long or vice versa). Accordingly, the de-interlaced and scaled output version of the input data array preferably comprises scaled output versions of each of plurals input columns or rows of data positions of the input data array.

For each input set (e.g. column or row) of data positions that will (and preferably does) comprise some first (integer) number (e.g. N) of data positions, a corresponding scaled output version of the input set (column or row) comprising some second, preferably different, (integer) number (e.g. M) of data positions is preferably produced. The second number of data positions (M) may be greater than or less than the first number of data positions (N), i.e. the input set (e.g. column or row) of data positions may be upscaled or downscaled.

In a preferred embodiment, when scaling the input data array, a data value (e.g. colour value) is determined for each data position of the de-interlaced and scaled output version of the input data array (for each data position of the output data array). This may be performed as desired. In a preferred embodiment, for each data position of each output set of data positions, the data value or values of one or more of the data positions of the corresponding input set of data positions is used to determine the data value in question.

The data value or values of any number of data positions of the input data array may be used to determine the data value for each data position of the output data array. For example, the data value of a single data position of the input data array may be used in respect of a, some or each data position of the output data array (this may be the case, e.g. when using nearest neighbour interpolation).

-7Alternatively, the data values of plural data positions of the input data array may be used in respect of a, some or each data position of the output data array (this may be the case, e.g. when using bilinear interpolation, bicubic interpolation, polyphase interpolation, and the like).

Where the data values of plural input data positions are used to determine the data value of an output data position, the data values of any plural number of input data positions may be used. In a preferred embodiment, the scaler has some fixed number of filter taps (which may be, e.g., defined by the hardware configuration of the scaler), and the data values of that number of input data positions are used to determine the data value for each output data position in the scaling operation.

The particular data position or positions of the input data array that is or are used to determine the data value for each output data position may be selected as desired. In a preferred embodiment, the input data position or positions that is or are used in respect of each output data position correspond to one or more nearest neighbour data positions.

The input data value or values may be used to determine the data value for each output data position in any suitable manner. For example, where the data value of a single input data position is used in respect of an output data position, the data value of the input data position may be used for the data value of the output data position.

Where the data values of plural input data positions are to be used to determine the data value for an output data position, then the plural data values may be combined to generate the output data value in any suitable manner. For example, the plural data values may be averaged to generate the output data value in question.

However, in a preferred embodiment, where the data values of plural input data positions are to be used to determine the data value for an output data position, a weighted average of the plural data values is determined and preferably used for the output data value. Preferably, a set of coefficients is determined, and is used to weight the contributions from each of the plural input data values to the output data value. The particular values of the coefficients may be selected as desired, e.g. and preferably, in accordance with the particular interpolation (filtering) technique that is being used.

-8In a preferred embodiment, the weighting (e.g. the set of coefficients) is determined on the basis of a phase value associated with the output data position in question. The phase is preferably an indication of how “aligned” the output data position in question is to the set of input data positions.

Preferably, an output data position has an associated phase value of zero where that output data position is aligned with one of the data positions of the input (set of) data positions. Where an output data position is not (is other than) aligned with one of the data positions of the input (set of) data positions, then the output data position will preferably have some non-zero phase associated with it.

Thus, for example, where each input set of data positions to be scaled comprises a row or column of data positions of the input data array (and where each output set of data positions comprises a scaled row or column of data positions), an output data position preferably has a phase value of zero where that output data position has a position in the output row or column of data positions that is the same as the position of one of the input data positions in the row or column of input data positions. This would happen, for example, where the overall scaling ratio is precisely one. Where an output data position does not have (has other than) a position that is the same as the position of one of the data positions of the input row or column of data positions, then the output data position will have some nonzero phase associated with it.

As will be appreciated by those having skill in the art, by using a phase value to select the weighting (set of coefficients) that is used to combine the data values of plural input data positions to produce the data value for an output data position, the scaler can produce an improved output, e.g. with reduced aliasing artefacts and the like.

Thus, in a preferred embodiment, each output data position of each output set of data positions has a phase value associated with it, where the phase value indicates how (spatially) aligned the output data position is in relation to the set of input data positions. The phase value associated with each output data position is used to determine the weighting (the set of coefficients) to be used to combine the data values of plural input data positions in order to determine the data value for the output data position.

The phase value is preferably an integer, and is preferably defined to be zero at (the centre of) each data position of the input set of data positions, and varies between zero and some maximum (integer) phase value between (the

-9centres of) adjacent input data positions. The maximum phase value may be selected as desired. In a preferred embodiment, the scaler is configured to have some (integer) number of phase values (Λ/P) (which may, e.g., be defined by the hardware configuration of the scaler), and the phase value varies between zero and that number between (the centres of) adjacent input data positions. Other arrangements would be possible.

As will be appreciated by those having skill in the art, where each input set of data positions to be scaled comprises a row or column of data positions of the input data array (and where each output set of data positions comprises a scaled row or column of data positions), since both the set of input data positions and the set of output data positions has a regular periodic form, then the set of input data positions or phase values to be used for a given set of output data positions can be (and preferably is) characterised by two scaling parameters, namely an initial (phase) offset together with a delta (phase) value.

The initial (phase) offset defines the distance (e.g. in units of phase) from one end of the input set (row or column) of data positions to (the centre of) the first output data position. The delta (phase) value defines the distance (e.g. in units of phase) between (the centres of) adjacent output data positions.

In a preferred embodiment, the scaler is programmable in respect of one or more scaling parameters (e.g. and preferably, in respect of the initial (phase) offset and/or the delta (phase) value), and the scaling operation can be controlled by appropriately programming the scaler in respect of the one or more scaling parameters.

As described above, the present invention lies in the realisation that when it is desired to both de-interlace and scale an input data array, these two operations can be (and preferably are) performed simultaneously (i.e. in a single pass) by appropriate scaling using a single scaler. In this regard, the Applicants have furthermore recognised that simultaneous de-interlacing and scaling in the manner of the present invention can be achieved by appropriate selection (programming) of the scaling parameters. This accordingly means that simultaneous de-interlacing and scaling in the manner of the present invention can be implemented by making only relatively minor modifications to existing scalers, e.g. that may already be present in existing data processing systems.

Thus, in a particularly preferred embodiment, one or more scaling parameters (preferably the initial input (phase) and/or the delta (phase) value) are

- 10determined (selected) for scaling the input data array so as to produce a deinterlaced and scaled output version of the input data array, and are preferably provided to the scaler for the scaling operation. The scaler is preferably configured to scale the input data array using the one or more determined scaling parameters so as to produce the de-interlaced and scaled output version of the input data array.

Accordingly, in a particularly preferred embodiment, when it is desired to produce a de-interlaced and scaled output version of an input data array, the scaler is configured, preferably with one or more scaling parameters, such that when the scaler performs a (single) scaling operation for the input data array, the scaler will produce a de-interlaced and scaled version of the input data array. The input data array is preferably then provided to the scaler, and the scalar scales the input data array so as to produce the de-interlaced and scaled output version of the input data array.

The one or more scaling parameters may be selected as desired so as to produce an appropriately de-interlaced and scaled output version of the input data array.

It should be noted in this regard, that when performing de-interlacing, simply upscaling the input array by a factor of two would not result in an appropriately deinterlaced data array. This is because the sequence of data arrays (frames) to be de-interfaced preferably comprises interleaved even frames (fields) comprising only the even lies of an initial (original) data array and odd frames (fields) comprising only the odd lies of the initial data array, so that simply upscaling each data array (frame) in turn would result in flickering of the displayed image, whereby the displayed image would appear to jump up and down from frame to frame. Equally, where the initial (original) data array (frame) comprises an odd number of lines, the resulting even and odd frames (fields) will comprise different numbers of lines, and so will require different levels of scaling to produce equally sized (de-interlaced) output arrays.

However, the Applicants have recognised that this can be addressed by appropriate selection (programming) of the one or more scaling parameters, and in particular by appropriate selection (programming) of the initial (phase) offset.

Thus, in a preferred embodiment, the one or more scaling parameters (preferably the initial (phase) offset) are selected so that the de-interlaced and scaled output version of the data array (the output array) has the same size and position irrespective of whether the input data array comprises an odd frame (field)

- 11 or an even frame (field) and/or irrespective of whether the initial data array comprises an odd or an even number of rows (lines). This beneficially avoids problems with flickering.

In a preferred embodiment, this is achieved by selecting the one or more scaling parameters (and in particular the initial (phase) offset) depending on whether the input data array comprises an odd frame (field) or an even frame (field), where the one or more scaling parameters (preferably the initial (phase) offset) are preferably selected to be different for odd and even frames.

In a preferred embodiment the one or more scaling parameters (and preferably one or both of the initial (phase) offset and the delta (phase) value) are determined (selected) based on the scaling ratio, that is, the ratio of the (vertical and/or horizontal) size of the de-interlaced and scaled output data array to the (vertical and/or horizontal) size of a de-interlaced version of the input data array (or correspondingly the ratio of the (vertical and/or horizontal) size of the de-interlaced and scaled output array to the (vertical and/or horizontal) size of the initial (original) input data array). This is preferably done so as to implement the scaling part of the simultaneous de-interlacing and scaling operation of the present invention. Most preferably, the scaling ratio is used to determine both the initial (phase) offset and the delta (phase) value.

However, it should be noted that this is not achieved simply by multiplying the one or more scaling parameters that would otherwise be used for de-interlacing (without scaling) by the scaling ratio. On the contrary, the Applicants have recognised that simultaneous de-interlacing and scaling in the manner of the present invention requires a non-trivial selection of the one or more scaling parameters, and in particular a non-trivial selection of the initial (phase) offset.

In a particularly preferred embodiment, where the input data array comprises an odd frame (field) to be de-interlaced and scaled, then the scaling parameters used in the scaling operation are selected to be:

delta_phase = - scaling_ratio NP, and 1 1 initial_phase = - NP + - scaling_ratio NP, where NP is the number of filter phases of the scaler.

- 12 Similarly, in a preferred embodiment, where the input data array comprises an even frame (field) to be de-interlaced and scaled, then the scaling parameters used in the scaling operation are selected to be:

delta_phase = - scaling_ratio NP, and 1 1 initial_phase = NP + - scaling_ratio NP.

It will be appreciated from the above that for even frames, the initial (phase) offset may be negative. Thus, according to a preferred embodiment, the (scaler of the) data processing system is configured to be able to handle negative values of initial (phase) offset. This then facilitates appropriate simultaneous de-interlacing and scaling for even frames (fields). This may be achieved as desired, e.g. by configuring the data processing system to be able to handle a signed representation of the initial (phase) offset (e.g. by configuring the data processing system to be able to handle an additional bit, and using the bit to indicate whether the value of the initial (phase) offset is positive or negative).

The Applicants have furthermore recognised that it can often be necessary to rotate (e.g. by 90°, 180° or 270°) and/or flip (e.g. horizontally and/or vertically) an input data array prior to the data array being provided to a display for display. This is common, for example, in modern mobile devices such as smartphones, tablets, and the like, e.g. where the image displayed on the display is rotated when the mobile device is itself rotated.

Where it is desired to both de-interlace and rotate and/or flip an input data array for display, this may be done by firstly de-interlacing the input data array, and then rotating and/or flipping the de-interlaced data array. However, this will mean that the rotation and/or flip operation (and any other operations performed after the de-interlacing operation) must be performed using the relatively large de-interlaced data array, thereby consuming bandwidth and power of the data processing system.

In a preferred embodiment of the present invention, where it is desired to rotate (e.g. by 90°, 180° or 270°) and/or flip (e.g. horizontally and/or vertically) the input data array (prior to providing the data array to a display for display), this is done prior to the scaling operation, i.e. prior to the scaler scaling the data array so as to produce a de-interlaced and scaled output version of the data array. Accordingly, in a preferred embodiment, a particular data array is rotated and/or

- 13flipped, and the input data array that is provided to the scaler for scaling is a rotated and/or flipped version of the particular data array.

This arrangement means that the rotation operation (and any other operations performed before the de-interlacing operation) may be performed on the relatively small (interlaced) input data array, thereby reducing the bandwidth and power consumption of the data processing system. Furthermore, the rotation and/or flipping operation may be (and preferably is) carried out when reading the data array from memory (i.e. concurrently with the reading operation), and the rotated and/or flipped data array may then be provided to the scaler, which may be downstream of the rotation and/or flipping stage in the processing unit (processor) in question (e.g. in the display controller), i.e. internally to the processing unit.

However, this also means that the scaling operation will be (and in a preferred embodiment is) performed on a rotated and/or flipped version of the input data array to be de-interlaced.

Where, as described above, the data array is to be de-interlaced by scaling the input data array in one direction (dimension), i.e. in either the horizontal or vertical direction, the direction of scaling may be different to the direction of scaling that would otherwise be required for scaling a non-rotated (and/or non-flipped) input data array.

Accordingly, the scaler is in a preferred embodiment capable of scaling a data array in both directions (horizontally and vertically).

Moreover, as discussed above, in order to facilitate appropriate deinterlacing using a scaler, the one or more scaling parameters (e.g. the initial (phase) offset and/or the delta (phase) value) are preferably configurable (programmable), e.g. as described above (so as to address problems with flickering, etc.). Accordingly, in a particularly preferred embodiment, the scaler is programmable in respect of one or more scaling parameters (e.g. and preferably, the initial (phase) offset and/or the delta (phase) value) in both directions (horizontally and vertically). In particular, the scaler is preferably programmable in respect of the initial (phase) offset in both directions (horizontally and vertically).

Furthermore, as described above, the initial (phase) offset defines the distance (e.g. in units of phase) from one end of the input set (row or column) of data positions to (the centre of) the first output data position of the set of output data positions. The Applicants have recognised that where, as described above, the data array is to be de-interlaced by scaling the input data array in one direction

- 14(dimension), i.e. in either the horizontal or vertical direction, the appropriate end (side) of a particular data array that is to be rotated and/or flipped from which the initial (phase) offset should be defined may be different to the end (side) that would otherwise be required, e.g. for a data array that is not to be rotated and/or flipped or that is to be rotated and/or flipped differently.

Thus, in a preferred embodiment, the scaler is configured such that the initial (phase) offset can be defined with respect to plural different ends (sides) of the data array (i.e. the particular data array that is to be rotated and/or flipped), preferably any one of plural different ends (sides) of the data array (i.e. the particular data array that is to be rotated and/or flipped). Thus, for example, the initial (phase) offset can preferably be defined with respect to two or more or each of the top, the bottom, the left and the right side of the data array that is to be rotated and/or flipped, e.g. and preferably depending on how the data array is to be rotated and/or flipped.

Accordingly, the initial (phase) offset may preferably be defined as the distance (e.g. in units of phase) from the top, the bottom, the left and/or the right side of the particular data array that is to be rotated and/or flipped to (the centre of) the first output data position of the set of output data positions.

This then allows the scaler to be able to perform appropriate de-interlacing of rotated (and/or flipped) and non-rotated (and/or non-flipped) input data arrays (e.g. even or odd frames) using the scaler, thereby facilitating more efficient operation of the data processing system when it is desired to both de-interlace and rotate and/or flip a data array for display.

It is believed that the idea of performing the rotation and/or flipping operation prior to the scaling operation where it is desired to both de-interlace and rotate and/or flip an input data array for display is new and advantageous in its own right.

Thus, according to a third aspect of the present invention, there is provided a method of operating a data processing system that includes processing circuitry operable to rotate and/or flip a received input data array to provide a rotated and/or flipped output version of the input data array, and a scaler operable to scale a received input data array to provide a scaled output version of the input data array, the method comprising:

the processing circuitry rotating and/or flipping an input data array so as to produce a rotated and/or flipped version of the input data array;

- 15providing the rotated and/or flipped version of the input data array to the scaler; and the scaler scaling the rotated and/or flipped version of the input data array so as to produce a de-interlaced and rotated and/or flipped output version of the input data array.

According to a fourth aspect of the present invention, there is provided a data processing system, the data processing system comprising:

processing circuitry operable to rotate and/or flip a received input data array to provide a rotated and/or flipped output version of the input data array; and a scaler operable to scale a received input data array to provide a scaled output version of the input data array;

wherein the processing circuitry is configured to rotate and/or flip an input data array so as to produce a rotated and/or flipped version of the input data array;

wherein the data processing system is configured to provide the rotated and/or flipped version of the input data array to the scaler; and wherein the scaler is configured to scale the rotated and/or flipped version of the input data array so as to produce a de-interlaced and rotated and/or flipped output version of the input data array.

As will be appreciated by those skilled in the art, these aspects and embodiments of the present invention can and preferably do include one or more, and preferably all, of the preferred and optional features of the present invention described herein.

Thus, for example, the processing circuitry may be configured to rotate the input data array by 90°, 180° or 270° and/or to flip the input data array horizontally and/or vertically.

The scaler is preferably configured to be able to scale a data array in both directions (horizontally and vertically), e.g. and preferably as described above.

The scaler is preferably programmable in respect of one or more scaling parameters (e.g. and preferably, the initial (phase) offset and/or the delta (phase) value) in both directions (horizontally and vertically), e.g. and preferably as described above. In particular, the scaler is preferably programmable in respect of the initial (phase) offset in both directions (horizontally and vertically), e.g. and preferably as described above.

The scaler is preferably configured such that the initial (phase) offset can be defined with respect to any one of plural different ends (sides) of the input data

- 16array that is to be rotated and/or flipped (i.e. of the original input data array), e.g. and preferably as described above.

The scaler may be configured to scale the rotated and/or flipped version of the input data array so as to produce a de-interlaced, scaled and rotated and/or flipped output version of the input data array, e.g. and preferably as described above.

The scaler and the processing circuitry are preferably part of a given processing unit (processor) of the data processing system, preferably of a display controller. The rotated and/or flipped version of the input data array is preferably provided to the scaler internally to the given processing (processor) unit, e.g. internally to the display controller (i.e. without the processing unit (processor) writing out any intermediate data to memory).

In the various aspect and embodiments described herein, the output data array can be used as desired. It may be provided for further processing in the data processing system, e.g. for further processing by one or more processors of the data processing system. However, in a preferred embodiment, the output data array is provided to a display for display. This may particularly be the case where the scaling operation is being performed in a display controller of the data processing system.

As well as or instead of being output for display, the output data array can otherwise be provided (output) for use as desired. Thus it may, for example, be written out to memory from where it may then be used for further processing. Thus, in a preferred embodiment, the output data array is written to memory of the data processing system (e.g. as well as, or instead of, being displayed).

While it would be possible for the output data array to form the entire output data array, e.g. image, that is output (e.g. displayed) (and in one preferred embodiment this is the case), there could be situations where it is desired to combine the output data array with further data arrays to provide an overall output data array. This could be the case, for example, where the output data array is to be composited with other data arrays (surfaces), e.g. for display. Thus, in a preferred embodiment, the present invention further comprises combining the output data array with one or more further data arrays to provide an overall output data array. In these arrangements, the output data array can be combined with a further data array or arrays in any suitable and desired manner, e.g. according to the desired data array composition process. The further data arrays may

- 17themselves be de-interlaced and/or scaled and/or rotated and/or flipped in the manner of the present invention, if and as desired (or may be not de-interlaced, scaled, rotated and/or flipped at all).

The various stages of the data processing system may be implemented as desired, e.g. in the form of one or more fixed function units (hardware) (i.e. that is dedicated to one or more functions that cannot be changed), and/or as one or more programmable processing stages, e.g. by means of programmable circuitry that can be programmed to perform the desired operation. There may be both fixed function and programmable stages.

One or more of the various stages may be provided as separate circuit elements to one another. Additionally or alternatively, some or all of the stages may be at least partially formed of (provided by) shared circuitry.

It would also be possible for the display controller to comprise, e.g., two (or more) display processing cores, each configured in the manner discussed above, if desired.

As well as including the necessary scaler, processing circuitry, etc., in order to be able to operate in the manner of the present invention, the data processing system of the present invention can otherwise include any one or more or other components that such data processing systems can and/or normally comprise.

Thus, the data processing system may and preferably does also comprise one or more of, and preferably all of: a central processing unit, a graphics processing unit, a video processor (codec), a display controller, a system bus, and a memory controller.

The data processing system (and/or display controller) may be, and preferably is, configured to communicate with one or more of (and the present invention also extends to an arrangement comprising one or more of): an external memory (e.g. via a memory controller), one or more local displays, and/or one or more external displays. The external memory preferably comprises a main memory (e.g. that is shared with the central processing unit (CPU)) of the overall data processing system.

The display may be any suitable and desired display, such as for example, a screen. It may comprise the overall data processing system’s (device’s) local display (screen) and/or an external display. There may be more than one display output, if desired.

- 18Thus, in some embodiments, the display controller and/or data processing system comprises, and/or is in communication with, one or more memories and/or memory devices that store the data described herein, and/or store software for performing the processes described herein. The display controller and/or data processing system may also be in communication with and/or comprise a host microprocessor, and/or with and/or comprise a display for displaying images based on the data generated by the display controller.

Correspondingly, according to a further aspect of the present invention, there is provided a data processing system comprising:

a main memory; a display;

one or more processing units operable to generate input surfaces for display and to store the input surfaces in the main memory;

a display controller, the display controller comprising a scaler operable to scale a received input data array to provide a scaled output version of the input data array; and processing circuitry configured to provide an input data array to be deinterlaced to the scaler;

wherein the scaler is configured to scale the input data array so as to produce a de-interlaced and scaled output version of the input data array.

According to a further aspect of the present invention, there is provided a data processing system comprising:

a main memory; a display;

one or more processing units operable to generate input surfaces for display and to store the input surfaces in the main memory; and a display controller, the display controller comprising rotation processing circuitry operable to rotate and/or flip a received input data array to provide a rotated and/or flipped output version of the input data array and a scaler operable to scale a received input data array to provide a scaled output version of the input data array;

wherein the rotation processing circuitry is configured to rotate and/or flip an input data array so as to produce a rotated and/or flipped version of the input data array;

- 19wherein the data processing system is configured to provide the rotated and/or flipped version of the input data array to the scaler; and wherein the scaler is configured to scale the rotated and/or flipped version of the input data array so as to produce a de-interlaced and rotated and/or flipped output version of the input data array.

The operation in the manner of the present invention is preferably repeated for plural output data arrays to be generated, e.g., and preferably, for a sequence of frames to be displayed.

The present invention can be implemented in any suitable system, such as a suitably configured micro-processor based system. In an embodiment, the present invention is implemented in a computer and/or micro-processor based system.

The various functions of the present invention can be carried out in any desired and suitable manner. For example, the functions of the present invention can be implemented in hardware or software, as desired. Thus, for example, unless otherwise indicated, the various functional elements, stages, and means of the present invention may comprise a suitable processor or processors, controller or controllers, functional units, circuitry, processing logic, microprocessor arrangements, etc., that are operable to perform the various functions, etc., such as appropriately dedicated hardware elements and/or programmable hardware elements that can be programmed to operate in the desired manner.

It should also be noted here that, as will be appreciated by those skilled in the art, the various functions, etc., of the present invention may be duplicated and/or carried out in parallel on a given processor. Equally, the various processing stages may share processing circuitry, etc., if desired.

Furthermore, any one or more or all of the processing stages of the present invention may be embodied as processing stage circuitry, e.g., in the form of one or more fixed-function units (hardware) (processing circuitry), and/or in the form of programmable processing circuitry that can be programmed to perform the desired operation. Equally, any one or more of the processing stages and processing stage circuitry of the present invention may be provided as a separate circuit element to any one or more of the other processing stages or processing stage circuitry, and/or

-20any one or more or all of the processing stages and processing stage circuitry may be at least partially formed of shared processing circuitry.

Subject to any hardware necessary to carry out the specific functions discussed above, the components of the data processing system, such as the display controller, can otherwise include any one or more or all of the usual functional units, etc., that such components include.

It will also be appreciated by those skilled in the art that all of the described embodiments of the present invention can, and in an embodiment do, include, as appropriate, any one or more or all of the features described herein.

The methods in accordance with the present invention may be implemented at least partially using software e.g. computer programs. It will thus be seen that when viewed from further embodiments the present invention provides computer software specifically adapted to carry out the methods herein described when installed on a data processor, a computer program element comprising computer software code portions for performing the methods herein described when the program element is run on a data processor, and a computer program comprising code adapted to perform all the steps of a method or of the methods herein described when the program is run on a data processing system. The data processor may be a microprocessor system, a programmable FPGA (field programmable gate array), etc.

The present invention also extends to a computer software carrier comprising such software which when used to operate a display controller, or microprocessor system comprising a data processor causes in conjunction with said data processor said controller or system to carry out the steps of the methods of the present invention. Such a computer software carrier could be a physical storage medium such as a ROM chip, CD ROM, RAM, flash memory, or disk, or could be a signal such as an electronic signal over wires, an optical signal or a radio signal such as to a satellite or the like.

It will further be appreciated that not all steps of the methods of the present invention need be carried out by computer software and thus from a further broad embodiment the present invention provides computer software and such software installed on a computer software carrier for carrying out at least one of the steps of the methods set out herein.

The present invention may accordingly suitably be embodied as a computer program product for use with a computer system. Such an implementation may

-21 comprise a series of computer readable instructions either fixed on a tangible, non-transitory medium, such as a computer readable medium, for example, diskette, CD-ROM, ROM, RAM, flash memory, or hard disk. It could also comprise a series of computer readable instructions transmittable to a computer system, via a modem or other interface device, over either a tangible medium, including but not limited to optical or analogue communications lines, or intangibly using wireless techniques, including but not limited to microwave, infrared or other transmission techniques. The series of computer readable instructions embodies all or part of the functionality previously described herein.

Those skilled in the art will appreciate that such computer readable instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Further, such instructions may be stored using any memory technology, present or future, including but not limited to, semiconductor, magnetic, or optical, or transmitted using any communications technology, present or future, including but not limited to optical, infrared, or microwave. It is contemplated that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation, for example, shrink-wrapped software, pre-loaded with a computer system, for example, on a system ROM or fixed disk, or distributed from a server or electronic bulletin board over a network, for example, the Internet or World Wide Web.

Various embodiments of the present invention will now be described by way of example only and with reference to the accompanying drawings, in which:

Figure 1 shows schematically a data processing system in accordance with an embodiment of the present invention;

Figure 2 shows schematically a display controller in accordance with an embodiment of the present invention;

Figure 3 shows schematically a display controller in accordance with an embodiment of the present invention;

Figure 4 shows schematically a data processing system in accordance with an embodiment of the present invention;

Figures 5 and 6 illustrate schematically the interlacing technique in accordance with embodiments of the present invention;

Figure 7 shows schematically the process of upscaling a data array using polyphase filtering;

-22Figure 8 shows schematically the process of downscaling a data array using polyphase filtering;

Figure 9 shows schematically the polyphase filtering technique that may be used in embodiments of the present invention;

Figure 10 shows schematically the nearest neighbour scaling technique that may be used in embodiments of the present invention;

Figure 11 shows schematically the process of upscaling a data array using nearest neighbour scaling;

Figure 12 shows schematically the process of downscaling a data array using nearest neighbour scaling;

Figure 13 shows schematically the process of de-interlacing an odd frame using polyphase filtering, where the de-interlaced output frame has an even number of lines;

Figure 14 shows schematically the process of de-interlacing an even frame using polyphase filtering, where the de-interlaced output frame has an even number of lines;

Figure 15 shows schematically the process of de-interlacing an odd frame using polyphase filtering, where the de-interlaced output frame has an odd number of lines;

Figure 16 shows schematically the process of de-interlacing an even frame using polyphase filtering, where the de-interlaced output frame has an odd number of lines;

Figure 17 shows schematically the process of simultaneous de-interlacing and scaling of an odd frame using polyphase filtering in accordance with an embodiment of the present invention;

Figure 18 shows schematically the process of simultaneous de-interlacing and scaling of an even frame using polyphase filtering in accordance with an embodiment of the present invention;

Figure 19 shows schematically all of the possible combinations of rotating a data array by 0°, 90 °, 180 ⁰ or 270 °, and flipping the data array horizontally and/or vertically that may be performed in embodiments of the present invention;

Figure 20 shows schematically a process of de-interlacing and then rotating an input data array;

Figure 21 shows schematically a process of rotating and then de-interlacing an input data array in accordance with an embodiment of the present invention;

-23Figure 22 shows schematically the process of de-interlacing a rotated data array;

Figure 23 shows schematically the process of de-interlacing a rotated and vertically flipped odd frame using polyphase filtering, where the de-interlaced output frame has an even number of lines; and

Figure 24 shows schematically a process for determining the appropriate edge from which an initial phase value should be defined when de-interlacing rotated and/or flipped data arrays in accordance with an embodiment of the present invention.

A preferred embodiment of the present invention will now be described with reference to the Figures.

Figure 1 shows schematically a data processing system in accordance with an embodiment of the present embodiment. The data processing system comprises a central processing unit (CPU) 1, graphics processing unit (GPU) 2, video codec 3, display controller 4 and a memory controller 5. As shown in Figure 1, these communicate via an interconnect 6 and have access to off-chip main memory 7. The CPU 1, and/or the GPU 2, and/or the video codec 3 generate output surfaces (data arrays) and store them, via the memory controller 5, in a frame buffer in the off-chip memory 7. The display controller 4 then reads output surfaces from the frame buffer in the off-chip memory 7 via the memory controller 6 and sends them to a display 8 for display.

Figure 2 shows schematically a display controller 4 in accordance with an embodiment of the present invention. In Figure 2, the rectangles represent functional units of the display controller, while the arrowed lines represent connections between the various functional units.

Figure 2 shows the main elements of the display controller 4 that are relevant to the operation of the present embodiment. As will be appreciated by those skilled in the art there will be other elements of the display controller 4 that are not illustrated in Figure 2. It should also be noted here that Figure 2 is only schematic, and that, for example, in practice the shown functional units and stages may share significant hardware circuits, even though they are shown schematically as separate stages in Figure 2. It will also be appreciated that each of the stages, elements and units, etc., of the display controller 4 as shown in Figure 2 may be implemented as desired and will accordingly comprise, e.g., appropriate circuitry and/or processing logic, etc., for performing the necessary operation and functions.

-24In the present embodiment, the display controller 4 comprises a read controller in the form of a Direct Memory Access (DMA) read controller 10. The read controller 10 is configured to read one or more surfaces from main memory 7 (not shown in Figure 2) via an interface such as an Advance extensible Interface (AXI). The one or more surfaces will typically be in the form of (optionally compressed) RGB data.

Optionally co-located with the read controller 10 is a decoder 11 which, where present, can be used to (selectively) decode (decompress) received compressed surfaces as necessary, before onward transmission of the one or more decoded (decompressed) surfaces. The decoder 11 may comprise an ARM Frame Buffer Compression (AFBC) decoder (AFBC is described in US A1 2013/0034309).

Similarly, rotation processing circuitry 12 can be used to selectively rotate one or more of the input surfaces as necessary before onward transmission of the one or more input surfaces.

In the illustrated embodiment, the read controller 4 is configured to (read) up to three different input surfaces (layers) which are to be used to generate a composited output frame. In this embodiment, the three input layers comprise one video layer, e.g. generated by a video processor (codec), and two graphics layers, e.g. two graphics windows generated by a graphics processing unit (GPU). Hence, Figure 2 shows the displayer controller onwardly transmitting three input surfaces (display layers) via three layer pipelines or channels, namely video channel 13a, a first graphics channel 13b, and a second graphics channel 13c. Any or all of the transmitted input surfaces may have been subjected to decoding (decompression) by decoder 11 and/or rotation by rotation unit 12, as discussed above.

Although the embodiment of Figure 2 illustrates the use of three input surfaces, it will be appreciated that any number of input surfaces (layers) may be used in the present invention, depending on the application in question (and also depending on any silicon area constraints, etc.). Equally, any number of layer pipelines or channels may be provided and used, as desired.

The display controller 4 of the present embodiment optionally comprises a multiplexer/data-flow control 14. Where present, the display controller may be configured such that multiplexer 14 receives inputs from any one or more (or all) of the input surface channels. The multiplexer 14 may operate to selectively transmit any one or more (or all) of the received inputs (i.e. surfaces) to any one or more of the multiplexer’s 14 outputs.

-25The display controller 4 of the present embodiment optionally comprises a composition unit 15. Where present, the display controller 4 may be configured such that the composition unit 15 receives inputs directly from any one or more or all of the channels 13, and/or from the multiplexer 14. The composition unit 15 may operate to compose the received input surfaces to generate a composited output frame, i.e. by appropriate blending operations, etc. In the illustrated embodiment, the composited output frame may be onwardly transmitted by the composition unit 15 to multiplexer 14, and/or to post-processing pipeline 16.

The post-processing pipeline 16 is configured to selectively carry out any desired processing operation(s) on the (optionally composited) output surface (frame). The post-processing pipeline 16 may, for example, comprise a colour conversion stage operable to apply a colour conversion to the (optionally composited) output frame, a dithering stage operable to apply dithering to the (optionally composited) output frame, and/or a gamma correction stage operable to carry out gamma correction on the (optionally composited) output frame.

In the present embodiment, the post-processing pipeline 16 is configured to transmit the (processed) output frame to an output stage comprising a display timing unit 17 for appropriate display on a (local) display 8 (not shown in Figure 2). The display timing unit 17 is configured to send pixel data to the display with appropriate horizontal and vertical blanking periods.

The display controller 4 of the present embodiment comprises a scaling engine 18. The scaling engine 18 operates to (selectively) scale (i.e. upscale or downscale) any one or more received surfaces (frames) to generate a scaled surface (frame). The operation of the scaler 18 will be described in more detail below.

In the present embodiment, the display controller optionally comprises a write controller 19, e.g. in the form of a DMA write controller. Where present, the write controller 19 may be configured to write out received surfaces (frames) to external memory 8 (e.g. frame buffer), e.g. via AXI.

Thus, this embodiment of the present invention comprises a display controller that integrates a rotation unit 12 and a scaler 18, optionally together with a composition unit 15. The rotation unit 12 and the scaler 18 are embedded within the display controller, such that surfaces read by the display controller 4 may be rotated and/or scaled (and then optionally further processed, e.g. composited)

-26before being displayed, with only a single read (of each input surface) from the frame buffer being required.

Figure 3 shows schematically a display controller 4 in accordance with an alternative embodiment of the present invention. The display controller 4 of Figure 3 is similar to the display controller 4 of Figure 2, except that multiple scalers 18 are provided in the display controller. In particular, a scaler 18a-18c is provided in each of the layer pipelines 13a-13c. A scaler 18d may also be provided in the postprocessing pipeline 16.

More generally, one or more scalers 18 may be provided in any one or more or each of the data processing units 1, 2, 3, 4 of the data processing system of Figure 1. This is illustrated schematically by Figure 4, where scalers 18 are provided in each of the processing pipelines of two separate data processing units (which may be e.g. a central processing unit (CPU) 1, a graphics processing unit (GPU) 2, a video codec 3, or a display controller 4) of the data processing system.

In the present embodiment, interlaced video data, comprising a sequence of interleaved even and odd (video) frames (fields), is generated by the video codec 3 (video processor). The interlaced video data could also be generated by the GPU 2, or by a digital camera image signal processor (ISP), or other image processor.

Each of the even and an odd video frames (fields) may be stored in a respective frame buffer in memory 7.

As illustrated by Figure 5, each video frame 20 (e.g. decoded by the video codec 3, generated by the GPU 3 or generated by an ISP) is effectively split into two separate interlaced video frames (or “fields”), namely an odd frame (field) 21 and an even frame (field) 22. Each even frame 22 comprises each of the even lines of the corresponding original video frame 20, and each odd frame 21 comprises each of the odd lines of the corresponding original video frame 20.

Each video frame is propagated through the data processing system in the form of these two (even and odd) frames. This is illustrated by Figure 6. A stream of original video frames 20a-20d is effectively converted into a stream of corresponding interleaved odd 21a-21d and even 22a-22d frames.

When it is desired to display the video content, each of the odd 21 and even 22 frames must be separately de-interlaced, and then provided to the display 8 for display. This has the effect of doubling the frame rate, without using additional bandwidth, relative to the rate at which the VPU 3 generates frames.

-27In the present embodiment, odd 21 and even 22 frames for display are also scaled (upscaled or downscaled) before being provided to the display 8 for display.

This is done, in accordance with the present invention, by simultaneously de-interlacing and scaling the interlaced video data for display using the scaler 18 of the display controller 4.

In the present embodiment, the scaler 18 is configured to scale (upscale or downscale) a received input frame to provide a scaled output version of the input frame using polyphase filtering. It would, of course, be possible to use other scaling techniques.

Each input frame 21, 22 to be scaled will comprise an array of data positions, with each data position taking a particular data value. Accordingly, each frame comprises plural columns of data positions and plural row or lines of data positions. In the present embodiment, the input frame is an image for display, and so each data value of the array comprises a colour value. However, it would be possible for each data value to represent some other parameter.

In the present embodiment, the scaler 18 is operable to scale each input frame in the vertical direction by separately scaling each column of data positions. That is, for each column of Λ/ data positions of the frame, a scaled version of that column of Λ/ data positions is generated, which will comprise some other number M of data positions.

As shown in Figures 7 and 8, an output column 31 of M data positions is generated from an input column 30 of N data positions. Figure 7 illustrates an example of upscaling (where M>N), and Figure 8 illustrates an example of downscaling (where M<N). In particular, Figure 7 illustrates upscaling from 10 to 16 pixels, whereas Figure 8 illustrates downscaling from 10 to 4 pixels.

The scaler 18 is configured to use some fixed number of filter taps and some fixed number of phases. In particular, Figures 5 and 6 illustrated examples of scaling using a 6-tap, 16-phase polyphase filter. However the techniques described herein can be straightforwardly extended to any number of filter taps and phases.

The number of filter taps defines how many of the input data values are used to generate each output data value. For each output data position, the data values of the nearest input data positions are used in generating the output data value. This is illustrated by Figures 7 and 8, where the data values of the data positions indicated by each of the numbers that each vertical arrow crosses are used to generate each output data value. Where data positions falling outside of

-28the range of input data positions are indicated, the data value of the nearest data position may be used instead.

These data values are combined using a weighting that is selected depending on a phase value of each output data position.

A phase value is defined for each output data position, with the phase defining how aligned the output data position in question is with the set of input data positions 30. As shown in Figures 7 and 8, the phase scale is defined in relation to the positions of the set of input data positions 30. The phase is zero at the centre of each of the input data positions, and varies linearly up to the maximum phase value between adjacent input data positions.

Thus, an output data position would have a phase value of zero where it is aligned with one of the plural input data positions. Output data positions that are not aligned with an input data position will have some non-zero value of phase.

By using the phase value to select the weighting (set of coefficients) that is used to combine the data values of plural input data positions to produce the data value for an output data position, the scaler 18 can produce an improved output, e.g. with reduced aliasing artefacts and the like.

In the present embodiment, since both the column of input data positions 30 and the column of output data positions 31 has a regular periodic form, then the set of phase values for a given column of output data positions can be characterised by two scaling parameters, namely an initial phase offset together with a delta phase value.

As shown in Figures 7 and 8, the initial phase offset defines the distance, in units of phase, from one end of the input column of data positions 30 to the centre of the first output data position. The delta phase defines the distance, in units of phase, between the centres of adjacent output data positions.

The scaler 18 is programmable in respect of the initial phase offset and the delta phase, and the scaling operation can be controlled by appropriately programming the scaler 18 in respect of these scaling parameters.

Figure 9 illustrates polyphase filtering, where the symbol Zi_m,„ denotes the coefficient for the mth tap and nth phase.

Polyphase filtering makes use of fast impulse response (FIR) filters with impulse responses divided into phases. The technique allows filtering with an accuracy equal to NP/2 of an input sample (where NP is the number of filter phases), and can give high quality results even with a small number of filter taps.

-29In order to compute each output sample, the phase of the scaling filter corresponding to the output sample must be computed (to select a proper set of scaling coefficients), together with the indexes of input samples required to perform the filtering. These parameters are computed according to Equations 1 and 2 below:

, z . _r, /initial phase+n-delta phase , , ,,, indexfii) = floor (^---—----h 0.5 \ - 1, (1) phase (η) = floor (mod (initial_phase + n delta_phase + (?) delta phase = ^slze-ⁱⁿ j\jp (3) size_out where n is the index of the output sample, index(ri) is the index of the “middle input sample, phase(ri) is the filter phase for output sample n, initial_phase is the initial phase offset, delta_phase is the phase increase corresponding to one output sample (as defined by Equation 3), size_in is the input size of the scaled image, size_out is the output size of the scaled image, NP is the number of filter phases, floor(a) indicates that a should be rounded down to the nearest integer (i.e. towards minus infinity), and mod(a, b) indicates the modulus after division of a by b.

In order to compute one output sample, NT input samples are needed (where NT represents the number of filter taps). Equation 1 calculates the index of a sample from a “middle” tap of the scaling filter. The middle tap is tap floor(NT/2) of the filter, where floor(a) denotes rounding of a to the nearest integer towards minus infinity. This means that the following input samples are filtered during computation of the nth output sample:

index(n) + -γ, index(n) + ...,index(n) -^-+1, if NT is an even number; and index(n) + ¹, index(n) + -y— — 1, ...,index(n) — if NT is an odd number.

The value of the initial phase offset determines the phase value used to be used to compute the first output pixel.

-30Although the present embodiment is described in terms of the scaler 18 scaling a received input frame using polyphase filtering, it would also be possible for the scaler 18 to use other types of interpolation for the scaling operation, such as nearest neighbour interpolation, bilinear interpolation, bicubic interpolation, etc.

Figure 10 illustrates the nearest neighbour technique. The nearest neighbour algorithm involves replicating or removing input samples in order to produce an output data array of the desired size.

Figure 10 illustrates an example of scaling from a 3x3 data array to a 4x6 data array using the nearest neighbour technique. As shown in Figure 10, each of the data values AA-CC of the original input data array is replicated multiple times in the scaled output version of the input data array.

Figures 11 and 12 illustrate the nearest neighbour technique in terms of separately scaling each column of data positions (i.e. in a corresponding manner to that described above). Figure 11 illustrates an example of upscaling from 5 to 8 pixels, and Figure 12 illustrates an example of downscaling from 10 to 4 pixels.

For each output data position of the output column of data positions 31, it is determined which input data position of the input column of data positions 30 should be used (replicated) to produce the data value for that output data position.

As in the polyphase method described above, this process can be characterised by two scaling parameters. The first scaling parameter determines which input data position should be used in respect of the first output data position. This parameter is equivalent to the “initial phase” of polyphase filtering described above. The second scaling parameter determines which input data positions should be used in respect of each of the other output data positions, and has the form of a distance equal to the input resolution divided by the output resolution.

This distance (“delta” in Figures 11 and 12) is equivalent to the “delta phase” in polyphase filtering described above.

Using the naming convention of polyphase filtering, the nearest neighbour technique can be described using Equations 4 and 5 below:

out(n) = in (floor (tnUia^Jrase^aeu^^se + O.s) - l), (4) delta phase = ^slze-ⁱⁿ j\jp size_out (5)

- 31 where n is the index of output data position, out(n) is the value of the nth output data position, initial_phase is the initial phase offset, delta_phase is the phase increase corresponding to one output data position (defined by Equation 5), NP is the number of filter phases, and floor(a) denotes rounding of a to the nearest integer towards minus infinity.

Accordingly, when using nearest neighbour scaling, the “initial phase” can be understood as the distance from one side (e.g. the left side) of the input set of data positions to the place in the input set of data positions from which the data value for the first output data position is taken, the “delta phase” can be understood as the number of input data positions corresponding to one output data position, and “NP’ can be understood as the precision at which the value of each output data position is selected from the input data positions.

The techniques of the present invention and embodiments can correspondingly be applied to other scaling techniques.

In the present embodiment, simultaneous de-interlacing and scaling in the manner of the present invention is achieved by appropriate programming of the scaling parameters.

It should be noted in this regard, that when performing de-interlacing, simply upscaling the input array by a factor of two would not result in an appropriately deinterlaced frame. This is because a sequence of interleaved even frames 22 (fields) comprising only the even lies of an original frame 20 and odd frames 21 (fields) comprising only the odd lies of the original frame 20 must be de-interlaced, so that simply upscaling each frame in turn would result in flickering of the displayed image, whereby the displayed image would appear to jump up and down from frame to frame. Equally, where the original frame 20 comprises an odd number of lines, the resulting even 22 and odd 21 frames (fields) will comprise different numbers of lines, and so will require different levels of scaling to produce equally sized (de-interlaced) frames for display.

However, it is possible to achieve de-interlacing by appropriately setting the initial phase and the delta phase in dependence on whether the input frame is an even frame 22 or an odd frame 21, and in dependence on whether the original frame 20 from which the even or odd frame in question is derived had an even or odd number of lines.

- 32 As illustrated by Figure 13, where the original frame 20 had an even number of lines, for an odd frame 21, appropriate de-interlacing can be achieved by setting the delta phase and initial phase as follows:

delta_phase = j NP, and initial_phase =

Note that in this case, this is the same value as defined by Equation 3, i.e. as for normal upscaling.

As illustrated by Figure 14, where the original frame 20 had an even number of lines, for an even frame 22, appropriate de-interlacing can be achieved by setting the delta phase and initial phase as follows:

delta_phase = j NP, and initial_phase = 0.

Note that in this case, this is the same value as is defined by Equation 3, i.e. as for normal upscaling.

As illustrated by Figure 15, where the original frame 20 has an odd number of lines, for an odd frame 21, appropriate de-interlacing can be achieved by setting the delta phase and initial phase as follows:

delta_phase = j NP, and initial_phase =

Note that in this case, this is not the same value as defined by Equation 3 (as for normal upscaling), but the value is instead for upscaling by two times (not for upscaling from (n) to (2n-1)).

As illustrated by Figure 16, where the original frame 20 had an odd number of lines, for an even frame 22, appropriate de-interlacing can be achieved by setting the delta phase and initial phase as follows:

delta_phase = j NP, and

-33initial_phase = 0.

Note that in this case, this is not the same value as is defined by Equation 3 (as for normal upscaling), but the value is for upscaling by two rimes (not for upscaling from (n) to (2n+1)).

In the present embodiment, the scaler is operable to perform simultaneous de-interlacing and scaling.

As illustrated by Figure 17, where the input frame comprises an odd frame 21 (field) to be de-interlaced and scaled, then the scaling parameters used in the scaling operation are selected to be:

delta_phase = - scaling_ratio NP, and 1 1 initial_phase = - NP + - scaling_ratio NP, where NP is the number of filter phases of the scaler, and scaling_ratio is the scaling ratio between the de-interlaced and the output frame.

As illustrated by Figure 18, where the input data array comprises an even frame 22 (field) to be de-interlaced and scaled, then the scaling parameters used in the scaling operation are selected to be:

In the present embodiment, the mode of operation of the display controller 4 is controlled by an application, e.g. running on the CPU 1, by the application generating instructions which are interpreted by a driver for the display controller 4 (that is running on the CPU 1) to generate appropriate commands to the display controller 4 to operate as required by the application. In particular, where the application requires an input frame to de-interlaced and scaled, then the application generates appropriate instructions which are interpreted by the driver to generate appropriate commands to the display controller 4 to operate in the manner of the present embodiment, i.e. to perform simultaneous de-interlacing and scaling using the scaler 18. The driver determines the appropriate values of the scaling

-34parameters (the delta phase and the initial phase), and sends appropriate commands to the display controller 4 to operate the scaler 18 using the determined scaling parameters.

It will be appreciated from the above that for even frames, the initial phase offset may be negative. Therefore the scaler 18 is configured to be able to handle negative values of initial phase offset. This then facilitates appropriate simultaneous de-interlacing and scaling for even frames (fields).

The above preferred embodiment has been described in terms of simultaneous de-interlacing and scaling of an input data array. It may additionally or alternatively be desired to perform rotation and/or flipping of the input frame prior to display. This is common, for example, in modern mobile devices such as smartphones, tablets, and the like, e.g. where the image displayed on the display is rotated when the mobile device is itself rotated.

Figure 19 illustrates all the possible combination of rotation and flipping that it may be desired to implement. As shown in Figure 19, it may be desired to rotate a given frame for display by 0°, 90°, 180°, or 270°. It may additionally or alternatively be desired to flip a frame for display vertical and/or horizontally.

As illustrated by Figure 20, where it is desired to both de-interlace and rotate and/or flip an input data array for display, this is typically done by firstly deinterlacing the input data array using a de-interlacer 40, and then rotating and/or flipping the de-interlaced data array using rotation circuitry 12. The operations are performed in this order because a standard de-interlacer 40 cannot perform deinterlacing on rotated content.

However, performing the operations in this order would mean that where upscaling is to be performed, the upscaling operation would be performed at the beginning of the processing pass, thereby requiring that more data positions are processed by some of the downstream processing blocks (e.g. by the rotator 12).

In addition, many standard processing operations operate on a line by line basis (i.e. process each received data array line by line), and in some cases may not therefore be performed between de-interlacing by the de-interlacer 40 and rotation by the rotator 12.

As illustrated by Figure 21, in the present embodiment, where it is desired to rotate (e.g. by 90°, 180° or 270°) and/or flip (e.g. horizontally and/or vertically) the input data array (prior to providing the data array to a display for display), this is

-35done prior to the scaling operation, i.e. prior to the scaler scaling the frame so as to produce a de-interlaced and scaled output version of the frame.

This arrangement can reduce the overall processing that must be performed by the display controller 4, e.g. where upscaling is to be performed, and can allow standard processing operations that operate on a line by line basis to be performed between rotation and de-interlacing. Furthermore, the rotation and/or flipping may be carried out when reading the data array from memory by rotation circuitry 12, and the rotated and/or flipped data array may then be provided to the scaler 18, which may be downstream of the rotation and/or flipping stage 12, internally to the display controller 4.

However, this also means that the scaling operation will be performed on a rotated and/or flipped version of the input frame to be de-interlaced.

This is illustrated by Figure 22. As show in Figure 22, where is desired to rotate a frame for display, the direction in which the frame must be de-interlaced may be different to the direction of the display lines of the output display panel 8. Thus, the interlaced frames must be de-interlaced by interpolation in the vertical direction of the display device or by interpolation in the horizontal direction of the display device.

Therefore, in the present embodiment, the scaler 18 is configured so as to be able to perform scaling in both the vertical and horizontal directions. In particular, the scaler 18 is configured so that the initial phase offset and the delta phase are configurable (in the manner descried above) for both the horizontal and vertical directions. This then allows the scaler 18 to perform de-interlacing for any type of rotated and/or flipped input frames and allows more efficient operation of the system.

Furthermore, where the frame is to be de-interlaced by horizontally or vertically scaling the input frame in one direction (dimension), the appropriate side (end) of the input frame (i.e. the original content) that is to be rotated and/or flipped from which the initial phase offset should be defined may be different to the side (end) that would otherwise be required, e.g. for an input frame that is not to be rotated and/or flipped or which is to be rotated and/or flipped differently.

Accordingly, the scaler 18 is configured such that the initial phase offset can be defined with respect to any one of the different sides of the input frame. Thus, for example, the initial phase offset can be defined with respect to the top, the

-36bottom, the left and the right side of the input frame depending on how the input frame is to be rotated and/or flipped.

This allows the scaler 18 to be able to perform appropriate de-interlacing of rotated (and/or flipped) and non-rotated (and/or non-flipped) even or odd frames, thereby facilitating more efficient operation of the display controller when it is desired to both de-interlace and rotate and/or flip a frame for display.

Figure 23 illustrates the process of de-interlacing a rotated and vertically flipped odd interlaced video frame, where the original frame 20 had an even horizontal size. Figure 23 is based on 16-phase polyphase filter, but can be straightforwardly extended to any number of phases.

As illustrated by Figure 23, in this case the scaling parameters used in the scaling operation are selected to be:

delta_phase = j NP, and 15 initial_phase = ^—

Note that in this case, this is the same value as is defined by Equation 3 (as for normal upscaling). It can also be seen that the initial phase is negative, and should be computed for the last line, since the data positions from the last line will processed first in the scaler 18.

Table 1 shows the appropriate edge from which the initial phase should be defined in respect of each of the possible combinations of rotation and/or flipping.

Rotation and/or flipping	De-interlacing direction (scaling direction)	Direction for which phase offset is required	Edge from which phase offset is calculated
(Natural orientation) or (vertical flip + horizontal flip + rotation 180°)	vertical	vertical	top
(Horizontal flip) or (vertical flip + rotation 180°)	vertical	vertical	top
(Vertical flip) or (horizontal	vertical	vertical	bottom

flip + rotation 180°)
(Vertical flip + horizontal flip) or (rotation 180°)	vertical	vertical	bottom
(Rotation 90°) or (vertical flip + horizontal flip + rotation 270°)	horizontal	horizontal	left
(Rotation 90° + horizontal flip) or (vertical flip + rotation 270°)	horizontal	horizontal	left
(Rotation 90° + vertical flip) or (horizontal flip + rotation 270°)	horizontal	horizontal	right
(Rotation 90° + vertical flip + horizontal flip) or (rotation 270°)	horizontal	horizontal	right

Table 1

Figure 24 illustrates a process for determining the appropriate edge from 5 which the initial phase should be defined when de-interlacing rotated and/or flipped data arrays in accordance with the present embodiment. As shown in Figure 24, it is firstly determined whether or not the input data array has been rotated by 90⁰ or 270° (step 50). In the present embodiment, where the input data array has been rotated by 90⁰ or 270°, de-interlacing must be performed in the horizontal direction, whereas where the input data array is not rotated by 90 ⁰ or 270° (e.g. is not rotated, or is rotated by 180 °, then de-interlacing must be performed in the vertical direction.

Where de-interlacing is to be performed in the vertical direction, it is then determined whether the input data array has been vertically flipped or has been rotated by 180⁰ (step 51). If this is the case, then the initial phase offset to be used

-38for de-interlacing is computed in the vertical direction with respect to the last line of the input data array. If this is not the case, then the initial phase offset to be used for de-interlacing is computed in the vertical direction with respect to the first line of the input data array.

Where de-interlacing is to be performed in the horizontal direction, it is then determined whether the input data array has been vertically flipped or has been rotated by 270° (step 52). If this is the case, then the initial phase offset to be used for de-interlacing is computed in the horizontal direction with respect to the last line of the input data array. If this is not the case, then the initial phase offset to be used for de-interlacing is computed in the horizontal direction with respect to the first line of the input data array.

It will be appreciated from the above that the present embodiment provides an optimisation of an image scaler 18 that facilitates full support of video deinterlacing, and in particular simultaneous de-interlacing and scaling of the de15 interlaced video in a single pass, using the same scaler, and support for deinterlacing of rotated and flipped layers in the image scaler 18.

Claims

Claims

1. A method of operating a data processing system that includes a scaler operable to scale a received input data array to provide a scaled output version of the input data array, the method comprising:

when it is desired to produce a de-interlaced and scaled output version of an input data array:

providing the input data array to the scaler; and the scaler scaling the input data array so as to simultaneously de-interlace and scale the input data array and to thereby produce a de-interlaced and scaled output version of the input data array.
2. The method of claim 1, wherein the input data array comprises an even or an odd frame to be de-interlaced for display.
3. The method of any claim 1 or 2, wherein the method comprises: configuring the scaler with one or more scaling parameters such that when the scaler scales the input data array, it will produce a de-interlaced and scaled output version of the input data array; and wherein the step of the scaler scaling the input data array comprises the scaler scaling the input data array using the one or more scaling parameters.
4. The method of claim 3, further comprising determining one or more of the one or more scaling parameters based on the ratio of the size of the output data array to the size of a de-interlaced version of the input data array.
5. The method of claim 3 or 4, wherein:

the one or more scaling parameters include an initial offset; and the scaler is operable to use negative values of initial offset.
6. The method of any one of the preceding claims, wherein the input data array that is provided to the scaler comprises a rotated and/or flipped version of a data array.

-407. A method of operating a data processing system that includes processing circuitry operable to rotate and/or flip a received input data array to provide a rotated and/or flipped output version of the input data array, and a scaler operable to scale a received input data array to provide a scaled output version of the input data array, the method comprising:

the processing circuitry rotating and/or flipping an input data array so as to produce a rotated and/or flipped version of the input data array;

providing the rotated and/or flipped version of the input data array to the scaler; and the scaler scaling the rotated and/or flipped version of the input data array so as to produce a de-interlaced and rotated and/or flipped output version of the input data array.

8. The method of any one of the preceding claims, wherein:

the scaler is operable to scale a received input data array horizontally; and the scaler is operable to scale a received input data array vertically.

9. The method of any one of the preceding claims, wherein: the scaler is operable to use a horizontal initial offset; and the scaler is operable to use a vertical initial offset.

10. The method of any one of the preceding claims, wherein:

the scaler is operable to use an initial offset defined with respect to one side of a data array; and the scaler is operable to use an initial offset defined with respect to a different side of a data array.

11. A data processing system, the data processing system comprising:

a scaler operable to scale a received input data array to provide a scaled output version of the input data array;

wherein the data processing system is configured, when it is desired to produce a de-interlaced and scaled output version of an input data array, to provide the input data array to the scaler; and

-41 wherein the scaler is operable to scale a received input data array so as to simultaneously de-interlace and scale the input data array, to thereby produce a deinterlaced and scaled output version of the input data array.

12. The data processing system of claim 11, wherein the input data array comprises an even or an odd frame to be de-interlaced for display.

13. The data processing system of claim 11 or 12, wherein:

the data processing system is operable to configure the scaler with one or more scaling parameters such that when the scaler scales an input data array, it will produce a de-interlaced and scaled output version of the input data array; and the scaler is configured to scale an input data array using the one or more scaling parameters.

14. The data processing system of claim 13, wherein the data processing systems is configured to determine one or more of the one or more scaling parameters based on the ratio of the size of the output data array to the size of a de-interlaced version of the input data array.

15. The data processing system of claim 13 or 14, wherein:

the one or more scaling parameters include an initial offset; and the scaler is operable to use negative values of initial offset.

16. The data processing system of any one of claims 11-15, wherein the input data array comprises a rotated and/or flipped version of a data array.

17. A data processing system, the data processing system comprising: processing circuitry operable to rotate and/or flip a received input data array to provide a rotated and/or flipped output version of the input data array; and a scaler operable to scale a received input data array to provide a scaled output version of the input data array;

wherein the processing circuitry is configured to rotate and/or flip an input data array so as to produce a rotated and/or flipped version of the input data array; wherein the data processing system is configured to provide the rotated and/or flipped version of the input data array to the scaler; and

-42wherein the scaler is configured to scale the rotated and/or flipped version of the input data array so as to produce a de-interlaced and rotated and/or flipped output version of the input data array.

5 18. The data processing system of any one of claims 11-17, wherein:

the scaler is operable to scale a received input data array horizontally ; and the scaler is operable to scale a received input data array vertically.

19. The data processing system of any one of claims 11-18, wherein:

10 the scaler is operable to use a horizontal initial offset; and the scaler is operable to use a vertical initial offset.

20. The data processing system of claims 11-16, wherein:

the scaler is operable to use an initial offset defined with respect to one side 15 of a data array; and the scaler is operable to use an initial offset defined with respect to a different side of a data array.

21. A computer program comprising computer software code for performing the 20 method of any one of claims 1 to 10 when the program is run on data processing means.

AMENDMENTS TO CLAIMS ARE FILED AS FOLLOWS

08 09 17

Claims

1. A method of operating a data processing system that includes a scaler operable to scale a received input data array to provide a scaled output version of

5 the input data array, the method comprising:

when it is desired to produce a de-interlaced and scaled output version of an input image data array:

providing the input image data array to the scaler; and the scaler scaling the input image data array so as to simultaneously de10 interlace and scale the input image data array and to thereby produce a deinterlaced and scaled output version of the input image data array.

2. The method of claim 1, wherein the input data array comprises an even or an odd frame to be de-interlaced for display.

3. The method of any claim 1 or 2, wherein the method comprises: configuring the scaler with one or more scaling parameters such that when the scaler scales the input data array, it will produce a de-interlaced and scaled output version of the input data array; and

20 wherein the step of the scaler scaling the input data array comprises the scaler scaling the input data array using the one or more scaling parameters.

4. The method of claim 3, further comprising determining one or more of the one or more scaling parameters based on the ratio of the size of the output data

25 array to the size of a de-interlaced version of the input data array.

5. The method of claim 3 or 4, wherein:

the one or more scaling parameters include an initial offset; and the scaler is operable to use negative values of initial offset.

6. The method of any one of the preceding claims, wherein the input data array that is provided to the scaler comprises a rotated and/or flipped version of a data array.

08 09 17
7. A method of operating a data processing system that includes processing circuitry operable to rotate and/or flip a received input data array to provide a rotated and/or flipped output version of the input data array, and a scaler operable to scale a received input data array to provide a scaled output version of the input

5 data array, the method comprising:

the processing circuitry rotating and/or flipping an input image data array so as to produce a rotated and/or flipped version of the input image data array;

providing the rotated and/or flipped version of the input image data array to the scaler; and

10 the scaler scaling the rotated and/or flipped version of the input image data array so as to produce a de-interlaced and rotated and/or flipped output version of the input image data array.
8. The method of any one of the preceding claims, wherein:

15 the scaler is operable to scale a received input data array horizontally; and the scaler is operable to scale a received input data array vertically.
9. The method of any one of the preceding claims, wherein: the scaler is operable to use a horizontal initial offset; and

20 the scaler is operable to use a vertical initial offset.
10. The method of any one of the preceding claims, wherein:

the scaler is operable to use an initial offset defined with respect to one side of a data array; and

25 the scaler is operable to use an initial offset defined with respect to a different side of a data array.
11. A data processing system, the data processing system comprising:

a scaler operable to scale a received input data array to provide a scaled

30 output version of the input data array;

wherein the data processing system is configured, when it is desired to produce a de-interlaced and scaled output version of an input image data array, to provide the input image data array to the scaler; and

08 09 17 wherein the scaler is operable to scale a received input image data array so as to simultaneously de-interlace and scale the input image data array, to thereby produce a de-interlaced and scaled output version of the input image data array.

5
12. The data processing system of claim 11, wherein the input data array comprises an even or an odd frame to be de-interlaced for display.
13. The data processing system of claim 11 or 12, wherein:

the data processing system is operable to configure the scaler with one or

10 more scaling parameters such that when the scaler scales an input data array, it will produce a de-interlaced and scaled output version of the input data array; and the scaler is configured to scale an input data array using the one or more scaling parameters.
15 14. The data processing system of claim 13, wherein the data processing systems is configured to determine one or more of the one or more scaling parameters based on the ratio of the size of the output data array to the size of a de-interlaced version of the input data array.

20 15. The data processing system of claim 13 or 14, wherein:

the one or more scaling parameters include an initial offset; and the scaler is operable to use negative values of initial offset.
16. The data processing system of any one of claims 11-15, wherein the input

25 data array comprises a rotated and/or flipped version of a data array.
17. A data processing system, the data processing system comprising: processing circuitry operable to rotate and/or flip a received input data array to provide a rotated and/or flipped output version of the input data array; and

30 a scaler operable to scale a received input data array to provide a scaled output version of the input data array;

wherein the processing circuitry is configured to rotate and/or flip an input image data array so as to produce a rotated and/or flipped version of the input image data array;

08 09 17 wherein the data processing system is configured to provide the rotated and/or flipped version of the input image data array to the scaler; and wherein the scaler is configured to scale the rotated and/or flipped version of the input image data array so as to produce a de-interlaced and rotated and/or

5 flipped output version of the input image data array.
18. The data processing system of any one of claims 11-17, wherein:

the scaler is operable to scale a received input data array horizontally ; and the scaler is operable to scale a received input data array vertically.
19. The data processing system of any one of claims 11-18, wherein: the scaler is operable to use a horizontal initial offset; and the scaler is operable to use a vertical initial offset.

15
20. The data processing system of claims 11-16, wherein:

the scaler is operable to use an initial offset defined with respect to one side of a data array; and the scaler is operable to use an initial offset defined with respect to a different side of a data array.
21. A computer program comprising computer software code for performing the method of any one of claims 1 to 10 when the program is run on data processing means.

Intellectual

Property

Office

Application No: GB1619708.9 Examiner: Mr Iwan Thomas