GB2352601A

GB2352601A - Graphical data-processing

Info

Publication number: GB2352601A
Application number: GB9917527A
Authority: GB
Inventors: Danny Halamish
Original assignee: Pixelfusion Ltd
Current assignee: Pixelfusion Ltd
Priority date: 1999-07-26
Filing date: 1999-07-26
Publication date: 2001-01-31
Also published as: GB9917527D0; AU6171500A; WO2001008103A1

Abstract

A method of processing image data blocks representing image surfaces for display on a display device. Respective image data blocks are combined to form a compound image data block and a corresponding compound control data block. The compound image data block is processed in accordance with the compound control data block.

Description

2352601 GRAPHICAL DATA-PROCESSING The present invention relates to

graphical data processing, and in particular to methods and apparatus for displaying graphical overlays on display devices.

BACKGROUND OF THE INVENTION

Computer graphics images are conventionally displayed on a display device, such as a CRT monitor, by generating data for the image to be displayed and storing that data in a memory area, known a's the "frame buffer". The data stored in the frame buffer is then transferred to the display device. In the case of a CRT monitor this is performed by an electron beam scanning across the display screen. Updating the image displayed on the device is achieved by updating the is frame buffer datal and then scanning the new data onto the display device. A display screen can be scanned at any appropriate rate, for example the scan refresh rate of the display could be 60Hz, 80Hz or 120Hz. Other refresh rates are, of course, possible. Each complete scan of the device is known as a "framelf.

A computer system generates the pixel data to be displayed on the display device and transfers that data to a portion of the system memory known as the "frame buffer". Primary surface data relates to the background image or main image to be displayed. other image data, known as overlay or secondary surfaces, are stored in other parts of the frame buffer from the primary surface data. For example, the operating system generates a "desktop" corresponding to the visible image at each pixel of the device, be that a background or the contents of a window. This desktop is the primary, and sometimes only, surface to be displayed.

Overlay surfaces, sometimes referred to simply as overlays, are surfaces with special hardware- supported capabilities. overlay surfaces are frequently used to display live video, recorded video, or still bitmaps, such as produced by digitizers, decoders, 3D renderers, etc. overlay surfaces are intended for display over a primary surface without changing the contents of the primary surface data in any way.

Figures 1 and 2 illustrate display of an overlay surface 2 and a primary surface 1. An overlay surface 2 is analogous to a clear sheet on which an image is drawn. When the sheet is placed in front of the monitor, it is possible to see the image drawn on the sheet, and those parts of the primary surface 1 which show through the clear areas of the sheet. When the sheet is removed, the primary surface 1 is unchanged.

In f act, the actual display of an overlay surface 2 is also analogous to holding a clear sheet in front of the display. When an overlay surface is to be displayed, the display device receives primary surface information and overlay surface information. The device determines in real time, for each dot of the display, which of the primary surface 1 and overlay surface 2 is visible. If the overlay surface 2 is visible, then the overlay surface data is displayed and if the primary surface is visible then the primary surface data is displayed.

From the viewpoint of the operating system, the primary surface is unchanged by the overlay surface, since it is only the display device which is updated by the overlay surface data. The primary surface data generated by the graphics system remains unchanged.

The display device determines whether the primary surface or the overlay surface is to be displayed.

Using this method, the display device shows a composite of the primary surface and the overlay surf ace, and can provide transparency and stretching effects, without modifying the contents of either surface.

In order to determine which parts of an overlay surf ace are to be displayed, and on which pixels of the device this display is to occur, source and destination areas and colour keys are used. The source and destination areas are illustrated in Figures 3 and 4.

The source area 4 preferably describes a rectangle with the overlay surface 2 to be displayed. The destination area 6 preferably describes a portion of the primary surface 1 on which the overlay surf ace 2 will be displayed.

Source and destination areas 4, 6 do not have to be the same size. A destination areas 6 can be specified as being smaller or larger than the source area 4, so that the overlay surface is sized to fit the destination as appropriate.

Source and destination colour keys are used to control transparency of the overlay surface. A source colour key is used to determine which pixels in the overlay surface should be considered transparent, i.e.

those will allow the contents of the primary surface to show through. Similarly, a destination colour key is used to determine those parts of the primary surface that will be covered up by the overlay surf ace when it is displayed. The source key can be a f ixed value or can be a range of values, which allows f or errors and noise in the source image.

Colour keying allows part of the overlay surf ace to be transparent, such that the overlay surface can appear to be of arbitrary shape.

A source colour key decision is made by comparing the colour of the overlay surface with the source colour key. If the two match, then the overlap surface at that pixel is not to be displayed. Figures 5 and 6 illustrate source colour keying. The overlay surface includes a source image 7 surrounded by a source key area 8. The source key area 8 is used to def ine a rectangular bounding box around the source image 7.

Rectangular boxes are more straightforward to process than arbitrarily shaped images. When the overlay is displayed on the primary surface, those areas filled with the source key value are not displayed. This results in the displayed image of Figure 6.

A destination colour key decision is made by comparing the primary surface colour at a pixel to the destination colour key value. When the colour and colour key match, the overlay surface (determined by the source colour key decision) is displayed. Thus, to use the destination colour key method, an area of the primary surface is filled with the destination colour key value. The area defines the region in which the overlay is to be shown. The visible parts of the overlay are then shown in the area defined by the destination colour key value.

Figures 7, 8 and 9 illustrate the concept of destination keying. The primary surface 1 delivers a destination area 10 which is filled with the destination key value. Figure 8 illustrates the source overlay image 11 for display over the primary surface 1. The overlay image will be displayed over any region of the primary surface I which displays the destination key value. The resulting image is shown in Figure 9.

Since the data must be sent to the display device at a constant rate, and the process of resolving transparency of overlays must take place at the same rate, determined by the refresh rate of the display device the number of overlay surfaces that can be processed at any one time is limited. This is particularly the case when overlay surfaces overlap one another.

It is therefore desirable to provide a method and apparatus which can overcome the limitations of previously considered overlay processing techniques.

SUMMARY OF THE PRESENT INVENTION

According to one aspect of the present invention there is provided a method of processing image data blocks representing image surfaces for display on a display device, wherein respective image data blocks are combined to f orm a compound image data block and a corresponding compound control data block, and wherein the compound image data block is processed in accordance with the compound control data block.

According to another aspect of the present invention, there is provided a method of processing data representing overlay surfaces to be displayed on a display device having a plurality of pixels, the method comprising:

receiving overlay data representing respective overlay surfaces to be displayed; receiving key data relating to respective key values corresponding to the said overlay surfaces; combining the received overlay data to produce compound overlay data representing a single compound overlay surface; and combining the received key data to produce compound key data representing a compound key surface corresponding to the compound overlay surface, wherein the compound overlay data is for display on the device in accordance with the compound key data.

Such a method may also comprise, for each pixel of the device:

receiving primary surface data relating to a primary surface to be displayed; comparing the primary surface data with the compound key data for the pixel concerned; and if the primary surface data matches the compound key data, displaying the compound overlay data, or if the primary surface data does not match the compound key data, displaying the primary surface data.

According to another aspect of the present invention, there is provided apparatus for supplying data to a display device having a plurality of pixels, is the apparatus comprising:

overlay receive means for receiving overlay data representing respective overlay surfaces to be displayed, and key data relating to respective key values corresponding to the said overlay surfaces; and overlay processing means operable to combine the received overlay data to produce compound overlay data representing a single compound overlay surface, and to combine the received key data to produce compound key data representing a compound key surface corresponding to the compound overlay surface, and output means operable to output the compound overlay data and the compound key data.

Such apparatus may comprise:

receive means connected for receiving primary data relating to a primary surface to be displayed, and for receiving the compound overlay data and compound key data, comparison means operable to compare, for each pixel of the display device, the primary data with the compound key data for the pixel concerned; and transmit means operable to transmit the compound overlay data to the display device if the primary data matches the compound key data, or to transmit the primary data to the display device if the primary data does not match the compound key data.

According to another aspect of the present invention, there is provided a method of processing data in which respective data blocks are combined to form a compound data block and a corresponding compound control data block, the compound data block being processed in accordance with the compound control data block.

BRIEF DESCRIPTION OF THE DRAWINGS is Figures I and 2 illustrate the display of overlay surfaces on a display device; Figures 3 and 4 illustrate source and destination rectangles respectively; Figure 5 and 6 respectively illustrate source keying; Figure 7, 8 and 9 illustrate destination colour keying; Figure 10 illustrates overlapping overlay surfaces.

Figures 11 and 12 illustrate compound overlay and key surfaces respectively; Figures 13 and 14 illustrate a method embodying one aspect of the present invention; and Figure 15 illustrates one hardware solution for carrying out the method of Figure 14.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

As described with reference to Figures 1 to 9, overlay surfaces to be displayed on a primary surface must be processed just before display in order to determine whether the overlay surf ace or primary surface is to be displayed.

Figure 10 illustrates two overlay surfaces 12 and 14 which overlap one another to produce an overlapping region 16. With the previously considered overlay surface processing systems described with reference to Figures 1 to 9, the processing of the overlapping region 16 causes complications and so limits the number of overlay surfaces supported by the system. The problems caused by overlapping overlays are due to the fact that extra processing is required to determine the visibility of each of the overlay surfaces in the overlapping region 16, and that this processing must be computed within a short fixed period of time.

In systems embodying the present invention, however, any number of overlay surfaces can be displayed over a primary surface. As described above, images are displayed as a series of frames, and in embodiments of the present invention, during display of a frame, overlay data relating to a future frame is calculated ready for processing by hardware overlay system.

A method embodying the present invention will now be described with reference to the pair of overlapping overlay surfaces 12 and 14 shown in Figure 10. The overlay surfaces 12 and 14 overlap to produce an overlapping region 16. Before display of the two overlapping overlay surfaces, it is necessary to determine the contents of the overlapping region 16.

Since each overlay surface will have its own source and destination colour keys, the processing of the overlapping region 16 can be complex.

The two overlay surfaces 12 and 14 are combined into a single compound overlay surface OC as illustrated in Figure 11. The compound overlay surface OC extends to form a single rectangular surface bounding the overlay surf aces 12 and 14 of Figure 10.

The compound overlay includes a region corresponding to overlay surface 12, and a region corresponding to overlay surface 14. The regions 18 outside of those areas are filled with a source key value OT, such that the areas 18 are not displayed during display of the compound overlay surface.

In addition, a compound destination key surface KC is calculated and includes a region 13 corresponding to the destination key value of overlay surface 12 and a region 15 corresponding to the destination key of overlay surface 14. The remainder of the surface KC is can be filled with any value, since those areas correspond to the source key areas 18 of the compound overlay surface OC, and thus will not be displayed. In order to determine the values for each pixel in the compound overlay and compound destination key surfaces, the value for the most visible (i.e. top) overlay surface at the pixel concerned is used.

The compound overlay surface OC and a compound destination key surface KC, are then used for the actual display calculations of the display device. As mentioned above, the composition of the compound surfaces is performed in advance, possibly during display of the immediately previous frame by the display device. For pixels which display no or one overlay surface, these calculations are simple and hence fast. The calculations concerning the overlapping region 16- (Figure 7) are more complex, and hence take more time. However, since the calculation of the compound overlay surface for the next frame must only be completed by the end of the scanning period for the previous frame, then the processing time requirement is simply that the average processing time for each pixel meets the scanning time conditions.

This means that the "pre -processing" of the compound overlay surface and compound colour key surface can take advantage of the pixels having single or no overlay surfaces to be displayed and can use the time made available from that fast processing to enable processing of the overlapping region 16 to be performed. The processing of the compound surface need only take place where changes occur in the frame.

Unchanged pixels need not be regenerated.

The compound overlay surface OC, and compound colour destination surface KC data is transferred to the display device and is processed along with the primary surface data as before. However, instead of having a single destination key value for the whole overlay, the destination key value is determined on a pixel by pixel basis from the compound destination key surface. The key values can therefore vary across the screen area, and so multiple overlays can be processed.

Figure 13 illustrates steps in a method for producing the compound overlay values for each pixel of the display. At step A the number of overlay surfaces to be displayed at the pixel is determined. If no overlay surfaces are to be displayed, the pixel values are set to the compound overlay source key value (step B) and processing moves to the next pixel (step C).

If, however, one or more overlay surfaces are to be displayed, then the pixel values are set to equal the values indicated -by that the f irst visible overlay surface (step D). Alternatively, the visible overlay may be transparent or blended and so other overlays will contribute to the overall image. The compound destination key value f or that pixel is set to the value f or the f irst visible surf ace. Processing then proceeds to the next pixel (step C).

Figure 14 illustrates steps in the processing of the primary surface, the compound overlay surface and compound destination key surf ace information to determine which is to be displayed on the display device. At step t, the primary surface pixel value is compared with the compound destination key value for that pixel. If the destination key and primary surface values match (step u), then the compound overlay pixel value is compared (step v) with. the compound source key value. If the values do not match (step w) then the compound overlay pixel value is displayed (step x) Ue.

is the pixel belongs to region OA or OB of the compound overlay surface of Figure 11). If, however, either the compound destination key value does not match the primary surface pixel value, or the compound overlay pixel value matches the compound source key value, then the primary surface is displayed (step y) Processing then moves to the next pixel (step z).

In this way, a method embodying the invention can be used to display any number of overlay surfaces, and can support source and destination colour keying, scaling, colour space conversion, per-overlay look up tables and alpha blending, while only using the hardware resources for two simple non-scaled overlays for displaying the surfaces.

Figure 15 illustrates hardware for implementing the method of Figure 14. The primary surface pixel value 21 and. the compound key surface pixel value 22 are compared by a comparator 23 to produce a compound overlay control signal 24. The control signal 24 is used to control a multiplexer 27, which has as its inputs the compound overlay surface pixel value 25 and the primary surface pixel value 26. The output of the multiplexer 27 provides a display data value 28 for transmittal to the display device. As described above, if the primary surf ace value is the same as the key value, then the compound overlay is displayed, and if not then the primary surface is displayed.

It will be readily appreciated that the method described above for combining overlay and primary image surfaces could be used to process any type of data, even if that data is not for actual display. For example, it is conceivable that sophisticated data encryption techniques would be possible using such a method.

Claims

CLAIMS:

1. A method of processing image data blocks s representing image surfaces for display on a display device, wherein respective image data blocks are combined to form a compound image data block and a corresponding compound control data block, and wherein the compound image data block is processed in accordance with the compound control data block.

2. A method as claimed in claim 1, wherein the compound image data block is combined with a background image data block in dependence upon the compound is control data block.

3. A method as claimed in claim 2, wherein the image data blocks are overlay surfaces, and the background image data block is a primary image surface.

4. A method as claimed in claim 3, wherein the primary image surface is a desktop surface.

5. A method of processing data representing overlay surfaces to be displayed on a display device, the method comprising:

6. A method as claimed in claim 5, for a display device having a plurality of pixels, the method comprising, for each pixel of the display device:

receiving primary surface data relating to a primary surface to be displayed; comparing the primary surface data with the compound key data for the pixel concerned; and transmitting the compound overlay data to a display device if the primary surface data matches the compound key data, and transmitting the primary data to a display device if the primary data does not match the compound key data.

7. A method as claimed in claim 5, for a display device having a plurality of pixels, the method comprising, for each pixel of the device:

receiving primary data relating to a primary surface to be displayed; comparing the primary data with the compound key data for the pixel concerned; comparing the compound overlay data with a key value if the primary data matches the compound key data; and transmitting the primary data if the primary data does not match the compound key data, or if the compound overlay data matches the key value; and transmitting the compound overlay data if the compound overlay data does not match the key value.

I -is-

8. Apparatus for supplying data to a display device having a plurality of pixels, the apparatus comprising:

overlay receive means operable to receive overlay data representing respective overlay surfaces to be displayed, and key data relating to respective key values corresponding to the said overlay surfaces; and overlay processing means operable to combine the received overlay data to produce compound overlay data representing a single compound overlay surface, and to combine the received key data to produce compound key data representing a compound key surface corresponding to the compound overlay surface; and output means operable to output the compound overlay data and the compound key data.

9. Apparatus as claimed in claim 8, comprising:

receive means connected operable to receive primary data relating to a primary surface to be displayed, and for receiving the compound overlay data and the compound key data; comparison means operable to compare, for each pixel of the display device, the primary data with the compound key data for the pixel concerned; and transmit means operable to transmit the compound overlay data to the display device if the primary data matches the compound key data, or to transmit the primary data to the display device if the primary data does not match the compound key data.

10. Apparatus as claimed in claim 8, comprising:

receive means connected operable receive primary data relating to a primary surface to be displayed, and for receiving the compound overlay data and the compound key data; comparison means operable to compare, for each pixel of the device, the primary data with the compound key data for the pixel concerned, and to compare the compound overlay data with a key value if the primary data matches the compound key data; and transmit means operable to transmit the primary data if the primary data does not match the compound key data, or if the compound overlay data matches the key value, and operable to transmit the compound overlay data if the compound overlay data does not match the key.

11. A computer system including apparatus as claimed in claim 8, 9 or 10.

12. An integrated circuit including apparatus as claimed in claim 8, 9 or 10.

13. A games console including apparatus as claimed in claim 8, 9 or 10.

14. A set top box including apparatus as claimed in claim 8, 9 or 10.

15. A computer graphics systems including an image generating unit, and apparatus as claimed in claim 8, 9 or 10.

16. A method of processing image data substantially as 30 hereinbefore described with reference to the accompanying drawinga.

17. Apparatus for processing image data substantially as hereinbefore described with reference to, and as shown in, the accompanying drawings.

18. A method of processing data in which respective data blocks are combined to form a compound data block and a corresponding compound control data block, the compound data block being processed in accordance with the compound control data block.

19. A method as claimed in claim 18, wherein the compound data block is combined with a base data block in dependence upon the compound control data block.

20. A method as claimed in claim 19, wherein the respective data blocks represent overlay images, and the base image data block represents a primary image.

21. A method as claimed in claim 20, wherein the primary image is a desktop surface of a computer system.