AU628731B2 - Improved keying methods for digital video - Google Patents

Description

Signature of applicant or Australian attorney &t TO: THE COMMISSIONER OF PATENTS This form must be accompanied by either a provisional specification (Form 9 and true copy) or by a complete specification (Form 10 and true copy).

Printed by R. D. RUBIE, Commonwealth Government Printer, Canberra s F bc:__i

A&

S F Ref: 133054 FORM COMMONWEALTH OF AUSTRALIA PALTETS ACT 8 7 3 1 COMPLETE SPECIFICATION

(ORIGINAL)

FOR OFFICE USE: Class Int. Class Application Number: PJ9585 11 April 1990 Lodged: eo o *a o o 04 o 0 0 086 Cognated With: PK0988 Lodged: 3 July 1990 Accepted: Published: Priority; Related Art: Name ar; Address of Applicant: 0 8 4 4 0 on ~d 0 80a 04 0 Rank Cintel Limited Natton Road Ware Hertfordshire SG12 OAE UNITED KINGDOM 0 Address for Service: 0*84 0 8 4 Spruson Ferguson, Patent Attorneys, Level 33 St Martins Tower, 31 Market Street, Sydney, New South Wales, 2000, Australia Complete Specification for the invention entitled: Improved Keying Methods for Digital Video The following statement is a full description of this invention, including the best method of performing it known to me/us Mr OF RECEIPT S1060 815/5 O/04/9 1 b,

AUSTRALIA

S F 133054 JED/2025D el. (Sole Figure)

ABSTRACT

IMPROVED KEYING METHODS FOR DIGITAL VIDEO A digital video key signal generator for digital video special effects uses two cascaded prog:ammable lookup tables (20,24). One table (20) is addressed by the two chroma component signals and the second table (24) is addressed by the luma signal and the output of the first table. The generator is particularly suitable for use with CCIR Rec. 601 digital o signals, and can be used for a wide variety of compositing functions, using luma keys, chroma keys, combinations thereof, 'lS" and multiple chroma keys simultaneously.

o lt 4 0440 a t tt i I 1 IMPROVED KEYING METHODS FOR DIGITAL VIDEO Background of the Invention This invention relates to the generation of keying signals for use with digital video signals. Keying signals or key signals are used to define regions of an image to which a certain operation is to be applied. Typically, a keying signal is used to make a newsreader who is in fact located in a studio appear in front of a picture of a remote location. The background picture may be derived from a slide or solid state store for example. A key signal may also be used to define parts of a picture in which certain colours are to be altered.

The keying signal is usually generated from a video a signal itself. For example, the newsreader in a studio, o mentioned above, will be seated in front of a blue backdrop.

0 0a The output of a foreground camera viewing the newsreader is 0 analysed to determine which areas of the picture are blue, and oa^" thus to generate a key signal which is used to switch in the background picture in those regions. The key signal may be termed a "matte" or "stencil", and may be single valued (ON or OFF), but is preferably multivalued to allow a fading operation particularly at the boundary of objects for example.

All this has been known in the analogue video processing field for many years.

Key signal generators in the analogue field generally use 0°0° large amounts of special purpose analogue circuitry such as ooo comparators. In the digital field they use a digital equivalent of this technique.

*o0o. The present invention is concerned with the adaptation of these techniques for use with digital signals, particularly 0 0 though not exclusively to digital video signals encoded in accordance with CCIR Recommendations 610 and 656. In accordance with these recommendations, out of every four signal samples, two are luma (luminance) samples, and two are chroma (chrominance) samples. The two chroma samples correspond respectively with the R-Y and B-Y colour component signals of the PAL system and are associated in time with one only of the two luma samples.

A-

L i i Summary of the Invention We have appreciated that a simple form of key signal generator has particular utility in the generation of key signals for compositing and special effects with digital video signals.

The present invention is defined in the appended claims, to which reference should now be made.

In accordance with this invention it is possible to make a key generator for the real-time generation of digital video keys at video rates, with user-programmable and very flexible key generation rules. It is possible for the key signal to be responsive to both the chroma and luma signals, and can be operated to combine multiple keys, for example. Nevertheless the key generator is small and inexpensive.

0 e BRIEF DESCRIPTION OF THE DRAWING 06 0 The invention will now be described in more detail by way o00 of example, with reference to the sole figure of the accompanying drawing, which is a block diagram of a key generator circuit 0o embodying the invention used to control a simple compositing 0 00 device.

Detailed Description of the Preferred Embodiment The key signal generator 10 shown has two chroma inputs, namely an input 12 for the R-Y component signal and an input 14 0a0 for the B-Y signal, and a luma input 16 for the luma signal.

00 All three inputs comprise 8 bit samples in accordance with CCIR Recommendations 601 and 656.

A first programmable read-only memory (PROM) constituting a programmable lookup table is connected to the inputs 12 and 14 to receive the two chroma inputs. The PROM has 64K or 65536 locations which can be addressed by the two '8-bit inputs, and is 8 bits "deep", i.e. each location contains an 8-bit word which is applied to the PROM output 22 when that location is addressed.

A second PROM 24 also having 64K locations each of 8 bits is connected to receive 8 bits from the luma input 16 and 8 bits from the output 22 of the PROM 20. The second PROM 22 provides an 8-bit key signal output 26.

2 i C 1^1_ 1i l~tllYrl lllll~- I Each of the PROMs 20 and 24 may be constituted by a static random access memory (SRAM) such as the Hitachi HM6208HP-35, 64K x 4, 35ns access time SRAM. Two such devices are needed for each lookup table to give the full eight bits depth. SRAM's are used as they can conveniently be updated from time to time by a microprocessor.

Purely for the purposes of illustration the bottom half of the figure shows a simple compositing system. Two video signals A and B are received at inputs 40 and 42 and applied to respective multipliers 44 and 46. Multiplier 44 receives the keying signal directly from PROM 24, and interprets the 8-bit signal as a fraction a where a varies from 0 to 1 as the key 6oe signal varies from 0 to 255. The keying signal is also applied to a complementing circuit 48 which takes the complement of a S* with respect to one, giving 1-a. This signal is applied to oo 0o o multiplier 46. The outputs of multipliers 44 and 46 are then added in an adder 50 to produce a video output signal on an output 52.

By cascading two lookup tables in this way it is possible to provide a very large number of very complex keys. In operation, the first table 20 uses the two chroma signals as an address to generate a chroma key, with one 8-bit key value 0 6 o«o programmed into each location of the table. The table can be 0o a considered to be a map of "chroma space", with say R-Y horizontal *00o and B-Y vertical. Thus every possible combination of R-Y and 0000 B-Y has its own key value programmed into the chroma map. This key is programmable and can thus be set up in any way the user 0 c 0 wants. For example it can be interpreted as a linear key o 0 taking the values 0 to 255, or as 8 separate keys to operate 8 quite separate switches in ON/OFF mode, and so on.

The second lookup table 24 combines the chroma key with the luma signal. It is identical in physical construction to the chroma lookup table, but is programmed separately. This then provides a unique key output for each of the 64K possible combinations of luma and chroma map key in "luma-chroma-key" space. This final output can also be interpreted in any way the user wishes.

3 Examples of Keying Applications.

The key generator illustrated provides arbitrary, user alterable, real-time key generations for digital video signals, such as in accordance with CCIR Recommendations 601 and 656.

The arbitrary transformation is accomplished with two banks of 65,536 words of high-speed static RAM to produce an eight-bit key signal. A major advantage of the system illustrated is its flexibility. We have found it to be a particularly convenient tool for enabling the generation of a very wide range of possible types of key signal. Nevertheless it is very simple in construction and is thus not expensive or difficult to make even though it is capable of real-time working. PROMs of appropriate size capable of operating at video rates are readily available.

oooo A number of examples of how the system may be used will now be given.

0J 0 0 00 0 a. Luma Key.

°0:0 The luma key is the simplest of the applications. This oo generates a key which is derived only from the luma component of a the signal, e.g. the key is ON for all colours darker than a threshold. For this use, the chroma map is filled with a constant value, or alternatively the luma map is filled with 256 copies of the one map, so that the chroma value is ignored.

a 0 b. Hard-edged Single Chroma Key.

PA hard-edged chroma key can be set up so that a region of chroma space (the chroma key colours) is programmed with 255 (full ON), with all other areas (non-chroma key areas) programmed n00° to 0. This is the same as normal aliased (hard-edged) chroma .00 o keying. Note that it is only necessary to distinguish the key area from the non-key area, so one bit of the 8 is sufficient, and the other 7 bits can be ignored.

At the same time, the luma map can be set up, for example, to ignore the darkest and lightest of colours. For example, if the chroma key colour is mid blue, then very dark blue and very pale blue should not generate a key output. The output of this map (either 1 bit or more) is a hard-edged chroma key defined both by a chroma space and by a luma range.

-4-

A

c. Hard-edged Multiple Chroma Key, In an extension of this principle, using all 8 bits of keys, the chroma map can be set up so that there are up to 256 independent chroma regions, with the output of the chroma lookup table being defined as the region in which the chroma of that pixel falls. Each of these regions can be defined independently and arbitrarily, provided that they do not overlap.

This key is then fed to the luma map, which is programmed to output an 8-bit key (which is interpreted any way the user wishes which is programmable for each colour region, with a luma profile attached. The simplest application of this is to define a certain number of the 256 colour regions as chroma-key oseo regions, and the others as non-key regions, then use the usual S' luma profiling (remove dark and bright) to generate a one-bit key o which is true for all pixels within the defined regions. It is also possible to form 8 independent one-bit keys out of this map, or simply output the region number if it was in the right luma o range. Similar extensions will be apparent to those skilled in the art.

d. Soft-edged Chroma Key.

A soft-edged (anti-aliased) chroma key can be set up so o that a region of chroma space (the chroma key colours) is programmed with 255 (full ON), a surrounding region is programmed 0,0° in the range 1 to 254, fading out to 1 as it goes outwards, with all other areas of (non-chroma key areas) programmed to 0.

There is no reason to limit this to one region as it is feasible 0 to define blue-green and yellow as both chroma-key regions, for example. As well as this, the profile of the fall-off from key o to non-key need not be linear (or even monotoric), but can be whatever the user desires.

At the same time, the luma map can be set up, for example to ignore the darkest and lightest of colours by multiplying the first key by a profile. For example, if the chroma key colour is mid blue, then very dark blue and very pale blue should not generate a key output. Of course, the profile of the fall-off from key to non-key need not be linear (or even monotonic), but 5 i can be whatever the user desires. The output of this map is a soft-edged chroma key defined both by a chrona space (or spaces) and by a luma profile.

e. Combinations of Keys.

One of the major advantages of this technique for generating keys is its immense flexibility. The effects above can be combined: for example with a multiple-region linear key.

In this example, the aim is to show a figure of a news reader (who is filmed in front of a chroma-key background) in front of a pre-recorded backdrop, and at the same time make the eyes (a dull grey colour) bright blue. One thing that is necessary for this is that the key signal be encoded with the information as to whether each pixel is in the newsreader, the chroma-key 6 3 background, or the eyes. This can be defined in the following 6 o o) way: one bit determines whether the key applies to the eyes or the background, while the other 7 bits are a linear key oa°o identifying the match of the colour with the chroma-key colour or 06 eyes.

The key signal can therefore be readily produced. The chroma map is programmed to generate a linear key as described for the two regions. This key is then sent to the luma map, o{o ere the one-bit key is passed straight through while the 7-bit aa t key is modified by a luma profile (different for the two colour regions, of course). This means that the two colour regions VP° have been iniquely defined in three-dimensional colour space.

o a -6-

Claims (2)

1. A digital video key signal generator, comprising: first and second chroma inputs for receiving first and second chroma digital signals, a first programmable lookup table device coupled to the first and second chroma inputs to be addressed by the first and second chroma signals and to output a chroma key signal in response thereto, a luma input for receiving a luma digital signal, and a second programmable lookup table device coupled to o, the luma input and to the output of the first programmable lookup SO table device to be addressed by the luma signal and the chroma "4 key signal and to output an output key signal in response 0 9 G 'thereto.

2. A machine method of generating a digital video key signal, comprising the steps of providing a first programmable lookup table device and a second programmable lookup table device, programming said devices with lookup tables, applying first and second chroma digital signals to address said first o programmable lookup table device to output a chroma key signal in a response thereto, and applying a luma digital signal and said a' chroma key signal to address said second programmable lookup table device to output an output key signal in response thereto. st"" DATED this TENTH day of APRIL 1991 SRank Cintel Limited Patent Attorneys for the Applicant SPRUSON FERGUSON -7-