GB2245795A - Generating convoluted digital video signals - Google Patents

Abstract

To generate a convoluted digital video signal, e.g. one in which each pixel in an output image is derived from the corresponding pixel of an input image and pixels in the neighbourhood of it in proportions determined by a convolution function, the input signal is held in an input framestore 14 the output of which is applied to some of the address inputs of a lookup table 16. An intermediate framestore 24 accumulates intermediate results and applies its output to some of the address inputs of a second lookup table 26. The remaining address inputs of the lookup tables are connected to a control signal input 18. An adder 20 combines the outputs of the lookup tables and recirculates the resultant to the intermediate framestore. The intermediate framestore is capable of being written to or read from with an offset of one or more pixels vertically and/or horizontally, and an offset path is traversed covering all the input pixels which contribute to a given output pixel, finishing up at the original zero-offset position. Other functions can be performed by appropriately re-programming the lookup tables, e.g. special effects such as a "snail trail" for moving objects, fading, colourising, edge enhancement, anti-aliassing and other filtering. At least one of the tables, e.g. 26, may merely give a 1:1 transfer, and for purely temporal convolution the store 14 is transparent. <IMAGE>

Description

GENERATING CONVOLUTED DIGITAL SIGNALS Background of the Invention This invention relates to the generation of convoluted digital signals.

By a "convoluted digital signal" is meant a signal in which each pixel in an output image is derived from the corresponding pixel of an input image and pixels in the neighbourhood of it, in proportions determined. by a convolution function. Typically such a function is presently achieved by a transversal filter, but that is relatively complex and inflexible. In general the known convolution generators require purpose-built hardware and can offer linear transfer functions only.

Summarv of the Invention The present invention is defined in the independent claims 1 and 3 below to which reference should now be made.

Advantageous features of the invention are defined in the subclaims.

In a preferred embodiment of the invention, described in more detail below, an input digital video signal is held in an input framestore the output of which is applied to some of the address inputs of a first lookup table in a memory, typically a programmable read- only memory. An intermediate framestore accumulates results and applies its output to some of the address inputs of a second lookup table. The remaining address inputs of the lookup tables are connected to a control signal input.

An adder combines the outputs of the first and second lookup tables, and recirculates the resultant to the intermediate framestore. The intermediate framestore is capable of being written to or read from with an offset of one or more pixels vertically and/or horizontally. An offset path is traversed covering all the input pixels which contribute to a given output pixel, finishing up at the original zero-offset position.

The generator is extremely flexible, and can be used for many different convolution functions spatially and/or temporally.

It can also provide the functions of video fader and colouriser, and can be used for special effects, allowing very effective and efficient hardware usage.

Brief Description of the Drawings The invention will be described in more detail by way of example with reference to the accompanying drawings, in which: Figure 1 is a block diagram of a convolution generator embodying the invention; Figure 2 is a schematic diagram illustrating the operation of the convolution generator of Figure 1; and Figure 3 is a diagram used in explaining the operation of Figure 2.

Detailed DescriPtion of the Preferred Embodiment The preferred embodiment illustrated is an implementation of a non-real time video convolution using a multi-pass technique for the creation of intermediate result terms. The technique can be applied to convolution of arbitrary dimensions of both spatial and temporal forms, and to combinations of the two, all operating on the same hardware. The technique can also be used in a real time situation if the order of the convolution is not too large, with adaptation of the hardware to operate at higher speed. An example of this is 2 by 1 convolution to eliminate interlace flicker in video displays, which is performed by spatially convolving components on adjacent lines. Another example is edge enhancement of video images, which is done by convolving spatially pixels that are adjacent to each other on each scan line.Temporal convolution of two frames could be used to create a snail-trail video effect, where moving features leave a "halo" behind them as they move.

The convolution operation is useful in the digital video field as it allows anti-aliasing and other filtering functions to be performed. These are especially useful because of the widespread use of interlaced video systems and the need to limit the bandwidth of video signals for transmission purposes.

Interlaced video systems operate with a frame rate at a sub-multiple of the field display rate, typically with a 2:1 interlace. With these systems there exists the potential for visible flicker where sharp edges occur in the original picture.

A convolution filter applied to such an image can smooth out the sharp edges resulting in a reduction of the visible flicker.

The preferred convolution generator 10 illustrated in Figure 1 has a digital video signal input 12 connected to an input framestore 14. Suitable framestores are well-known and readily available. A first programmable read-only memory (PROM) 16 containing a lookup table A is connected with its address inputs coupled to the output of the input framestore 14 (8 bits) and to receive a separate control signal (8 bits) which is applied to a control signal input 18. The output of PROM 16 is applied to one input of an adder 20. The output of the adder is applied to the generator output 22 but is also fed back to the input of an intermediate framestore 24, similar to framestore 14.

A second PROM 26 containing a lookup table B is connected with its address inputs coupled to the output of theintermediate framestore 24 (8 bits) and to receive the control signal (8 bits) from control signal input 18. The output of PROM 26 is applied to the second input of the adder 20.

In the preferred convolution generator embodying the invention, the lookup tables are used to perform a multiplier or other arbitrary function on an incoming digital video value and to add this to a value stored in the intermediate framestore 24.

The convolution terms are formed by shifting the position of the result in the intermediate fram-estore relative to the position of the original video value. On subsequent iterations a different transfer function is selected in the lookup tables and the resulting value is added on to a value from the intermediate framestore and stored with a shift in position in the intermediate framestore. After the appropriate number of cycles of the video data has occurred the resulting output video data contains a convolution of the input video which may either be stored in the intermediate framestore or output to another device. The number of cycles through the intermediate framestore that is required depends solely on the order of the convolution being performed, for example a three-by-three convolution requires a total of nine cycles.

In the intermediate framestore the initial frame storage position is able to be set to a different place from that of the initial readback position. This achieves the positional offset that is necessary to combine the terms of the convolution that are separated either spatially or temporally. An example of such a framestore is described in our United Kingdom Patent Application No.9107133.2 of even date entitled "Video Framestore Access" (claiming priority from Australian Patent Application No.PK0677 entitled "Versatile Video Framestore Access Architecture").

The preferred embodiment of the present invention provides arbitrary, user-alterable convolution of digital video signals encoded in accordance with CCIR Recommendation 656 at a speed suitable for user interaction. The arbitrary convolution is accomplished with 65,536 words of high speed static RAM. This achieves at real-time video speed each new term of the convolution; thus the overall speed of the convolution is near real-time. Real time in the sense used here refers to the ability to process and generate video encoded using the CCIR-656 standard at the full video rate of 25 or 30 frames per second.

The block diagram of Figure 1 shows the main video paths in the preferred implementation of the convolution generator. The video paths shown correspond to the luma video signal only. The chroma path is essentially similar, so the hardware would be duplicated for the chroma section. In CCIR-656, digital video is comprised of two luma and two chroma samples in each pixel time.

The two chroma values represent the blue minus luma (B-Y) and red mirius luma (R-Y) samples and are associated with the first luma sample. The second luma sample has no associated chroma samples.

The incoming video signal consists of 16 bits in parallel, of which 8 bits represent luminance and the other 8 bits represent chrominance. The luma portion is an unsigned binary value whereas the chroma value is an 8-bit offset binary value.

For this reason it is necessary to use different transfer functions in the maps for luma and chroma, and for increased flexibility it may also be desirable to duplicate the chroma map hardware so that the two chroma components operate with different naps. This can be done by demultiplexing the two chroma components before they enter the two lookup tables and re-multiplexing the output from the lookup tables before the adder. Using two separate chroma maps allows more flexible colourising and other special effects but is not strictly required for convolution.

Convolution Process In order to perform the convolution operation it is necessary to program the lookup tables with the desired transfer functions for each term in the convolution. This can be done before the convolution begins or in the blanking period between generation of each new term. The advantage of pre-computing and storing the transfer function before the convolution begins is that it gives low host processor overhead during the execution of the convolution.

The input framestore 14 is set up to generate a still video image during the generation of the convolution if it is a purely spatial convolution. In this case the video input frame should be captured in the framestore before the convolution process begins. In the case of a purely temporal convolution the input framestore would be set up to be transparent to the incoming video.

The intermediate framestore 24 is cleared and is set up to generate a full frame video signal on its output. On the input side it is set up so that it writes to the framestore at a position offset from where it would read from the framestore at the same point in the frame. This would typically mean writing either one pixel after or before or one line after or before the position at which the read is taking place. The actual offset determines where the next term in the convolution comes from in the input frame and which transfer function should be selected.

The sequence of the offsets is arranged so that all positions that are to contribute to the convolution are covered. During each cycle the input frame remains unchanged in the input framestore 14, but the image in the intermediate framestore 24 is gradually built up. Each pixel in the intermediate framestore is formed of predetermined proportions of the corresponding pixel in the original image and the various pixels adjacent to or in the neighbourhood of it. The offsets accumulate until the start position is again reached at which point the convolution is complete.

For each new term in the convolution the control signal is altered so that the required transfer function for that particular term is selected in the lookup tables. A simple approach to generating the lookup tables would use a 1:1 transfer from input to output in lookup table B attached to the intermediate framestore. It would also have a simple division by the coefficient of each convolution term in lookup table A attached to the input framestore. This would result in a linear convolution sum. The process would be susceptible to accumulated roundoff errors if 8-bit data values are used throughout.

The diagram of Figure 2 shows an example of operation of the convolution technique. This example shows the result at each stage in the convolution process for the terms that make up the convolution at position 'a' in the input framestore. The diagram shows a 2 by 2 spatial convolution with a pass through the map in lookup table B and the convolution multiplier functions in lookup table A. Figure 3 shows at (a) the four pixel positions in the input image which contribute to pixel a at the output, at (b) the respective weights w, x, y and z applied to the pixels a, b, c and d, these weights being held as convolution multipliers in lookup table A, and at (c) the constant multiplier used in this example to give a "pass" through lookup table B.

The four iterations are shown progressing downwardly in Figure 2. In the figure "mult" stands for "multiply by". On each iteration of the process a different function is chosen in lookup table A, in this example multiplication by w, x, z and y.

In iteration 0 the result is produced by the adder 20 for every pixel by shifting one pixel to the right in the intermediate framestore. In iteration 1 the result is shifted down by one pixel. On the next two iterations it is shifted left and up by one pixel respectively. The result is left in the intermediate framestore with the pixels in the correct position. Note that the choice of ordering of the direction of the shifts and multiplier functions is arbitrary.

Claims (4)

1. A method of generating a convoluted digital video signal, said method comprising the steps of: providing an input digital video signal via an input framestore to a first address input of a first lookup table; providing a control signal as a second address input to said first lookup table and a second lookup table; combining data selected from each said lookup table to form said convoluted signal; and feeding back said convoluted signal via an intermediate framestore to a first address input of said second lookup table.

2. A method according to claim 1, in which the signal is fed through the intermediate framestore a plurality of times subject to respective offsets of pixel position, the offsets being on a path that returns to the initial pixel position at the end of the generation operation.

3. Apparatus for generating a convoluted digital signal, comprising: a video signal input for receiving an input digital video signal; an input framestore connected to the video signal input; a first lookup table having a plurality of address inputs and at least a data output; a second lookup table having a plurality of address inputs and at least a data output; an adder having inputs connected to the data outputs of both the first and second lookup tables; an intermediate framestore connected to the output of the adder; and a control signal input; in which some of the address inputs of the first lookup table are connected to the output of the input framestore and some of the address inputs of the second lookup table are connected to the output of the intermediate framestore and the remaining inputs of the first and second lookup tables are connected to the control signal input.

4. Apparatus according to claim 3, in which the intermediate framestore is capable of being written to or read from with an offset, and including means for causing the writing or reading to follow an offset path to traverse all the input pixels which contribute to a given output pixel.