GB2214751A

GB2214751A - Video signal coding

Info

Publication number: GB2214751A
Application number: GB8802149A
Authority: GB
Inventors: Vincent Considine; Amarjit Singh Bhandal
Original assignee: Plessey Co Ltd
Current assignee: Plessey Co Ltd
Priority date: 1988-02-01
Filing date: 1988-02-01
Publication date: 1989-09-06
Anticipated expiration: 2008-02-01
Also published as: GB8802149D0; GB2214751B

Abstract

A system for implementing a brute force search block matching technique e.g. in a video coding arrangement using motion estimation, the system comprising signal input means for receiving time sequential sets of signals, means for storing a previous set of signals and a block of signals from a current set of signals, and an array of processing elements 8 wherein each element is arranged to perform comparison operations on respective signal elements from a predetermined part 12 of the previous set of signals and the block 4 of signals from the current set of signals whereby to match in a plurality of comparison 5, said block of signals with substantially all blocks of a similar size in said predetermined port. <IMAGE>

Description

VIDEO SIGNAL CODING This invention relates to a system for use in coding information by changes in sets of signals which- occur in a time sequence, and is particularly though not excusively appliable to video sig#nals to be transmitted via a CODEC system in compressed form by detecting changes in pixels between adjacent frames.

The coding of video signals by comparing respective pixels in adjacent frames is a well known process and relies on the fact that normally the video image changes slowly with respect to frame frequency. Various techniques have been employed for comparison and one particular technique is known as the brute force search motion estimation technique. This technique is described for example in R. Shrinivasan and to K. R. Rao, "Predictive Coding based on efficient motion estimation", in ICC 1984, Proc, pp 521-526, May 1984. Briefly the technique typically involves taking a block of pixels of a current frame and comparing the block with all possible blocks of similar size in a larger area of a previous frame, the best match being assumed to represent the same block, but displaced by a certain amount.It is therefore possible to code the block simply by defining a vector representing the movement of the block.

The technique is difficult to implement because of the large amount of computing required.

The present invention has as its object the provision of a computing system which can implement the brute force technique in a relatively simple and inexpensive form, preferably for implemention on one or more VLSI chips.

The present invention is based on the provision of an array of dedicated processor elements which can simultaneously perform a number of computations on groups of pixel elements in order to carry out a matching operation in a short time. Since the processor elements may be relatively simple in construction, they may be implemented on a single VLSI chip and may therefore be manufactured inexpensively.

The present invention therefore provides in one aspect a system for implementing a brute force search block matching technique, the system comprising signal input means for receiving time sequential sets of signals, means for storing a previous set of signals and a block of signals from a current set of signals, and an array of procesing elements wherein each element is arranged to perform comparison operations on respective signal elements from a predetermined part of the previous set of signals and the block of signals from the current set of signals whereby to match, in a plurality of comparison operations, said block of signals with all blocks of similar size in said predetermined area.

Thus in accordance with the invention, the brute force search block matching technique can be carried out quickly and simply with an inexpensive system.

As preferred the array of elements is arranged to take a single element from the block of signals from the current set and to compare in a simultaneous set of comparison operations, the single element with the corresponding elements in all possible blocks of the predetermined area of the previous signal set. However other methods of comparison may be envisaged, as for example comparing in each simultaneous set of comparison operations, all signal elements in the current block with a selected block in the previous set of signals.

As preferred the signals may be video signals, one signal set corresponding to a frame. However other signals may be processed in accordance with the invention . Usually the signals will be outputs from light or acoustic sensors where the output contains characteristic patterns which change relatively slowly.

In the case of video signals, it will normally be convenient to choose the current block as an array of 8x8 pixels, to be compared with an area of 20x20 pixels of an area of a previous frame, this representing the maximum extent the block may have moved between frames. An array of 15x15 proccessing elements is employed, each element accepting a single pixel from the current block, and comparing such pixel element with each corresponding element in the 15x15 possible blocks within the 20x20 pixel area.

By carrying out 8x8 consecutive computation operations the blocks within the area may be matched with the current block.

Each processing element preferably includes means for comparing and computing the signal intensity differences of compared signals and means for accumulating a partial sum of the results of a sequence of comparison operations.

A problem which arises in the use of an array of processing elements is the need for fast memory operations to enable a new set of data to be presented to the processing array. As preferred shift register memories are employed which enable, so far as concerns the current block, a new pixel to be presented to the processing element array for each set of computations. So far as concerns the area of the previous frame, the pixels thereof are moved sequentially through the processing array from one processing element to the next between successive computation operations. Thus this arrangement of memory enables a fast movement of data to avoid delay in computation operations and also to avoid complicated data transfer operations.

Thus in accordance with a further aspect of the invention there is provided a system for implementing a brute force search block matching technique, the system comprising signal input means for receiving time sequential sets of signals, means for storing a previous set of signals and a block of signals from a current set of signals, and processing means arranged to perform a plurality of consecutive comparison operations on respective signal elements from a predetermined part of the previous set of signals and the block of signals from the current set of signals wherein said block of signals and said predetermined part of the previous set of signals are stored in respective first and second shift register means to enable different portions of the predetermined part and/or the block of signals to be presented to the processing means for each computation operation by a shift of a single position of the signals in the first or second shift register means.

Said processing means may comprise the array of processing elements as set forth above. The memories and the array of processing elements may be implemented on one VLSI chip or on seperate VLSI chips.

A preferred embodiment of the invention will now be described with reference to the accompanying drawings, wherein; Figure 1 is a block diagram of the preferred system for implementing a brute force search block matching technique according to the invention; Figure 2 is a schematic representation of the memory arangement for the search window memory of figure 1; Figure 3 is a schematic representation of the memory arrangement for the current block memory of Figure 1; and Figure 4 is a more detailed block diagram of an element of the processing array of Figure 1.

Referring now to the drawings, Figure 1 shows a block diagram of the preferred system according to the invention which is intended to be implemented on a single VLSI chip. Pixels elements from a video signal intended to be transmitted over some form of transmission medium (landline, VHF transmission, etc.) are fed into the system and are presented one pixel at a time as they arrive in the system at an input 2 to a current block memory which stores 8x8 pixel elements. The output from the block 6 presents the last stored pixel element to each processing element of a 15x15 processing array 8. The pixel element of the previous frame of video signals are stored in a separate memory area and are fed via an input 10 to a search window memory 12-comprising a 22x22 array of pixel elements. Such pixel elements are compared with the pixel elements of the current block memory in array 8 and the results of the comparisons are summated in comparison logic 14 in order to derive from logic output 16 a motion vector which represents the displacement of the current block from the position of the block in the search window.

Figure 2 shows schematically in greater detail the form of search windows 12 as comprising an array of memory elements 20 connected together in a shift register format so that search window pixel elements introduced via input 10 are stored in a first element 20(I) and are sequentially shifted through the array of memory elements comprising 22 rows and 22 columns. The processing element array 8 is -coupled with the search window memory so that each processing element incorporates one memory element of the search window memory whereby as the pixels are shifted through the memory, they are also shifted sequentially through the processing element array.

Referring to figure 3 this shows in schematic form the construction of the current block memory 4 whereby a pixel element introduced at this input 2 is stored in a first storage element 4(I) and is thereafter shifted sequentially through the 8x8 array of memory elements connected together in shift register form. The last pixel element stored in the memory is provided to the output 6 for presentation to the processing element array.

Referring now to figure 4 this shows a single processing element of the array in more detail as including a register 30 which stores a search window pixel element and which corresponds to the memory element of the search window. A line 32 presents the pixel element on line 6 from a current block memory and a comparator 34 is provided to compute the absolute pixel difference between pixel in register 30 and the pixel on line 32. A multiplexor 36 is coupled to comparator 34 and to a block match in line 38 from the processing element immediately above the processing element in the array.

Output of multiplexor 36 is coupled to comparison logic 40 which provides outputs to a partial sum register 42 and further multiplexor 44, 46. The output of multiplexor 44 is coupled to a block match register 48 and thence to a block match output line 50. The multiplexor 46 is coupled to an address register representing the address of the processing element 52 and to a block address line immediately above the processing element shown. The output of the multiplexor 46 is coupled to an address register 56 and thence to a block address out line 58.

This in operation of the processing element, the current block pixel is compared with the search window pixel in register 30 and this result is stored in partial sum register 42. Simultaneously, a similar operation occurs in the other processing elements so that each processing element compares the current block pixel with first pixel from each of the 15x15 possible blocks within the search window of 22x22 elements. A search window of 22x22 elements is chosen as representing the likely motion of a block of video elements from one frame to the next. After this computation operation the data in the search window memory is shifted one position to the right so that the next pixel of each block is presented to the corresponding processing element, and a similar comparison operation is carried out with the current block pixel, the result being accumulated in partial sum register 42.The same process occurs for 8 shifts of the shift register memory so that a single row of each block is compared with a similar row of the current block memory.

After the results for a single row have been accumulated, the data in the search window is shifted through 15 positions so that the next row of each block within the search window memory is presented to the appropriate processing element. Comparison operations are then carried out on the next row of each block of the search window with the corresponding row of the pixel elements of the current block memory, the results being accumulated in the partial sum register 42. These operations are repeated for all 8 rows of each block, so that at the end of the comparison operation, each sum register 42 stores a value which represents the degree of matching between the current block in memory 4 and the respective block of the search memory.

It is then necessary to compare the results stored in the respective registers 42 and to this end the data from the top row of processing elements is presented on lines 38 to the multiplexor 36 of the processing elements in the row immediately beneath the top row.

A comparison operation then takes place in order to compare the sum stored in the top element with the sum stored in the elements immediately beneath it. The best match, i.e., the lowest sum is selcted and fed through multiplexor 44 to register 48 where it is stored. At the same time, the address of the processing element having the best match is stored in register 56. A similar set of comparison operations then takes place between the data stored in registers 56 and 48 and the data stored in the registers 42 and 52 of the row of processing elements immediately hereafter so that in a set of 15 comparison operations the lowermost row of processing elements will store the best match for each column of blocks.

The comparison logic of Figure 1 is then employed to compare these best matches and work out which of these best matched represents the closest match to the correct block. This is assumed to present the actual block. It is then a straightforward match to provide a vector representing the displacement of the current block to the position of the block in the previous frame.

It may thus be seen the present invention provides a chip architecture for implementing the brute force search motion estimation technique using a regular array of processing elements.

The architecture overcomes the problems associated with the large amount of data transfer required to implement a brute force search quickly. It uses novel arrangements for the search window memory and current block memory resulting in simple components that easily achieve the required data transfers. The array elements perform absolute pixel difference to obtain the block match values and comparison of the block match values. The comparison of block match values within the array is new and results in reducing the array output data.The architecture is applicable to hardware implementation. The processing element array may be located anywhere within the search window memory or it may be seperate from it. The block memory need not all be included as part of the architecture. The whole architecture may be pipelined to improve performanceThe architecture may be incorporated into a digital video coded system to reduce the transmissin bandwidth requirements of the overall codec.

Claims

1. A system for implementing a brute force search block matching technique, the system comprising signal input means for receiving time sequential sets of signals, means for storing a previous set of signals and a block of signals from a current set of signals, and an array of processing elements wherein each element is arranged to perform comparison operations on respective signal elements from a predetermined part of the previous set of signals and the block of signals from the current set of signals whereby to match in a plurality of comparison operation, said block of signals with substantially all blocks of a similar size in said predetermined area.

2. A system as claimed in claim 1 wherein said sets of signals comprise frames of video pixel element signals.

3. A system as claimed in claim 2 wherein said block of signals comprises an 8x8 block of pixel elements, said predetermined part comprises a 22x22 block of pixel elements, and said array comprises a 15x15 array of processing elements.

4. A system as claimed in claims 2 or 3 wherein the array is operative to compare a single pixel from said block with corresponding pixels in all blocks of a similar size in said predetermined area in a single set of simultaneous computation operations.

5. A system as claimed in claim 4 wherein in a predetermined number of computation operations each processing element compares each pixel of the current block with the respective pixels of one block if said blocks of a similar size, and to accumulate the results of computation operations.

6. A system as claimed in claim 5 wherein subsequent to the block comparisons, the accumulated results of the comparison operations are compared with one another in order to ascertain the best match of the current block with said substantially all blocks of a similar size, whereby to compute a motion vector.

7. A system as claimed in any preceding claim wherein the block of current signals and the predetermined part of the previous set of signals are stored in first and second shift register means respectively whereby after each computation operation of the said array, the signals are shifted at least one position in the shift register means.

8. A system as claimed in claim 7 wherein each processing element is coupled to a respective memory element of the shift register means for said predetermined part.

9. A systen for implementing a brute force search block matching technique, the signal comprising signal input means for receiving time sequential sets of signals, means for storing a previous set of signals and a block of signals from a current set of signals, and processing means arranged to perform a plurality of consecutive comparison operations on respective signal elements from a predetermined part of the previous set of signals and the block of signals from the current set of signals, wherein said block of signals and said predetermined part are stored in respective first and second shift register means to enable different elements of said block and said predetermined part to be presented to the processing means for each computaion operation by a shift of one or more positions of the signal elements in the first and/or second shift register means.

10. A system as claimed in claim 9 wherein said processing means comprises an array of processing elements.

11. A system as claimed in claim 11 wherein each element of the array is coupled to a respective memory element of the second shift register means, and to an output line of the first shift register means.

12. A system as claimed in any of claims 9 toll, wherein said sets of signals comprise frame of video signal picture element signals.

13. A system for implementing a brute force search block matching technique substantially as described with reference to the accompaying drawings.