NO175558B - Video tags - Google Patents

Description

The present invention relates to a video encoder, and in particular an encoder that uses conditional refresh whereby data representing changes in an image between partial images is sent to update a stored image at a decoder.

Transformation coding can also be used, where each part of an image to be coded typically comprises a compilation of image elements which are divided into blocks of NxM image elements. Each block of picture elements is transform coded, for example by applying a discrete cosine transform (DCT). In general, a block of NxM image elements will be transformed into an assembly of NxM coefficients. The resulting array of coefficients is then quantized by dividing each coefficient by a variable quantization factor. The quantized coefficients can be coded in a variable length code, for example Huffman code. Several codes using variable length may be available, in which case a decision is made as to which one to use for a particular coefficient, depending on the statistics of the coefficients.

In such encoding methods, data is generated at an uneven rate at the transmitter, and an output buffer is usually used to ensure that transmission takes place at an even data rate.

The degree of filling of the output buffer can be used to determine the quantization factor, so that when the buffer fills up, the quantization factor is increased. However, when the buffer is emptied, the quantization factor can be reduced.

It is also common practice not to send coefficients representing each block of each subframe of the image, but to send them only for those blocks where changes have occurred, with respect to the corresponding preceding block, that exceed a threshold. The threshold can also be determined as a function of the output buffer storage level.

Problems arise when administrative information is to be included in the data stream before the variable length coding and the transmission.

European Patent Application No. 180,345, published May 7, 1986, discloses a method and apparatus for image encoding, wherein a line sync signal, an encoding mode signal and a length code signal are multiplexed to provide a multiplexed signal which is supplied to a buffer storage where it is subjected to speed adjustment and then output to a transfer path.

A paper entitled "An Architecture for Hybrid Coding of NTSC TV Signals" by Jalali and Rao, IEEE International Conference on Communications, Denver, Colorado, 14-18 June 1991, Volume 2 of 4, Pages 22.4.1-22.4.5, IEEE New York US, shows a DPCM encoder where the contents of the block code buffer and the coefficient code buffer are read to provide data for transmission. If the size of any of the coded prediction errors in the coefficient code buffer is greater than a threshold /3, the code block is considered active, and a "1" is stored in a block code buffer together with the location codes. The block code is sent before sending the prediction errors. On the other hand, a "0" is stored in the block code buffer if the size of all the coded coefficients in the coefficient code buffer is less than /3. In this case, the block code is sent, and all the coefficient words belonging to that block are ignored. Since encoding and transmission cannot be performed simultaneously, two identical sets of block code buffers and coefficient code buffers are provided. These buffers work in a manner similar to "ping-pong".

US Patent No. 4,394,774 describes an apparatus for encoding a video signal, where a buffer following a variable length encoder provides feedback control for the preceding variable length encoder. The actual content of the buffer is thus directly proportional to the number of transmitted bits that this content represents.

An article entitled "Commercialised Interframe Codec for NTSC Color Video Signals" by Mizui and Inamura, Japan Telecommunications Review, Vol. 21, No. 2, April 1979, pages 109-117, shows a video codec in which a buffer layer is placed between a generator for video data signals and an encoder for variable length words.

In accordance with the present invention, there is provided a video encoder comprising: a generating device for generating from an input video signal data at a variable rate;

a variable length encoding means for encoding the data using a variable length code, such as a series of code words; and

a buffer between the generating means and the variable length coding means. The video encoder is characterized in that it includes a calculation device for calculating the sum of the lengths of the coded words which will be produced by the variable length coding device from the instantaneous contents of the buffer storage.

The invention will now be described by means of an example with reference to the attached drawings where: Fig. 1 is a block diagram of the first steps in a video codes; Fig. 2 shows a function block kernel above an encoder which includes a preferred embodiment of the invention; and

Fig. 3 is a block diagram of a suitable decoder.

Figure 1 illustrates the front end of a video encoder. Digital input signals are delivered via an input 1 and are differentially coded by subtraction in a subtracter 2 of a predicted picture element value from the relevant, received picture element value. The difference values are then encoded using a transformation encoder 3 where NxM (for example 8 x 8) blocks of elements are subjected to a transformation, for example the discrete cosine transform (DCT), to produce an equal number of NxM coefficients. The coefficients are quantized in a quantizer 4 and fed to an output 5. For future reference it should be noted that the quantizer 4 may be arranged to output coefficients only up to the last coefficient in a block that is different from zero. If the prediction error or prediction error is zero or small for a particular block, it may actually be that no coefficients will be output for that block. It may also vary the output sequence to reduce the number of coefficients to be encoded, for example in the manner described in International Patent Application No. PCT/GB88/00519 and European Patent Application No. 88306060.0.

The predictive coding takes place mainly from sub-picture to sub-picture, and the predictor mainly comprises a sub-picture store 6. In order to avoid errors at the receiver, however, the transformation coding and the quantization are included (in a conventional way) in the predictor loop which thus includes an inverse quantizer 7, and an inverse transformation codes 8; also a filter 9 - in that the components within the dashed lines in figure 1 form a local decoder.

With regard to the inverse quantizer 7, it is of course not possible to recover information that is lost by quantization. The inverse quantizer shall reverse any non-linearity and any other reversible coding functions performed by the quantizer. The arrangement shown in Figure 1 is adaptive, i.e. that the coding method is varied depending on the image content. This is illustrated as being carried out by a control unit 10 which is also assumed to determine which blocks have a sufficiently low difference from the previous partial image, so that they do not need to be coded by the quantizer at all. Possible variations are: - switching between differential and absolute coding (control line 11),

- variation of the filter parameters (control line 12)

- variation of the quantizer characteristics, also variation of other coding functions that may be included in the quantizer, such as variation of the sequence with which the coefficients are transferred (control line 13) - the encoder may also include motion compensation generally indicated by means of a motion compensation unit 14, and this may again be adaptive.

For the sake of simplicity, the inputs to the control unit 10 are not shown. A later part of the encoder includes a buffer to ensure a constant transmitted data rate despite the uneven rate at which the data is generated, and the buffer state is communicated to the control unit (line 15) so that the average rate of generation can be regulated to prevent buffer overflow or emptying . In addition, the control unit 10 can have inputs connected to one or more of the points marked * according to the function it is to perform. All these functions are known in and of themselves and will therefore not be discussed in detail. It is, of course, necessary that information indicating the coding type that has been used be delivered to the recipient the data is sent to, and the front-end section in Figure 1 has outputs 16 for such administrative information.

The administrative information (or block preamble) to be transmitted for a block is assumed to include:

Block position - calculated later

Mode (intra/inter-part image) - from control line 11 Filter data - from control line 12 Quantization type - from quantizer 4 Code indicating whether

motion compensation is used

Block motion vector - from motion compensation ion unit 14 Coefficient sequence - from quantizer 4.

A signal CL is also emitted which indicates whether no coefficients are emitted for the relevant block.

In the arrangement shown in Figure 1, video signals are processed in real time.

A block of NxM image elements gives rise to a block of NxM coefficients. The instantaneous rate at which the coefficients are generated is thus the video pixel rate (eg 5.37 MHz) although the average generation rate (which is variable) will be lower due to non-transmission of (a) data for unchanged blocks , and (b) final coefficients that have the value zero.

It is assumed that the administrative information is generated in blocks corresponding to the image blocks. The possibility of also generating administrative blocks for a complete image or groups of blocks within an image will be discussed later. It is assumed that variable length coding will be used for the transmission of the coefficients. The coder must

(i) perform the variable length encoding

(ii) assemble the coefficient information and the administrative information in a desired sequence or order (iii) buffer the data whose generation rate will vary due to the factors discussed above, as well as the variable length encoding, for adaptation to a desired constant output data rate.

In principle, it would be possible to buffer the coefficients and the administrative data separately and read them out at the output clock rate. For any block, however, the administrative data may be present without coefficients, or both may be absent, or (when the image or group of administrative data is present) coefficients may be present without block introductions. This together with the variable length coding makes managing the buffer filling rate a complex operation.

The next step in the coder, as shown in figure 2, therefore serves to assemble the coefficient data and the administrative data or introductions. The coefficient data and the administrative data are written into respective first-in, first-out (FIFO) stores 21, 22 at video rate. They are selectively read out at an instantaneous speed that is higher by a sufficient margin so that the stores are never overfilled. This can conveniently be done by a multiple of the clock frequency for the encoder output, which can typically be from 384 kbit/s to 1.544 or 2.048 Mbit/s, in the two latter cases a reading speed of 6.144 or 6.176 MHz can be used. The lines connecting the elements in Figure 2 are marked with numbers indicating the number of parallel data connections in the relevant connection. In some cases where there is only one connection, the transmitted signal has been indicated.

Coefficient data is received from output 5 of Figure 1 (with a "last coefficient" flag added to the last coefficient in a block). Before the data is entered into the storage, a VLC selector or word identity code is added to each coefficient by means of a VLC entry unit 23, according to the information provided by

the "quantization class" output from Figure 1. This identifies for later use which of a number of sets of variable length codes should be used for quantization of the coefficient. The coefficient data and word identity codes are written into the FIFO storage 21 by means of a write control unit 24.

Meanwhile, the administrative data is assembled into a sequence by means of a precoder 25 together with a block position code from a position calculator 26 (synchronized with the operation of the front end in Figure 1) and word identity codes to determine the VLC code set. to be used. It forwards this to a writing control unit 27 together with the signal CL.

Although not essential in principle, the encoder may provide for the encoding of administrative information with respect to a complete sub-image of the image, or of a group of blocks. These may typically include some or all of the following, namely: a picture introduction that occurs at the beginning of the picture;

a group header that precedes a group of blocks; and

a block introduction as previously discussed.

The reason for this arrangement is that it may be possible to reduce the total amount of data to be transferred, for example a single movement sector or

the quantization strategy may be sufficient for the entire image, or a group of blocks. When a whole group of blocks does not need the transmission of image information, a single introduction will be sufficient. Non-encoding-related information regarding the image can also be entered.

The introductions can typically include some or all of the following:

Image Intro-

Start code

Buffer state

Image number (a modulo n number (e.g. n = 64)

incremented by 1 for each successive image) Image type (split screen, document, etc.)

Group introduction

Group position on

Mode (intra/inter-frame)

Filter data

Quantization type

Code indicating whether information for the block should follow

Group motion vector

These are assumed to be collected or assembled in a lead-in generator 28 and delivered to the writing control unit 27.

At the beginning of each group of blocks, the controller 27 receives a "group of blocks" preamble sequence. At the beginning of each frame, the unit 27 receives a frame-in lead sequence. This occurs while the FIFO storage 21 is filled up with the first block of coefficient data and the encoder 25 calculates the administrative data for the first block. The unit 27 stores the administrative data for image and group in the FIFO storage 22 together with their word identity codes

(the word identity code indicates to the VLC encoder which special administrative data the word represents).

At the beginning of the next block, i.e. the second block in the relevant group, the control unit 27 receives the data from the precoder 25 and then determines according to instructions in the data received by the precoder 25 and the "group of blocks" and "image" preamble sequences, which of the administrative data shall are transferred and write them into FIFO storage 22 along with their associated word identity codes. (For example, motion vector formation may be omitted when the motion compensation code indicates that no motion vector is required, or when the information is contained in the "group of blocks" preamble. If the signal CL indicates that the coefficient data has been generated for the block, unit 27 then writes a unique switch word to FIFO storage 22 and indicates that coefficient data is to be entered at that point in the data stream, and then writes an end-of-block code.

There are four possible combinations of coefficient data from FIFO storage 21 and administrative block data from FIFO storage 22: (i) no administrative data and no coefficient data, in which case nothing is stored in either FIFO storage 21 or FIFO storage 22 for that block. (ii) administrative data plus coefficient data, in which case administrative data, a switch word and an end-of-block code are stored in FIFO storage 22 and coefficient data are stored in FIFO storage 21. (iii) administrative data and no coefficient data, i in which case administrative data is stored in FIFO storage 22, but no switch word, end block code or coefficient data is stored. (iv) coefficient data and no administrative data; this can occur when the image or group introduction contains sufficient information to characterize the relevant block. In this case, coefficient data is stored in the FIFO storage 21, and only a switching word and a block end code are stored

in the FIFO warehouse 23.

The administrative data is written into the FIFO stores 21, 22 using clock signals <pw for the video rate, readout is effected using a higher frequency clock signal øR when a read command is received from a control unit 29.

The purpose of the readout control unit 29 is to read data from the two FIFO stores in order to insert the two data types in a desired sequence in the following way: An image preamble is followed by a block group preamble or another image preamble.

A block group preamble is followed by administrative block data or another block group preamble.

Administrative block data is followed by a coefficient block (in which case the data contains the word switch) or by additional administrative data (switch absent).

A group of coefficients is followed by administrative block data or by a further group of coefficients.

To achieve this, the control unit 29 contains a read pointer that can be switched between a first state where it reads data from FIFO storage 22 and a second state where it reads data from FIFO storage 21. The pointer is represented by way of illustration by a rocker 2 01 which when reset (first state) or set (second state) enables the passage of a read command to the relevant storage via port 292 or 293.

Initially, the device is set in the first state, and administrative data is read out and fed into a buffer store 30 until occurrences of a switch word, indicating that coefficient data is required for a block, are recognized by a switch word detector 294 which switches the device to the other condition. Then, coefficient data is read from storage 21 to buffer storage 30. This continues until all the coefficients have been read and a "last coefficient" flag is detected by a flag detector 295 which resets the unit to the initial state, whereby further administrative data can be read. The control unit 29 then continues in this way to read from the two stores. If any of the stores are emptied, this is signaled to the control unit, which delays a possible read command to the relevant store until the signal disappears.

Thus, when administrative data is stored in the FIFO storage 22, but no coefficient data is present regarding that block, then no switching word is stored by the write control unit 27 or read by the control unit 29, and thus the latter continuously reads two or more consecutive blocks of administrative data. When, on the other hand, a block of coefficients is stored in the FIFO storage 21, but there is no associated administrative data, then a switching word and a block end code will be written into the FIFO storage with respect to each such coefficient block. This means that the controller 21 will respond to each switch word and read a block of coefficients from the FIFO store 21. At the end of each block, when the "last coefficient" flag is received, the controller 29 will return to read the FIFO store 22 until the next switching word occurs.

The administrative data and the coefficient data have now been collected in the required sequence or order, although they may contain gaps that occur when the FIFO stores 21, 22 become empty. Conventionally, conversion to variable length code (VLC code) can then be performed followed by entry into buffer storage 30 for buffering into a continuous data stream for output, but buffering of variable length words requires complex shift and control arrangements for storage and retrieval of words of different lengths using a fixed word size store. In Figure 2, the buffer storage is therefore first carried out by means of the buffer storage 30 already mentioned, which is followed by a variable length encoder 31 and a parallel in, serial out register 32 (PISO register).

The data read from the FIFO stores 21, 22 are stored in the buffer store 30 under the control of the control unit 29 (apart from the switching word, which is no longer used). Since a constant bit rate is required at the output of the PISO register 32, the buffer store is not read at a constant rate; readout is controlled by a counter 3 3 as explained below. It will be remembered that each coefficient (or piece of administrative information) is followed by a word identity code that identifies which set of variable length codes is to be used. The data and word identity codes are supplied to the variable length encoder 31 (which can simply be a look-up table in the form of a read store) which outputs the coded data words to the register 32. They are also supplied to a further look-up table, a length detector 34, which transmits the relevant word length to the counter 33 to determine the number of output clock pulses øL to be delivered to the register 31. The counter also serves, when the last clock pulse has been sent to the register 32, to issue a read command for reading out the next data part ( and its word identity code) from the buffer store 30.

To enable continuous transmission at the fixed data rate determined by PISO 32, it is necessary to include feedback to the encoder of Figure 1 (and quantizer 4 in particular) to regulate the data generation rate. Conventionally, this is done on the basis of the degree of filling of the output buffer storage. Some decoders also require transfer of the transmitter buffer state to assist synchronization operations at the decoder, as described in European Patent No. 0103380 and US Patent No. 4,562,466. Because the variable length code in the present case follows the buffer store, the relevant data content of the buffer storage not directly related to the number of transmitted bits that the content represents. Therefore, it is necessary to determine what the buffer contents "would" be if they were in variable-length encoded form. This is achieved in the following way: First, a length detector 35 (identical to the detector 34) is connected to receive data and word identity codes delivered to the buffer input. The lengths it produces are summed in an accumulator arrangement consisting of an adder 36 and a latch circuit 37 which gives a running sum of the number of coded bits represented by the data entered in the buffer storage.

The instantaneous value of this total is stored in the buffer storage together with the most recent data part. When a chunk of data is read out, the associated sum now indicates the total number of output bits represented by the buffer's data readout. The difference between this and the entered sum (formed in a subtracter 38) is the "theoretical" buffer filling level which is communicated to the control unit 10 (via line 15 in figure 1), and can also be sent to the FIFO write control unit 27 if buffer state information is to included in the transferred administrative information. As shown, only the eight most significant bits of the sum are buffered and delivered to the subtracter. This provides a sufficient indication to prevent buffer overflow/emptying assuming a suitable buffer size.

In the FIFO storage 21, the transformation coefficients can be coded in either a one-dimensional or a two-dimensional variable length code. If one-dimensional variable length coding is used, each coefficient in each block, i.e. from "block start" to the last non-zero coefficient written into the FIFO store 21. For each coefficient, the word identity code generated in the code selector unit 23 is also written into the FIFO store as previously explained (this shows the VLC encoder which VLC code set to be used for the coefficient in question). If two-dimensional variable length coding is used, the unit 2 3 additionally calculates the trailing length of zeros before each non-zero coefficient, and writes this into the FIFO storage 21 together with the coefficient size as a single word each time a non-zero coefficient, be reached.

Although the encoder described above can be used with a conventional decoder, the decoder shown in Figure 23 reflects some of the techniques used in the encoder. Video data is received by means of a line matching unit 40 synchronously with a line clock of 2.048 MHz (or 1.544 MHz). The data is processed and passed on to a synchronization detector 41 together with indication of any line errors detected by the line matching unit. The synchronization detector 41 continuously scans the received data stream for a picture start code (PSC) indicating the beginning of data for a new picture. If the code is incorrect, an error flag is set. The combination of the PSC error flag and the line error flag is fed to a clock generator unit 42.

At the beginning of a new picture, the serial data is received by a variable length encoder unit 43 and is decoded from fixed or variable length data to fixed length data and output in parallel form. For each data part, a word identity is generated by a sequencing unit 44. This has a read storage 441 which contains a sequence of stored words, a word readout being fed back via a latch circuit 442 (clocked by means of an output from the variable length decoder after each word is been decoded), to form the address for the next word. The storage is programmed so that after resetting to a base address by means of a frame start code, it produces a sequence of word identity codes corresponding to the transfer sequence described earlier. When the sequence may vary, the occurrence of words indicating the presence or absence of the following optional fields is recognized in a recognition circuit 443 which directs the entry of additional address lines in the memory 441 to change the sequence. Similarly, detection of the beginning of the coefficient data causes the sequencing circuit to enter a loop where the relevant word identity repeatedly addresses itself, until the recognition of an end-of-block code causes a further sequence change. The word identity is (along with a run-length value for two-dimensional variable length code coefficients) passed to a buffer adapter 45. If different variable length code sets are used for different parameters, word identity labels can be used to signal the VLC decoder 33 (via line 444) for to select the correct code sets.

The parallel output data from the variable length encoder together with the word identity encoder are resynchronized in a buffer adapter 45 to a multiple of the line rate, i.e. 6.176 MHz (or 6.144 MHz for a line speed of 1.544 Mbit/s). The data is then sent to a buffer unit 47 together with a buffer write-open signal 45 (BWE signal). A further function of the buffer matcher is to extract the frame number previously mentioned. This number is fed to an image repetition control unit 46.

The buffer unit 47 acts as a rate converter or elastic layer that adapts the incoming data that arrives at a uniform rate, but becomes uneven after VLC decoding, to the image-related data used at a different uneven rate. A calculation of the buffer filling degree or buffer state is made in a calculation unit 48 which is identical to the units 35 to 38 in Figure 2, and passed to the clock generator unit 42. The last frame number read out by the buffer unit 47 is also extracted in this the unit and taken to the image repetition control unit 47.

The clock generator unit 42 generates a clock signal related to the picture element speed (eg 5.37 MHz) and picture related synchronization signals. The nominal clock frequency is generated as a normal value and is always used if line or PSC errors have been detected. However, it can

is varied by the value of the buffer state.

Data is read out of the buffer unit 47 under the control of a demultiplexer unit 49, signaled by means of a buffer read opening signal. The speed with which the data is extracted from the buffer unit 47 is determined by the image data content. at the beginning of a "picture" or a "block group" synchronization signal from the clock generator unit, the demultiplexer 49 reads data from the buffer unit. The word identity of the data contents is checked as they are read, and as soon as an image start code is detected, further readings from the buffer unit are blocked until the next image signal or block group signal. The input data is stored in the first-in, first-out storage, synchronized to 6.176 MHz (or 6.144 MHz) and the readouts from the FIFO stores synchronized to a pixel rate of nominally 5.37 MHz. There is a FIFO storage 490 for the administrative image data, 491 for the coefficients, and 492 and 493 for the word identities of the administrative data and coefficients, respectively.

A word identity decoder 494 determines from the word identities read out from the buffer storage, whether the associated data word is coefficient data or administrative data, and controls a demultiplexer 495 to direct these and the associated word identity to the correct FIFO storage. The output from the coefficient data FIFO store 492 is reconstructed into blocks (496) by means of a word identity decoder 497 and fed to the picture data input of the appropriate decoder, 50, which may be of identical construction to the local decoder shown in the block with broken lines in Figure 1. A further word identity decoder 498 serves to discern from the output of the word identity store 492 whether the output of the FIFO store 490 is a motion vector, filter, quantizer index, class, etc. and routes it via a demultiplexer 499 to the appropriate unit in the decoder 50.

The frame repetition control unit 46 checks that there are at least two frame start codes in the buffer unit 47 ready for the demultiplexer 49 to process a complete frame. If this condition is not met, the frame repetition controller issues a frame repetition command to the demultiplexer and instructs it to stop reading the buffer store and wait for the next frame synchronization signal before continuing its processing. The result is that the reconstructed image does not

is updated with new information and is effectively repeated.

Claims (6)

1. Video codes, comprehensive a generating means for generating from an input video signal data at a variable rate (1-29); a variable length encoding means for encoding the data using a variable length code, such as a series of code words; and a buffer storage (30) between the generating device (1-29) and the variable length coding device (31), characterized in that the encoder includes a calculation device (35-38) to calculate the sum of the lengths of the coded words that will be produced by the variable length encoding device (31) from the current content of the buffer storage (30). 2. Video codes according to claim 1, characterized in that the calculation device (35-38) includes a device (35) for, for each data record entered into the buffer storage (30), to determine the length of the corresponding coded word in accordance with the variable length code, an accumulation device (36) for to accumulate the sum of the thus determined lengths, and a subtraction device (38) to subtract from this sum the sum of the word lengths corresponding to the data records which have been read from the buffer storage (30). 3. Video codes according to claim 2, characterized in that it includes a control device (29) arranged to store in the buffer storage (30), together with a data record, the aforementioned sum of the thus determined lengths, and to read out this sum from the buffer storage (30) when the data record is read out, thereby providing the sum to be subtracted for the subtraction device (38). 4. Video codes according to one of the preceding claims, characterized in that it is designed to send the calculation device's determined sum of lengths as administrative information. 5. Video codes according to one of the preceding claims, characterized in that the generating device (1-29) is arranged to vary the data rate of the video signal under control of the calculation device on the basis of its determined sum of lengths. 6. Video encoder according to one of the preceding claims, characterized in that it further comprises: a first coding device (1-14) for generating, for blocks of image elements in an image to be coded, image data indicating the state of these elements; a second coding device (25) for generating administrative data with regard to blocks and/or groups of blocks with image elements, where the administrative data includes information about the coding type used by the first coding device (1-14); a first further buffer storage (21) for storing the image data; a second additional buffer storage (22) for storing the administrative data; a device for introducing flags into the first and second additional buffer stores (21, 22); and a device for selectively reading out data from either the first additional buffer store (21) or the second additional buffer store (22) for writing into the buffer store (30), the readout device responding to the appearance of a flag in the additional buffer store (21, 22 ) which is read, by ending reading from this buffer storage (21, 22) and starting reading from the other storage (22, 21), so that a desired sequence of image data and administrative data is obtained.