AU620120B2

AU620120B2 - Coding and decoding of high definition television images

Info

Publication number: AU620120B2
Application number: AU38580/89A
Authority: AU
Inventors: Philippe Guntzburger; Mohammad-Reza Haghiri; Marcel Le Queau; Frederic Ponsalas
Original assignee: Philips Gloeilampenfabrieken NV
Current assignee: Koninklijke Philips NV
Priority date: 1988-07-08
Filing date: 1989-07-05
Publication date: 1992-02-13
Anticipated expiration: 2009-07-05
Also published as: CN1018045B; FI901176A0; FR2634086A1; PT91091A; DK59090A; JPH03500357A; WO1990000846A1; AU3858089A; DK59090D0; EP0350122A1; CN1043061A

Description

:ii:

Q

L

li-ilii~i-

I

AUSTRALIA

PATENTS ACT 1990 GQ4FPLEIE ;SIEGIFIC ATION FO Q ThNDAELD RAcP N T 0

S.

0 0O

OS

S

*SS S

S

S@ 00

S

=I c INAL Name of Applicant: Actual Inventors: N. V. PHILIPS' GLOEILAMPENFABRIEKEN MOHAMMAD-REZA HAGHIRI, PHILIPPE GUNTZBURGER, MARCEL LE QUEAU, and FREDERIC FONSALAS S 0 0 Address for Service: PATENT AND TRADE MARKS DIVISION, PHILIPS INDUSTRIES HOLDINGS LIMITED 15 BLUE STREET, NORTH SYDNEY, NSW 2060 Invention Title: "PROCESSES AND DEVICES FOR THE ENCODING AND FOR THE DECODING OF HIGH DEFINITION TELEVISION IMAGES AND SYSTEMS FOR THE TRANSMISSION OF TELEVISION IMAGES INCLUDING SUCH DEVICES." The following statement is a full description of this invention including the best method of performing it known to us.

i i

B

i II

I

ji i i a PHF 88.572 28.8.91 10 210O &se 3. 0 T 0 0

SS

1 3/TO "Processes and Devices for the encoding and for the decoding I of high definition television images and systems for the transmission of television images including such devices" Description The present invention relates to a method of encoding image signals, comprising the steps of analysing said image signals to obtain motion vector data relating to motion between successive images of said image signals; bandwidth reducing encoding said image signals to obtain bandwidth reduced encoded image signals.

This method can be used, especially, in a system for the transmission of high definition television images, comprising a stage for the emission of encoded data i representative of the said images and, after transmission of these data at a determined field frequency and by means of a first channel having a limited passband requiring a processing to reduce the quantity of data to be transmitted, a stage for the reception of the transmitted data.

The present invention likewise relates, to an encoder for encoding image signals, comprising an encoder circuit having an input coupled to an input of said encoder, an image signal output and a motion data output, the encoder circuit itself including: means for analysing said image signals having an input coupled to said input of said encoder circuit and an output coupled to said motion data output to supply motion vector data relating to motion between successive images of said image signals; and means for bandwidth reducing encoding said image signals having an input coupled to said input of said encoder circuit and an output coupled to said image signal output to supply bandwidth reduced encoded image signals.

The present invention likewise relates to a method of decoding image signals, comprising the step: of processing bandwidth reduced image signals in accordance with received vector motion data relating to motion between successive images of said image signals to obtain decoded image signals, This metiod can also be used in a system as defined hereinabove.

1 1 I- 1 i l 1 1 PHF 88.572 2 28.8.91 The invention likewise relates, to a decoder for decoding image signals comprising a circuit for processing bandwidth reduced image signals in accordance with received motion vector data relating to motion between successive images of said image signals, said processing circuit having an output to supply decoded image signals.

These methods, encoders and decoders according to the invention can be used, especially, in a system for the transmission of high definition television images in accordance with the MAC standard. It will be recalled here ee• that the said transmission is ensured by means of an analog channel which carries the data having undergone the ad. compression and that with this analog channel there is 0.associated an auxiliary channel referred to as the digital 1 5 assistance channel permitting the transmission of e complementary data relating to the movements of the images transmitted by the analog channel.

In order to reduce the passband of the television signals with a view to their transmission, solutions have t 2 been proposed to maintain, in spite of the compression of 0. data which is necessary in order to match the quantity of these data to the limited passband of the transmission channel, a satisfactory spatial resolution in spite of the O movement of the images to be transmitted.

These proposals have lead to the availability of the best spatial resolution possible, irrespective of the speed of displacement of the content of the transmitted images.

However, it is found at the emission stage that i the succession of the compressed images, referred to as compatible images, may be affected by residual defects such as defects due to the subsampling and which manifest themselves in the form of jerks, due to the frequency of Hz of the images, in image regions processed by the branch with compensation of movement.

**It is an object of the invention to permit the I elimination or at the very least the reduction of such defects, while continuing to ensure in an effective manner' the reconstruction of the movement between successive 354AY PHF 88.572 3 28.8.91 compatible images. 1 4 To this end, the invention provides a method of encoding as defined in the preamble, characterized in that it likewise comprises a step of: temporally filtering said ba-1width reduced encoded image signals in accordance with said motion vector data to obtain filtered encoded image signals.

The invention also provides an encoder as defined in the preamble, said encoder being characterized by: a 10 circuit for temporally filtering said bandwidth reduced @000 encoded image signals in accordance with said motion vector data, said temporally filtering circuit having a motion data i.0 input coupled to said motion data output, an image signal o input coupled to said image signal output, and an output to supply filtered encoded image signals.

g In an advantageous embodiment, this encoder is characterized in that the said temporally filtering circuit comprises a field delay circuit having an input coupled to said image signal input of said temporally filtering 0000o0 circuit, a displacement correction arrangement coupled to O the field delay circuit, a weighted filtering arrangement Socoupled to the displacement correction arrangement, and a 000000 S multiplexer having a first input coupled to an output of oe Ssaid weighted filtering arrangement, a second input coupled 25 to said image signal input of said temporally filtering *00000 circuit, and an output coupled to said output of said S* temporally filtering circuit.

The invention provides also, in a corresponding manner at the reception state, a method of decoding as defined in the preamble, characterized by the prior step of: temporally filtering said image signals in accordance with said received motion vector data to obtain said bandwidth reduced image signals, whereby a field frequency of said image signals is equal to a field frequency of said bandwidth reduced image signals.

The invention also provides a decoder as defined in the preamble, said encoder being characterized by: a circuit for temporally filtering said image signals in 1 accordance with said received motion vector data, said t/2Q U L A4 10 0 See.

0* 0 00 00 SO 0 SOO S se a

S

2 0*0S 0 5 0 temporally filtering circuit having an image signal input coupled to an input of said decoder, a motion vector data input coupled to said processing circuit, and an image signal output coupled to an input of said processing circuit to supply said bandwidth reduced image signals, whereby a field frequency of said image signals is equal to a field frequency of said bandwidth reduced image signals.

According to an advantageous embodiment, the said decoder is characterized in that the said temporally filtering circuit comprises a field delay circuit, a displacement correction arrangement coupled to the field delay circuit, and a weighted filtering arrangement coupled to the displacement correction arrangement, said weighted filtering arrangement having an output coupled to said output of said temporally filtering circuit.

The particular features and advantages of the invention will now become evident in greater detail in the description which follows and in the accompanying drawings, which are given by way of non-limiting examples and in which: Figure la and lb show respectively the encoding part on the emission side and the decoding part on the reception side of a device for the estimation and for the compensation of movement for a system for the transmission of television images; Figures 2 and 4 show respectively the encoding part on the emission side and the decoding part on the reception side of another embodiment of a device for the estimation and for the compensation of movement for a system for the transmission of high definition television images; Figure 3a shows in greater detail an example of a stage for the estimation of movement in the device of Figure 2, Figures 3b and 3d explain the content of the estimation circuits of the stage of Figure 3a, and Figure 3c shows an illustrative embodiment of the cells making up these estimatii' circuits; Figures 5 and 6 show variant embodiments incorporating the devices of Figures 2 and 4 respectively; Figures 7 shows, in a system for the i iL-ii ii 4:: \t r -I Y-

H

PHF 88.572 28.8.91 0666 @ego o* 0 66 3 O 6 5 6 66 :1.

6I 66 61 6 2.5

S

6 transmission of high definition television images, an illustrative embodiment of an encoding device with which the temporal filtering circuit according to the invention may be associated; Figures 8a to 8c represent the three processing branches of the device of Figure 7, Figure 8d represents a variant of the branch of Figure 8c, and Figures 9a to 9c represent the corresponding image formats at the output of the said branches; Figure 10 shows the decision-making circuit of the eevice of Figure 7; Figure 11 shows an illustrative embodiment of a decoding device with which the inverse temporal filtering circuit according to the invention may be associated; Figure 12 shows a variant embodiment of the encoding device of Figure 7: Figures 13a to 13c are a detailed representation of the three processing branches of the device of Figure 12, and Figure 14 shows the temporal filter of the third of these processing branches; Figures 15 and 16 are a more detailed representation of the movement estimation circuit and the decision circuit of the encoding device of Figure 12; Figure 17 shows the modified decoding device associated with the variant encoding device of Figure 12; Figures 18a and 18b show respectively an encoding device incorporating a temporal filtering circuit according to the invention and a decoding device incorporating an inverse temporal filtering circuit according to the invention; Figures 19 and 20 are a more detailed representation respectively of the said temporal filtering circuit according to the invention and the said inverse temporal filtering circuit according to the invention; Figure 21 shows examples of values of the weighting coefficients depending upon the amount of the movement in the images; Figures 22 and 23 show variants of the direct and inverse temporal filtering circuits according to the e ~Ii: i i i!; i:

II

Y'-4

BI

A7V

IT

Y

PHF 88.572 6 28.8.91 I invention.

The device shown in Figure 1 in the case of application to a high definition television image transmission system comprises an encoding part on emission (Figure la) and a decoding part on reception (Figure ib), which cooperate in order to detect and to estimate the movement within the images to be transmitted and to match the processing of the image data to the greater or lesser magnitude of this movement. The images are detected here by a high definition television camera (not shown) which 0* analyses the scene by means of an interlaced or sequential scanning at 1250 lines and at a rhythm of 50 images per second. The camera does, of course, after matrixing of the 00 signals R, G, B, supply three types of signals: the luminance signal Y and the two colour difference signals U and V (or chrominance signal). Consequently, the description relates, for example, to the luminance signal but would be equally applicable to the chrominance signal.

It will therefore simply be stated that the output signal of e2b the camera is sampled, and that the resulting samples are o presented at the input of the encoding part at a cycle of 54 6: Megahertz in the case of an interlaced scanning and of 108 S* Megahertz in the case of a sequential scanning. As the transmission channel, in the case of the MAC standard, accepts only a rate of 13.5 Megahertz, a subsampling must be carried out prior to the said transmission.

It will be stated that the images could, in the j limiting case, be processed point by pint, but that it is simpler to break them down into N blocks of m x n points.! Instead of operating a scanning of the points, there is then performed a scanning of the blocks, to which N points representative of the said blocks correspond. i The estimation of movement mentioned hereinabove I is provided in the following manner. The encoding part i" shown in Figure la is constituted by a plurality of parallel branches, for example three, referred to here as 1, 2 and 3.

These branches receive the samples formed as indicated Shereinabove and each comprise a prefiltering circuit 101 and -a subsampling circuit 102. Although the sampling structures

<NT

PHF 88.572 7 28.8.91 are different from one branch to another, the subsampling rates are identical and, in this case, are equal to 4 for a succession of interlaced images and to 8 for a succession of sequential images. The characteristics of the filters are chosen such that foldover due to the subsampling is avoided.

The outputs of the three subsampling circuits are passed to a switching circuit 103 which, depending upon the instruction received on a fourth input connection 105, ensures the selection of one or of the other of the said outputs with a view to its subsequent transmission via the analog channel 10 of the transmission system.

The instruction present on the connection 105 of S. the switching circuit 103, which is normally identical for all the points of a block, is determined by a decision circuit 106, on the basis of a criterion which is in general linked either to a quantity measured on the basis of the input image or to the energy difference between the processed images, emanating for example from the circuits 101, and the original image. In the first case (a priori decision), the quantity measured may b, for example, the movement or the speed of the objects present in the image, and the decision is made directly as a function of the value adopted by this quantity. In the second case (a posteriori decision), the energy difference permits the determination 245 of the branch which leads to the best image reconstruction by means of the samples transmitted, and therefore the performance of the switching consequentially. The information delivered by the decision circuit 106 is passed to a channel 20 referred to as the digital assistance channel.

Similarly, on reception, the corresponding decoding part, represented in Figure Ib, comprises first of all a demultiplexing circuit 152 which, on the basis of the signal transmitted via the analog channel 10, supplies to three parallel postfiltering circuits 153 images having appropriate regular structures. Finally, a multiplexing circuit 154 receives the outputs of these postfiltering circuits and permits, on the basis of the multiplexed /A signal, the generation of an image which can be, visually TI A Y 7 T. v iTy PHF 88.572 8 28.8.91 displayed on a high resolution screen 155. The signal i transmitted via the digital assistance channel 20 is supplied in parallel to the circuits 152 and 154.

The subsampling structures, which differ from one branch to another as has been seen, may be purely spatial or alternatively may permit, in addition, the elimination of a certain number of images in the temporal direction. There is then available a proportionately higher number of samples to represent the spatial contents of the images but, on the other hand, in the case of movement, the temporal contents reflecting this movement are degraded.

Two types of degradations may be more particularly found in the reconstructed image: on the one hand, the uniform movements are impaired, the objects moving in jerks, and on the other hand the resolution suddenly diminishes as i soon as a fixed object starts to move, by reason of the prefiltering and the postfiltering. Both of these defects are visually very troublesome, and the presence of an estimation and of a compensation of movement provides a very significant remedy for these, by eliminating the jerks and by maintaining the resolution for a broader range of speeds.

The principle of compensation of movement is the following. In the sequence of images which is under OOS**g consideration, one image in two is first of all eliminated (that is to say, the spatial information available at a determined instant Thus, if the rate or temporal frequency is, for example, 1/T where T corresponds to the time interval separating two successive images, the time interval after temporal subsampling will be 2T. Using 2k-l, 2k, 2k+l, etc. to designate the successive ranks of the original sequence of images, this means that the images associated, for example, with the instants t+(2k-1), t+(2k+l)T, etc. or images in this case of odd rank, are eliminated.

In parallel with this elimination of images, movement data are determined by a movement estimation method providing for the allocation to each block of the images to be eliminated, in this case the odd images, of a Sa LI\ displacement vector D' such that the error of reconstruction M ,6 M PHF 88.572 28.8.91 10 *see #see o

B

a o 6*000: a 2 0* to *a 9

S

T

of the block is a minimum. These movement data are then used, on reception, to reconstruct the images eliminated before transmission, each block being reconstructed on the basis of the mean of the data of two consecutive images in the direction of the movement associated with the block. In this way, the defects due to the imprecision of the estimator are limited when the latter used, up to the present time, the solution of a reconstruction of an odd field (or an even field respectively) only on the basis of the preceding field of opposite parity, a solution which exhibited the disadvantage of deforming the contours of solid objects.

This principle of compensation of movement is used hereinafter, in the encoding part of the emission stage of a system for the transmission of television images, under the following conditions, in the case of the example described.

In the text which follows, the spatio-temporal subsampling which is performed permits the obtaining of a sampling rate of 4 (2 spatial, 2 temporal) for a succession of interlaced images and of 8 (4 spatial, 2 temporal) for a succession of sequential images. The movement estimator which will be used is based on the method referred to as b.ock matching (with a search range equal to: horizontal displacement vertical displacement Nevertheless, this choice is not limiting and an estimator of a different type could be used. It will likewise be stated here that the branches i, 2, 3 correspond to different speeds of displacement in the images, the base interval between images being, in the example described, of 20 and 40 milliseconds for the branches 1 and 2 respectively and the base interval between elementary points transmitted having the same spatial position in the image being of 80 milliseconds for the branch 3. It is quite clearly in the branches 2 and 3 that there is benefit in maintaining the best resolution, the movement being slower than in the case of the branch 1.

First of all, a description has been given of the application to the branch 2, the extensions to the branch 3 being described further on.

output, the encoder circuit itself including; means for analysing said image signals having an i input coupled to said input of said encoder circuit and an output coupled to said motion data output to supply motion vector data relating to motion between successive images of Si^ said image signals; and t il l l l l i' l i t i" *1*2 1 1 1 kd III PHF 88.572 28.8.91 0 0 a a a 0o 00 0 a 6 s. a 30 6 0 *a O C AA 7(* Vv J According to the example of Figure 2, the device shown comprises firstly, on emission, for the branch referenced 2, a spatial prefiltering circuit 201 receiving the input image, which is either a 50 hertz interlaced image, 2:1, 1250 lines, 54 M-samples 54 .106) per second, or a 50 hertz sequential image, 1:1, 1250 lines, 108 Msamples per second. This circuit 201 permits the obtaining of a sequential image which is limited in spatial bandwidth in order to avoid foldover due to the temporal subsampling.

The circuit 201 is followed, in the series, by a temporal sampling circuit 202 which divides by two the temporal cycle of the image (there is therefore a transition to 25 images per second), and then by a spatial subsampling circuit 203 which permits the reduction of the number of samples in each image plane (for example by means of a quincuncial line subsampling which eliminates one point in two). At the output of tt. circuit 201 there is likewise provided, in parallel the series connection of the circuits 202 and 203, a -ment estimation stage 204 which will now be describo detail.

The purpose of the movement estimation undertaken by the stage 204 is to determine for each block of the image of rank 2k+1 which is eliminated, a displacement vector D such that it is possible to obtain an approximation of the said eliminated image on the basis of the half-sum of the non-eliminated images which surround it, in the present case on the basis of the half-sum of the images 2k and 2k+2. In this case, this approximation is expressed by the relation which is given in the appendix, as are the other mathematical expressions which appear in the following part of the description. In this relation, X designates the current block of the image 2k+1, D the movement vector when the movement estimation has been applied to the images 2k and 2k+2, and I the approximation of the intensity of the point X of the current block of the image 2k+1.

This objective may likewise be formulated by stating that it is desired to associate with each block X of the image 2k+1 a vector Dx such that the expression is a minimum (this expression, in which DFD originates from the

I

ii

I

I T v> Si -It I PHF 88.572 28.8.91 T, 9'.

an 0 6 D p o o o o0 C o e p c.rpc expression "Displaced Frame Difference", is the error of approximation associated with the current block and is equivalent, for this block, as indicated by the expression to the sum cf the squares of the DFD approximation errors over all the points of the block). This known principle of examination of the correlation between blocks (described, in particular, in the article by J.R. Jain and A.K. Jain "Displacement measurement and its application in interframe image coding", which appeared in the journal IEEE Transactions on Communications, vol. COM-29, No. 12, December 1981, pages 1799 to 1808) is implemented, in the movement estimation stage 204 described here, in two steps which are distinct but extremely similar.

The movement estimation stage 204, shown by way of non-limiting example in Figure 3a, comprises, for the performance of these two steps, a set 340 of two estimation circuits 300 and 350 which are virtually identical, as well as three series image memories s41, 342 and 343. The estimation circuit 300, for example, comprises as indicated in Figure 3b which gives a more detailed representation thereof, nine identical cells 301 to 309 determining nine distortions (or estimation error as defined by the expression or the expression relating to the nine following displacements (D

X

Dy) These nine displacements are stored in a memory 345. Each one of the cells 301 to 309 itself comprises identical elements, and these elements, considered for example for the first of the nine cells, comprise, as indicated in Figure 3c showing this first cell, an adder 311, which is intended to form the half-sum of the images of rank 2k and 2k+2, a subtractor 321 and, in series with these two elements, a squaring and summing circuit 331. The adder 311 receives the output of the image memories 341 and 343, and the half-sum of the images 2k and 2k+2 thus obtained is supplied to the, for example, positive input of the subtractor 321 which receives on its input of opposite sign the output of the image memory 342 storing the image of rank 2k+1. The output of the subtractor 321 is supplied to the squaring and summing processed images, and a stage of time filtering weighted according to said motion information. The decoding method comprises a processing stage by decompression of the transmitted images according to one or the other of the three transmission rates and restitution of said decompressed images to the high definition format, the restitution of transmitted images according to one of the transmission rates being carried out from motion information also transmitted by a second channel so called digital assistance channel, a stage for selecting one amongst three images thus treated according to the decision signal transmitted through the second channel, and a prior stage for a reverse time filtering weighted according to said motion information.

4 1 1 1 {i i- w n esks i ia

I

NP

PHF 88.572 28.8.91 o oa a e o 0 0 0 a 900 0 a o oe o 0 0 2 5 )4 30 ao M o 4 2 a o 30

T

ex^ i V\ 41 i: f circuit 331, the output of which constitutes the output of the cell 301.

The nine respective outputs of the nine cells 301 to 309 are then supplied to a circuit for the comparison of the distortions 344, which compares the nine distortion values thus originating from the nine cells and determines the one which is the lowest. The one of the nine displacements which leads to this minimum block distortion is refereed to as Dmin and, after extraction from the memory 345, is passed to the second estimation circuit 350. This circuit 350, represented in Figure 3d, resumes precisely the same operations as the estimaton circuit 300, but on nine other values of displacements which are the following:

(D

x D Dminl+ Dmin Din 1 Din Dinl Din Dmini Dmin Dinl and which are on this occasion stored in a memory 395 receiving Dminl The estimation circuit 350 comprises nine cells 351 to 359, which are themselves composed of identical elements which, considered, for example, as previously for the first of the nine cells, comprise an adder, a subtracter and a squaring and summing circuit, which are provided in series. The outputs of the nine cells 351 to 359 are supplied to a circuit for the comparison of the distortions 394 which determines the lowest distortion and permits the selection of the corresponding displacement Di2 that is to say of that which minimizes the distortion for the current block X of the image 2k+1.

The displacement selected in this manner is then passed to the digital assistance channel 20, while the output of the spatial subsampling circuit 203 is passed to the switching circuit 103. On reception, the device for the compensation of movement receives a sequence of images at a rate of 25 images per second, these images being spatially subsampled, and reconstitutes a sequence of images of 50 images per second, 1250 lines, 1440 points per line, either in interlaced format 2:1 with 54 105 samples per second, or in sequential format with 108 10 6 samples per second.

I I i ii ii I i l r I j 1 b~ 0 .i~ BR Brdil JP Japon SD Soudan SCA Canada KP Rpublique populaire dmocratique SE SuMdd CF Rpublique Centraicaine de Cor&e SN Sn6gal CG Congo KR Rpublique de Core SU Union oviitique CH Sumue 11 Liechtenstein TD Tchad CM Cameroun LK Sn Lanka TG Tog DE Acemagne, R6publque id6ale d' WJ Luxmbourg US EttaUns d'Amtique DK DanemaLk M onaco PHF 88.572 13 28.8.91 This device for the compensation of movement on reception comprises firstly, as indicated in the illustrative embodiment of Figure 4, a spatial postfiltering circuit 401 carrying out a spatial interpolation to obtain a sequence of 25 images per second, 1250 lines per image, 1440 points per line. This circuit 401 is followed by a delay circuit 402 (the delay provided is in this case milliseconds), then by a switch 403 which permits, on the basis of two image sequences each having a period of milliseconds but offset by 20 milliseconds, the co reconstruction of a sequence having a period of milliseconds. At the output of the spatial postifltering 000 a 'circuit 401 there is moreover provided, in parallel with the Gas a series connection of the circuits 402 and 403, a stage for movement-compensated temporal interpolation 404.

0 00 0 S* This stage 404 comprises, on the one hand, two series image memories 441 and 443, which store the two images successively transmitted by the analog channel 10 and postfiltered, that is to say the two images of rank 2k and 5 s2k+2 respectively, and, on the other hand, an adder 444 which is placed at the output of the said memories and which permits the formation of the half-sum of the images .transmitted, in accordance with the expression where X represent t he coordinates of the current point, Din2 the 025 displacement allocated to the point and delivered by the .:cdigital assistance channel 20, I(X-Dn2, 2k) and I(X+Di, 2k+2)t the intensity at the points associated with X S" in the transmitted images of rank 2k and 2k+2 respectively (having regard to the estimated movement), and I the intensity at the point X of the eliminated image to be reconstructed. The output of the adder 444 constitutes the second input of the switch 403.

The switch 403 hence receives, on the one hand, transmitted every 40 milliseconds and, on the other hand, series image memories 441 and 443, which store the two 1:1 images which are the images I estimated according to the displacement transmitted by the channel 20 and the period of which is likewise 40 milliseconds with an offset of Smilliseconds with respect to the transmitted images.

transmitted, in accordance with the expression where X 1 1 1 1 The following statement is a full description of this invention including the best method of performing it known to us.

1 A i i I 1~ PHF 88.572 28.8.91 0o o 0 0 e 0 0o 0 0 0 0 0 25 Accordingly, this switch 403 delivers a sequence of images having a rate of 20 milliseconds between images. A format conversion circuit 405 transforms this sequence of sequential images into a sequence of interlaced images of high definition which is ready to be visually displayed. In the case where the succession of images of high definition is visually displayed in sequential format, the format conversion circuit 405 is omitted.

The device structures previously described may be modified in order to obtain a higher temporal subsampling rate, for example equal to 4. These modified structures are represented in Figure 5, for the emission part and in Figure 6 for the reception part.

The device for the estimation of movement on emission then comprises, as indicated in Figure 5 in the case of a succession of interlaced images, on the one hand the device of Figure 2, designated here by the reference 510, and, on the other hand, at the input of this device 510, a maintenance temporal filtering circuit, composed of a delay circuit 501 imposing a delay equal to T and of an adder 502, and a temporal subsampling circuit 503. The maintenance temporal filtering circuit (501, 502) receives the images of 1250 lines, 50 Hz, 2:1 and provides at its output a series of sequential images (1250 lines, 50 Hz, the sampling of which permits the obtaining, at the input of the movement estimation device 510, of a series of sequential images of 1250 1, 25 Hz, 1:1. This sequence of images is processed by the device 510 which finally delivers a series of sequential images of period 80 milliseconds. In the case of a succession of sequential images, the functions of the circuits 501 and 502 are already carried out, and these circuits are therefore omitted. The input images are then received directly by the circuit 503.

Likewise, on reception, the movement compensation device comprises, as indicated in Figure 6, on the one hand the device of Figure 4 designated here by the reference 520 and, on the other hand, at the output of this device 520, an interpolation circuit composed of a delay circuit 521 imposing a delay equal to T and of a switch 522. This ~i :i i r

I'

i i

I

r

SI*;

I -i ~l i 1 with received vector motion data relating to motion between successive images of said image signals to obtain decoded image signals.

This metnod can also be used in a system as defined hereinabove.

YIICI

PHF 88.572 28.8.91 o o ao o0 000 D o o 0 0 A 4 a O o o* o e0 o $0 0 25 i t C c f il interpolation circuit (521, 522) permits the transformation of the series of sequential images of 1250 lines, 25 Hz, 1440 points per line, present at the output of the device 520, into a sequence of interlaced images having a temporal rate of 20 milliseconds, that is to say into a series of images of 1250 lines, 50 Hz, 2:1, 54 M-samples per second, which is supplied to the multiplexer, the output of which constitutes the sequence of interlaced images of high definition which is ready to be visually displayed. This same device (521, 522) also permits the obtaining of sequential images of 1250 50 Hz, 1:1, 108 Msamples/second.

Moreover, it is also possible to apply the principle and the mode of implementation which have been described hereinabove in a transmission system constituted as follows. Figure 7 shows a possible embodiment, in a high definition television image transmission system, of a device for the encoding of data capable of being received by a decoding device such as that of Figure 11. Figures 12 and 17 will likewise show an imprved variant of the encoding device and the corresponding decoding device.

More specifically, the encoding device of Figure 7 comprises firstly, in parallel, three branches 701, 702, 703 which are referred to here respectively as the millisecond branch, the 40 millisecond branch and the 80 millisecond branch. These three branches 701 to 703 described hereinbelow each receive, on their common input E, the high definition images which may have the format of 1250 lines, 50 Hz, 2:1, 1440 points/line, and then organized in a succession of interlaced images, or alternatively the format 1250 50 Hz, 1:1, 1440 points/line and then organized in a succession of sequential images.

In the second branch 702 referred to as the millisecond branch, shown in Figure 8b, the processing described hereinabove is carried out, this being performed, in the case of Figure 2, by the circuits 201 to 203 and, in the present case of Figure 8b, by the circuits 721, 722, 723. More specifically, a temporal sampler 721 having a cycle of 1/2 delivers images of 625 25 Hz, 1:1, 1440 i:

B

ii d i: i: iwith compensation of movement.

It is an object of the invention to permit the elimination or at the very least the reduction of such defects, while continuing to ensure in an effective manner the reconstruction of the movement between successive I- Ir i Pk PHF 88.572 28.8.91 p./line, when the input E is in the interlaced format, or images of 1250 25 Hz, 1:1, 1440 points/line when the input E is in the sequential format. These images are then received by a spatial filter 722 supplying images of 1250 lines, 25 Hz, 1:1, 1440 points/line, and then by a quincuncial line spatial subsampler 723 delivering images of 1250 25 Hz, 2:1, 720 p./line. The output images of the spatial subsampler 723, which conform to the image format shown in Figure 9b, are supplied to a format modification circuit 725 (shuffle circuit) which, in two fields and in milliseconds every 20 milliseconds) passes them by groups of lines 5, 9, 13, etc. then 3, 7, 11, etc. to an input of a switching circuit 740, the function of which is specified further on. The images 0 "015 present at the input of the spatial subsampler 723 0" (connection S2) are likewise passed to a decision-making circuit 770 which is described further on.

In the third branch 703 referred to as the millisecond branch, represented in Figure 8a, the succession o.02D of images E is first of all temporarily filtered by a o° temporal filter 731, and then it passes via a spatial filter 732 to avoid foldovers due to the spatial subsampling carried out in the circuit 733, in accordance with one of the four phases represented in Figure 9a. The output of the 0 25 spatial subsampling circuit 733 is then passed, on the one hand, to the decision-making circuit 770 and, on the other o, hand, to a circuit 734 referred to as a format modification circuit which, in four fields and in 80 milliseconds (i.e.

every 20 milliseconds), passes them by groups of lines (1, 4 30 5, 9, 13, etc. 2, 6, 10, 14, etc. and so on over the 80 milliseconds, or alternatively in zig-zag) to the switching circuit 740.

Finally, in the first branch 701 referred to as the 20 millisecond branch, shown in Figure 8c, the sampling structure is on this occasion such that one point in four '1 only is retained. The branch 701 comprises a spatial filter I711, followed by a vertical subsampler 714 with a cycle of 1/2 in the case where the input images are sequential, in r, AL4 such a manner as to revert to the format 1250 50 Hz, I1- 4 j f i

I

,,i i' i?

I,

-M !i 35 bandwidth reduced image signals. t yl The invention also provides a decoder as defined "i in the preamble, said encoder being characterized by: a i circuit for temporally filtering said image signals in S accordance with said received motion vector data, said VT 0 '1 PHF 88.572 17 28.8.91 2:1, 1440 points/line. In this case of the sequential images, the construction of Figure 8c is replaced by that of Figure 8d. The branch 701 then comprises a spatial subsampler 712 with a cycle of 1/4 and in line quincunx, which provides images of 1250 1, 50 Hz, 2:1, 360 p./line.

These output images of the spatial subsampler 712, which conform to the image format shown in Figure 9c, are supplied to a format modification circuit 715 which, in two fields and in 40 milliseconds every 20 milliseconds) passes to a third input of the switching circuit 740, for example o4,1 Pfirst of all the samples situated in zig-zag on the broken line designated i, and then those situated likewise in zigzag but on a broken line designated 2 (see Figure 9c). In this case, only the pattern (in crosses) of Figure 9c is of Xssignificance. This pattern may be translated from one field to another. Thus, the broken line 2 may pass through any point close to a cross, including through the crosses themselves. As previously, the input images of the subsampler 712 (connection S 1 are likewise passed to a third input of the decision-making circuit 770).

It emerges from the description given hereinabove, made with reference to Figures 7, 8a to 8c, and 9a to 9c, that the switching circuit 740 accepts on its three inputs, designated respectively 741 to 743, three sequences of images which are compressed images since, in each one of the three branches 701 to 703, the elimination of a certain Snumber of image points has been carried out. It will moreover be noted that, in each one of the three sequences thus constituted, the images contain the same number of points or samples to be transmitted per tranche of milliseconds.

The switching circuit 740 then supplies on its output a sequence of points or samples in which the content corresponding to each block of the original images originates from one or the other of the three branches, depending upon the value of a decision signal received on an input 746 of this circuit 740 and originating from the ,;tA decision-making circuit 770.

*T

images which are compressed images ince in- each- x oneI of-- the- j 35 shows an illustrative embodiment of the cells making up these estimation circuits; .i Figures 5 and 6 show variant embodiments incorporating the devices of Figures 2 and 4 respectively; 2 i Figures 7 shows, in a system for the PHF 88.572 18 28.8.91 This decision-making circuit 770, described in the following paragraph, is preceded by a movement estimation circuit 760 similar to the movement estimation stage 204 of Figure 2. This circuit 760 is itself preceded by a spatial filter 750 to pass, if this is not already the case, to a non-interlaced format and to limit the passband. The purpose of the circuit 760, just like the stage 204, is to determine, for each block (or set of m x n samples) noninterlaced images of a certain rank (for example, images 2k+1 of odd rank) which are eliminated before transmission, a displacement vector D. More specifically this vector D must be such that it is possible to obtain an approximation J of the eliminated image of rank 2k+1 on the basis of the o oo i *half-sum of the non-eliminated images of rank 2k and 2k+2 which surround it, the DFD approximation error associated with each block being a minimum (it has been seen hereinabove that this search aiming at making the DFD error a minimum was already described in earlier documents, and that the example which was given thereof was given only by way of preferred embodiment).

The decision-making circuit 770 may now be described in detail. This circuit 770, represented in Figure 10, comprises three parallel channels each comprising a comparison circuit, a squaring circuit and a block by block summation circuit, the outputs of these three channels being passed to three corresponding inputs of a circuit 1040 for comparison of distortions and for selection of the branch index corresponding to the lowest of them.

The first channel, corresponding to the millisecond branch, comprises firstly a subtractor 1011, which receives, on the one hand, the input image of 1250 1., Hz, 1440 p./line and, on the other hane, the output S 1 of the filter 711 of the branch 701. This subtractor 1011 is followed by a squaring circuit 1017, and then by a summer 1018 on each block, the output of which expresses the distortion relative to the 20 millisecond branch and measured block by block. i NI Figure 21 shows examples of values of the i weighting coefficients depending upon the amount of the movement in the images; Figures 22 and 23 show variants of the direct and inverse temporal filtering circuits according to the ii i iv PHF 88.572 19 28.8.91 The third channel, corresponding to the millisecond branch, likewise comprises a subtracter 1031, which receives, on the one hand, via a delay circuit 1032 intended to compensate the delay introduced by the spatio- j temporal filtering of the fixed branch 703, the input image of 1250 1, 50 Hz, 1440 p./line and, on the other hand, via a storage circuit 1033, permitting the accumulation of four successive subsampled fields emanating from the output S 3 of the circuit 733 and from a postfiltering circuit 1034 performing the interpolation of the 80 millisecond branch, the output S 3 of the filter 733. This subtractor 1031 is followed by a squaring circuit 1037 and by a block by block -o summer 1038, the output ot which expresses the distortion relative to the 80 millisecond branch.

"T SY The second channel, corresponding to the millisecond branch, likewise comprises a subtractor 1021 receiving on one input the input image of 1250 50 Hz, 1440 p./line, and, on its other input, the image obtained as follows on the basis of the output S2 of the filter 722.

*0 This output S 2 is passed, on the one hand, to an input terminal of a switch 1026 and, on the other hand, to two series memories 1023 and 1024 which store respectively the i images of rank 2k and 2k+2 successively transmitted. These two memories 1023 and 1024 likewise receive the disp?.acement vector D, which is determined for each block by the movement estimation circuit 760 with a view to obtaining an f approximation of the eliminated image of rank 2k+l on the basis of the half-sum of images 2k and 2k+2. This half-sum is formed by an adder 1025 which is provided at the output of the two memories 1023 and 1024. The output of the adder 1025 is itself connected to the other input terminal of the switch 1026, the output of which, originating alternately from the output of the filter 1022 or from that of the adder 1025 in order to reconstruct an image in the interlaced format, is then supplied to the said other input of the subtracter 1021. This subtracter 1021 is followed, as in !l the two preceding cases, by a squaring circuit 1027 and then S, by a block by block summer 1028, the output of which Si expresses the distortion relative to the 40 millisecond i U 1 l I shown in Figure la is constituted by a plurality of parallel .U branches, for example three, referred to here as 1, 2 and 3.

These branches receive the samples formed as indicated hereinabove and each comprise a prefiltering circuit 101 and 1 i A a subsampling circuit 102. Although the sampling structures 'iJ PHF 88.572 20 28.8.91 I S branch.

The distortions, which are thus available at the output of the three parallel channels which have just been described, are supplied, as indicated hereinabove, to the circuit 1040 which compares them and selects the lowest of them to pass the corresponding branch index to the input 746 of the switching circuit 740. This branch index constitutes the said decision signal which commands, in the switching circuit 740, the selection either of the output samples of the branch 701 or of the output samples of the branch 702, or of the output samples of the branch 703, with this -restriction that, if the presence of an isolated decision is O oo detected in the sequence of the decisions, that isolated decision is constrained to be finally a decision identical to the closest eight decisions. The selection thus implemented therefore commands the transmission of one of 0*4 the three branch outputs.

On reception, the images actually transmitted are received and processed in the aecoding device of Figure 11 with a view to a reconstruction of the original high odefinition images. This decoding device comprises, first of o a" o all, for this purpose, three parallel branches 1701, 1702, 1703 which each receive the images actually transmitted and the outputs of which are received respectively on the inputs 1741, 1742, 1743 of a switching circuit 1740. These branches 1701 to 1703 are referred to respectively as the t 20, 40 and 80 millisecond branches.

In the branch 1702 referred to as the 40 ms branch, the sequence of the transmitted images is supplied to a dynamic interpolation circuit, comprising a circuit 1721 for the insertion of zeros between the transmitted signals, as well as a circuit 1722, placed at the output of the latter and introducing a delay of 20 milliseconds. This circuit 1721 generates from two successive fields an image in the format of Figure 9b, that is to say of rate milliseconds, in the non-interlaced format. This dynamic interpolation circuit is followed by an adder 1723 of the outputs of the circuits 1721 and 1722 respectively. The i: i image of 1250 25 Hz, 1:1, 1440 p./line at the output of /V TDA

I

three parallel postfiltering circuits 153 images having appropriate regular structures. Finally, a multiplexing circuit 154 receives the outputs of these postfiltering circuits and permits, on the basis of the multiplexed signal, the generation of an image which can be visually :i

.:F

i' i.

i j!

V~-

PHF 88.572 28.8.91 0 4 o ot 6 4 0o 0 0 o o o o o a e S o0 o 0 e o the adder 1723 is supplied to a spatial postfiltering circuit 1724, and then to an image reconstruction circuit comprising two series memories 1725 and 1726 and an adder 1727 forming the half-sum of the outputs of these memories, in accordance with the process already described previously, in relation to the adder 444 of Figure 4. The two memories 1725 and 1726 receive the displacement vector estimated on emission and transmitted by the digital assistance channel A switch 1728, which receives, on the one hand, the output of the memory 1725 and, on the other hand, that of the adder 1727, finally delivers an image of 1250 50 Hz, 2:1, 1440 p./line which is passed to the input 1742 of the switching circuit 1740. In the case where it is desired to display visually sequential images, the switch 1728 selects now the output of the memory 1725, now that of the adder 1727, with a cycle of 50 Hz, in such a manner as to deliver an image of 1250 lines, 50 Hz, 1:1, 1440 points/line.

In the branch 1703 referred to as the 80 ms branch, the sequence of the images actually transmitted at 625 50 Hz, 2:1, 720 p./line is first of all supplied to a circuit 1731 referred to as a dynamic interpolation circuit, which is intended to ensure, as previously, a reinsertion of zeros between the signals actually transmitted for this 80 ms branch. This circuit 1731 generates on the basis of four successive fields an image in the format of Figure 9a, that is to say of rate milliseconds, in the non-interlaced format. In the sequence of images thus obtained, the non-zero samples, in each image of the sequence, are situated in line quincunx. These images of 1250 50 Hz, 1:1, 1440 p./line at the output of the circuit 1731 are then supplied to a multiplexer 1732, and then to a temporal filter 1735, and then to a spatial filter 1736, at the output of which there is available an image of 1250 1, 50 Hz, 1:1, 1440 p./line. Finally, a circuit for conversion from non-interlaced format to interlaced format 1738 delivers an image of 1250 50 Hz, 2:1, 1440 p./line, which is passed to the input 1743 of the switching circuit 1740. In the case of a visual display of sequential images, the signal emanating from the spatial r

II

i i i i i i i I! iii ir \i 3 E;r In parallel with this elimination of images, I movement data are determined by a movement estimation method providing for the allocation to each block of the images to be eliminated, in this case the odd images, of a displacement vector D such that the error of reconstruction PHF 88.572 22 28.8.91 filter 1736 is directly passed to the input 1743 of the circuit 1740 (connection in broken lines).

The branch 1701, referred to as the 20 ms branch, itself comprises simply a dynamic interpolation circuit 1711, for the insertion of zeros as previously. This circuit 1711 generates on the basis of an input field an output field according to the format of Figure 9c, that is to say a field of rate 20 milliseconds or an image of rate milliseconds in the interlaced format. The branch 1701 then comprises a multiplexer 1712 and a spatial postfiltering circuit 1714 which delivers an image of 1250 50 Hz, 2:1, 1440 p./line which is then passed to the *o input 1741 of the switching circuit 1740. In the case of a visual display of sequential images, the signal emanating from the spatial postfiltering circuit 1714 is converted o CO into non-interlaced format by a format conversion circuit j""o o 1715 (represented in broken lines) delivering an image of 1250 lines, 50 Hz,. 1:1, 1440 points/line.

The output of the decision-making circuit 770 which, on emission, had been passed to the switching circuit 740, is likewise passed to the digital assistance channel "S 20, with a view to a reconstruction of this. information on decoding. The switching circuit 1740, just like the multiplexers 1712 and 1732, receive the said output information of the circuit 770 transmitted by the channel The circuit 1740 uses this decision signal to select in a corresponding manner the one of the outputs of the branches 1703, 1702 or 1701 which is appropriate: the multiplexers 1712 and 1732 either deliver simply the output signal of the circuit 1711 or 1731 respectively if the decision signal corresponds precisely to the branch concerned (the 20 ms branch 1701 or the 80 ms branch 1703 respectively), or deliver, in the opposite case, the output i of the switch 1728 of the 40 ms branch 1702, reestablishing, depending upon a particular case, the format of figure 9a (case of the multiplexer 1732) or that of Figure 9c (case of the multiplexer 1712). The reconstructed high definition image (1250 50 Hz, 2:1, 1440 p./line or 1250 50 Hz, A 1:1, 1440 p./line) is therefore finally available at the I AIT R j- i p 11M t il i j 1-0 l" U I g the movement being slower than in the case of the branch 1.

ka 1' I I I h

'I

I;

PHF 88.572 28.8.91 0409 0 o0 0 0 00 0 00 o o a a o os a o So o 0 o o a o o 0 0 0 0 0 0 0 a 0*0o o S j 30 j T I u ri a ",M^ff output of the switching circuit 1740.

A variant embodiment of the encoding device may again, as has been seen, be proposed. Figure 12 shows this other embodiment, and Figure 17 correspondingly shows the decoding device associated with this encoding device in the transmission system.

The encoding device of Figure 12 comprises first of all, in parallel, as in the case of Figure 7, three and 80 ms processing branches each receiving, on their common input E, a sequence of interlaced images of 1250 1., Hz, 2:1, 1440 p./line, which are constituted as indicated in Figures 13a to 13c.

The first two branches are identical to the branches 701 and 702 of Figure 7, that is to say that they comprise respectively the same elements 711, 712, 715 and 721, 722, 723, 725 as the latter, as they have been shown in detail in Figures 8c and 8b. The third branch 803, itself, differs from the branch 703 in the sense that it comprises, in place of the simple temporal filter 731, a temporal filter 831 which is movement-compensated and which operates on a horizon of 40 milliseconds, and comprising, for this purpose, as indicated in Figure 14 showing this temporal filter, three image memories 81, 82, 83 and an adder 84.

This temporal filter 831 delivers images of 1250 50 Hz, 1:1, 1440 p./line, and is then followed by the same circuits as those encountered in the branch 703, namely the temporal sampler 732 with a cycle of 1/4, which delivers images of 1250 12.5 Hz, 1:1, 1440 p./line, the spatial filter 733, which permits the limiting of the band of the signal and the avoidance of foldover of the spectrum due to the subsampling carried out, and the spatial subsampler 734 having a cycle of 1/2 and in line quincunx, which delivers images of 1250 12.5 Hz, 1:1, 720 p./line which conform to the image format of Figure 9a. The output images of the subsampler 734 are, as in the branch 703, supplied to a format modification circuit 735 identical to the circuit 735 and which itself passes them to the switching circuit 740.

As previously, the switching circuit 740 supplies on its output a sequence of points or samples in which the m 1L lA IL;ULO LU-L-euL DJ.OCK or tne image ZK+i.

This objective may likewise be formulated by stating that it is desired to associate with each block X of the image 2k+l a vector DX such that the expression is a Sminimum (this expression, in which DFD originates from the N I PHF 88.572 24 28.8.91 content corresponding to each block of the original images originates, by one of the inputs 741 to 743, from one or the j other of the three branches 701, 702, 803 depending upon the I value of the decision signal received on the input 746.

This decision signal originates from a decision circuit 870, which is itself preceded by an estimation stage consisting in a movement estimation circuit 860.

This circuit 860, shown in Figure 15, comprises, on the one hand, a first set of circuits which is identical to the movement estimation circuit 760 and is thus composed of three image memories 861, 862, 863 and of a movement estimation device 864. In parallel with this first set 861 to 864, there is provided a temporal subsampling circuit 865 intended to divide by two the temporal cycle of the succession of sequential images which is supplied to the said first set. This circuit 865, which likewise receives t the output of the movement-compensated temporal filter 831, is followed by a second set likewise comprising three image memories 866, 867, 868 and a movement estimation device 869.

The movement estimation circuit 860, which is

S

t itself preceded (see Figure 12) by a spatial filter 750 for conversion from interlaced format to non-interlaced format, is intended to supply no longer a succession of displacement vectors, but two groups of such displacement vectors, t 2b designated V 40 and V 80 in relation to the respective temporal cycle of the corresponding succession of images.

The decision circuit 870 is, itself also, of the same nature as the circuit 770, apart from the sole difference that the signal S 3 (see Figure 8a) received by the circuit 870 undergoes in this case a movement-compensated filtering. As shown by Figure 16, the circuit 870 thus comprises three channels in parallel but two of which, the i second and third channels, are identical to what they were i in the circuit 770 and comprise the same elements 1011, 1017, 1018 and 1021 to 1028. The third channel is actually modified in the sense that it now comprises, like the second channel of the decision circuit 770 of Figure 10, elements 1131 to 1138 which are absolutely identical to the elements A' AL\ 1021 to 1028 of this Figure 10 and contributing to the same images 2k and 2k+Z thus obtainea is supp±ieuu LU Lle, i example, positive input of the subtractor 321 which receives on its input of opposite sign the output of the image memory i 342 storing the image of rank 2k+1. The output of the !j s'a subtractor 321 is supplied to the squaring and summing PHF 88.572 25 28.8.91 objective.

Conversely, on reception, the images actually transmitted after encoding in the device of Figure 12 are processed in the decoding device of Figure 1.7 with a view to the reconstruction of the original high definition images.

This decoding device comprises, first of all, as in the case of Figure 7, on emission, and of Figure 11 on reception, three parallel branches 1701, 1802, 1803, referred to as the 40 and 80 ms branches, receiving the images transmitted' and the outputs of which are received on the inputs 1741 to 1743 of the switching circuit 1740.

The first branch 1701 is identical to what it was in the embodiment of Figure 11. The second branch 1802 is virtually identical to the branch 1702 of Figure 11, apart 15 from the sole difference that the interpolation circuit is 9 .s modified and now likewise comprises a multiplexer 1729, 0 o provided in series between the adder 1723 and the spatial postfiltering circuit 1724. This modified dynamic interpolation circuit is intended to generate a regular link by sampling either the data transmitted on two successive fields for the blocks processed in 40 ms and 20 ms, or the data originating from the 80 ms branch.

The third branch 1803 comprises, first of all, a dynamic inlerpolation circuit 1831 which, on the basis of S.2) 4four successive fields of the sequence of the transmitted images, reconstructs an image of format 1250 12.5 Hz, 1:1, 1440 and then a spatial filter 1832 at the output of which an image of 1250 12.5 Hz, 1:1, 1440 p./line is available. This image is then supplied to an image reconstruction circuit comprising two series memories 1833 and 1834, receiving the displacement vector V 80 estimated on emission and transmitted by the digital assistance channel an adder 1835 forming the half-sum of the output of these memories and a switch 1836 which receives, on the one hand, the output of the memory 1833 and, on the other hand, that of the adder 1835 and delivers an image of 1250 Hz. 1:1, 1440 p./line. This image is transmitted, on the one hand to a multiplexer 1837, which likewise receives the output of the spatial postfiltering circuit 1724 and, on the l< :~V LU.3 cL C1 t U5 lt L I- I J L images per second, 1250 lines, 1440 points per line, either in interlaced format 2:1 with 54 105 samples per second, or in sequential format with 108 106 samples per second.

I PHF 88.572 28.8.91 0 0 a 0 000 oa ea I I 20 0 C t>

I

Si other hand, to the multiplexer 1729 of the dynamic interpolation circuit of the second branch 1802. The multiplexer 1837 is itself followed by another image reconstruction circuit comprising, like the preceding one, two image memories 1838 and 1839, an adder 1840 forming the half-sum of these memories, and a switch 1841 which receives, on the one hand, the output of the memory 1838 and, on the other hand, that of the adder 1840 and delivers an image of 1250 50 Hz, 2:1, 1440 p./line. The two memories of 1838 and 1839 receive, on this occasion, only the half-displacement vector V 80 since the interpolation interval is 40 ms, that is to say of one-half amplitude, and the output of the switch 1841 is passed to the input 1743 of the switching circuit 1740.

This switching circuit 1740 receives not only the outputs of the three branches 1701, 1802, 183, but also, on its input 1746, the output of the decision-making circuit 870 obtained on emission and passed, like the estimated displacement vectors, to the digital assistance channel with a view to the reconstruction of this information on decoding. This output of the decision making circuit 870 is likewise supplied to the multiplexer 1837, as well as to the dynamic interpolation circuits of the 20 ms branch 1701 and the 40 ms branch 1802. As previously, the switching circuit 1740 uses this decision signal to select correspondingly the one of the outputs of the branches 1701, 1802, 1803 which is appropriate.

After the description of the various encoding and decoding devices which have just been detailed, it will be finally be stated that it is possible to provide, correspondingly on emission as on reception-, in each one of the first, second and third processing branches, a switch permitting the deactivation of the processing branch in which it is placed. Thus, an encoding device may in this case comprise, as has been described, the three branches mentioned, or may alternatively comprise only two of them, the first and third, or first and second, or second and third, or alternatively may comprise only one of these three alone. Of course, the structure of the decoding device is .ir i i C

I

ii a i i ii _f PIN T' 0 transmiLtea every 4u mi..iseconas ana, on rne ot-ner nana, 1:1 images which are the images I estimated according to the displacement transmitted by the channel 20 and the period of which is likewise 40 milliseconds with an offset of milliseconds with respect to the transmitted images.

;i I llllll lrr~ la~~ PHF 88.572 28.8.91 o 0 2*00 o 0 0t 3 0 0 99 g 0

J

p~eoo 00 0 po 0 oe a o1 9 directly linked, on this point, to that of the encoding device, and there will be strictly similar commands as regards the openings or closings of the respective switches of the corresponding branches, on encoding and on decoding.

The various modified embodiments which result from this are not described in greater detail, since they do not present any particular problem of construction.

It has been seen, in the foregoing description, that the objective of the principle and of the modes of implementation which have been proposed was to transform a series of high definition images into compressed images, the video passband compression thus effected being intended to permit the compatibility with the current television standard of 625 interlaced lines, 50 Hz, with a passband close to 6 MHz.

It is, however, found that the succession of the compressed images, which are referred to as compatible images, may be affected by defects which become -vident in the form of jerks (due to the frequency of 25 Hz of the images) in image regions processed by a 40 millisecond branch. The temporal filterings proposed according to the invention permit the elimination or at the very least the reduction of this defect, while ensuring in effective manner the reconstruction of the movement between successive compatible images.

Figure 18a shows the construction of this temporal filtering according to the invention in the encoding device, and Figure 18b correspondingly shows, and likewise according to the invention, the implementation of the inverse temporal filtering in the associated decoding device. In Figure 18a, the reference 1900 designates globally the encoding circuit as described previrously in one or the other of the proposed embodiments. This encoding circuit 1900 therefore receives the high definition images of 1.250 50 Hz, 2:1, 1440 and delivers the compressed images referred to as compatible images of 625 50 Hz, 2:1, 720 The sequence of the compressed images is then supplied to a temporal filtering circuit 1910, which likewise receives from the device 1900 the displacement vector D selected by e

I:

j 'j :1i j i .t: 1 i ii 'i 11 ii i.

id 1 0 vi b, as indicated in Figure 6, on the one hand the device of Figure 4 designated here by the reference 520 and, on the other hand, at the output of this device 520, an interpolation circuit composed of a delay circuit 521 imposing a delay equal to T and of a switch 522. This ii .i

J

I~i~il~ 4 i PHF 88.572 28.8.91 e*0 a 0« 0 00 *s 4 *a 0 030 00 «0 *6 04 0 0 4.

0 0 0 1 a Oca 4 3 0 the movement estimation circuit and the decision signal, designated DEC, emitted by the decision-making circuit (these quantities D and DEC are likewise supplied, it may be recalled, to the digital assistance channel 20 to be reused on reception in the decoding device). The output of the temporal filtering circuit 1910 is passed to the analog transmission channel On reception, as shown by Figure 18b, the signals originating from this channel 10 are received by an inverse temporal filtering circuit 1950, and the sequence of compressed images present at the output of this circuit is reconverted by the decoding circuit 1960 (in one of the embodiments of a decoding circuit which have been described) into a succession of high definition images ready to be visually displayed. The decoding circuit 1960 likewise sends back to the inverse temporal filtering circuit 1950 the displacement vector D and the decision signal DEC.

Figure 19 shows in greater detail an embodiment of the temporal filtering circuit 1910, which comprises, in the example described two delay circuits (or field memories) 1911 and 1912 in series, three displacement correction circuits 1913a, 1913b and 1913c connected respectively to the input of the first delay circuit 1911, to the output of the second delay circuit 1912 and to the point common to these two delay circuits, a read-only memory 1914 controlled by the frequency of the images (50 Hz), three multipliers 1915, 1916 and 1917 which are connected respectively to the output of the first, second and third displacement correction circuits 1913a, 1913b, 1913c, an adder 1918 of the three respective outputs of the three multipliers 1915 to 1917, and a multiplexer 1919.

The principle of this circuit 1910 comprises forming in the image zones purchased by the 40 ms branch a weighted sum of the successive images in the direction of the movement. This filtering is carried out on three successive fields, and the delays implemented by the delay circuits 1911 and 1912 are then of 20 milliseconds. The weighting factors passed to the multipliers, a for the multiplier 1917 and for the multiplier 1915 and r i Sft

I

i i i i'' 1 i i "i- 'i

I:'

ii :i j.^41 described hereinabove is carriea out, tnis Deing perLrouiMitu, in the case of Figure 2, by the circuits 201 to 203 and, in j the present case of Figure 8b, by the circuits 721, 722, 723. More specifically, a temporal sampler 721 having a x cycle of 1/2 delivers images of 625 25 Hz, 1:1, 1440 PHF 88.572 29 28.8.91 1916, are a function of the estimated displacement vector D, as is stated hereinbelow.

The displacement correction circuits 1913a, 1913b, 1913c are delay circuits, the delays of which are likewise, for two of them, linked to the vector D. The displacement correction circuit 1913c in fact introduces a delay T o which is equal to the sum of the two maximum offsets or delays which may be observed horizontally and vertically between two successive images of the sequence of the compressed images, having regard to the category of movement of the said images. The displacement correction circuits 1913a and ~o 1913b then introduce respective delays T o dT and T 0 -dT, 0 where dT represents the sum of the two delays or offsets 0 0 actually observed, having regard to the movement actually found and thus to the estimated displacement vector D (the 0 O quantity 2dT is the delay or offset corresponding to the 0 said estimated vector D of components D, DT).

The multiplexer 1919 receives, on the one hand, the output of the adder 1918 and, on the other hand, the compressed image available at the output of the delay circuit 1911, and likewise the decision signal DEC. When 00 this signal DEC indicates that an image block is processed by the 40 ms branch, the multiplexer 1919 selects the output of the adder 1918, that is to say the temporarily filtered compressed image, or otherwise it selects the output of the circuit 1911. Moreover, in the case where the multiplexer 1919 selects the filtered image, a is alternatively a 5 function of the displacement vector D as indicated hereinabove or on the other hand equal to 1, depending upon the parity of the fields (for example equal to I for the odd fields and a function of D for the even fields). The memory 1914 is intended to supply to the circuits 1913a, 1913b and 1913c the appropriate values of the components Dx and D of the displacement vectors D and to the multipliers 1915 to 1917 the appropriate values of a.

On reception, the filtering carried out by the inverse temporal filtering circuit 1950 is very similar to that of the circuit 1910. Figure 20 shows an embodiment of Z/ the circuit 1950, which in fact comprises two delay circuits /d1VT 1eti 1913c the aprpit auso h omoet n o A I structure is on this occasion sucn unad unti ju.uI 11 only is retained. The branch 701 comprises a spatial filter 711, followed by a vertical subsampler 714 with a cycle of 1/2 in the case where the input images are sequential, in I/ A4/ such a manner as to revert to the format 1250 50 Hz, 1951 and 1952, implementing a delay of 20 milliseconds each, three displacement correction circuits 1953a, 1053b and 1053c connected respectively to the input of the first delay circuit 1951, to the output of the second delay circuit 1952, and to the point common to these two delay circuits, two multipliers 1955 and 1956 by connected to the output of the displacement correction circuits 1953a and 1953b respectively, an adder 1958 of the output of the third displacement correction circuit 1953c and of the two respective outputs of the two multipliers 1955 and 1956, a multiplier 1957 of the output of the adder by 1/a, and a io.,multiplexer 1959 receiving, on the one hand, on a first 'I 1951 and As previously, this multiplexer selects one of its two input signals, depending upon the value adopted by the decision signal DEC which it receives on a third input, and passes the signal thus selected to the decoding circuit 1960, the value of a being moreover, as in the case of the 1953b direct filtering, equal to 1 or a function of the estimated displacement vector, depending upon the parity of the fields.

respectiveOn emission, just as on reception, the temporal filtering thus performed could degrade athe performance of the system as regards the noise if the value of the weighting factor a were not judiciously selected. In fact, the benefit of a low value of a is to reduce the jerks and to improve the quality of the compatible images, but the degradainput, the output ofntioned is the more marked, the lower s a.

The compromise adopted comprises, in fact, selecting for the factor a value which is a function of the displacement vector. The lower is the displacement vector, the closer is a to a 1 (low filtering), and, conversely, the larger is the movement, the lower is a and the more intense is the filtering and the extent to which the erks afunction of the eliminated.

Figure 21 shows a few discrete examples of values which can r t be adopted by the factor, depending upon the values whichrit of the thare adopted by the actual components D and D of the the benefit of a low value of a is to reduce the jerks and originates f rom one or the other of the three brancnes, depending upon the value of a decision signal received on an input 746 of this circuit 740 and originating from the decision-making circuit 770.

PHF 88.572 31 28. j estimated displacement vector in the plane of the images, for example a 1, ax 0.75, a =0.75, a 0.5, etc. for velocity components expressed in image points per second.

Of course, variants may again be proposed. In particular, in a preferred embodiment, the structure of the temporal filtering circuit 1910 may be modified by eliminating one of the parallel channels, as indicated in Fig.ire 22, as compared with Figure 19. The alements 1911 to 1919 of Figure 19 are now replaced by identical elements 2911 to 2919, with the exception of the elements 1912, 1913b, 1916 which are eliminated. Moreover, the weighting go factor passed to the multiplier 2915 is in this case equal to 1-a in place of (1-a/2 for the multiplier 1915. The inverse temporal filtering circuit of Figure 23, acorresponding to the direct filtering circuit of Figure 22, exhibits modifications which are similar in relation to Figure 20, namely that the elements 1951 to 1959 are replaced by identical elements 2951 to 2959 with the exception of the elements 1952, 1953b, 1956 which are eliminated and of the multiplier 1955 becomes a multiplier o4 2955 receiving the weighting factor 1-a in place of Moreover, the delays introduced by the displacement correction circuits 2913a and 2953a are now equal to T 0 +dT, and those of the circuits 2913c and 2953c, equal to T -dT.

00 OA4 4D~ge '~T0 iu±is on eacn DLoci, Lfne OULPUtL distortion relative to the 20 measured block by block.

millisecond branch and T IT PHF 88.572 28.8.91

APPENDIX

I(X-D, 2k) I(X+D, 2k+2) I(X, 2k+1)

Z

Blocks I I(X, 2k+1) I(X, 2k+1)]1 0409 4 0 444, 00 4 0 ~Z0 0 04 Jo 0 044 0 0 0~ 0 0 0'9 00 00 0 0 0

E

Blocks I DFD I(X Dmin 2 2k) I(X D~n2 2k+2) I 0 040444 b 0 04.04 0 00 Oh 0

I

to, iii S S

LI

I,1A <;4W~I Li

Claims

3. An encoder as claimed in Claim 2, characterized in that said temporally filtering circuit comprises a field delay circuit having an input coupled to said image signal input of said temporally filtering circuit, a displacement correction arrangement coupled to the field delay circuit, a weighted filtering arrangement coupled to the displacement ncorrection arrangement, and a multiplexer having a first 0/ t"srZS tospl adit euedecddiaesgas circuit for conversion from non-interlaced format to interlaced format 1738 delivers an image of 1250 50 Hz, 2:1, 1440 p./line, which is passed to the input 1743 of the switching circuit 1740. In the case of a visual display of sequential images, the signal emanating from the spatial r': PHF 88.572
28.8.91 I0 r C t CC C C t rt C C IC C Vrr .AAI input coupled to an output of said weighted filtering arrangement, a second input coupled to said image signal input of said temporally filtering circuit, and an output coupled to said output of said temporally filtering circuit. 4. A method of decoding image signals, comprising the step of; processing bandwidth reduced image signals in accordance with received vector motion data relating to motion between successive images of said image signals to obtain decoded image signals; characterized by the prior step of; temporally filtering said image signals in accordance with said received motion vector data to obtain said bandwidth reduced image signals, whereby a field frequency of said image signals is equal to a field frequency of said bandwidth reduced image signals. A decoder for decoding image signals, comprising; a circuit for processing bandwidth reduced image signals in accordance with received motion vector data relating to motion between successive images of said image signals, said processing circuit having an output to supply decoded image signals, characterized by; a circuit for temporally filtering said image signals in accordance with said received motion vector data, said temporally filtering circuit having an image signal input coupled to an input of said decoder, a motion vector data input coupled to said processing circuit, and an image signal output coupled to an input of said processing circuit to supply said bandwidth reduced image signals, whereby a field frequency of said image signals is equal to a field frequency of said bandwidth reduced image signals. 6. A decoder as claimed in Claim 5, characterized in that said temporally filtering circuit comprises a field delay circuit, a displacement correction arrangement coupled to the field delay circuit, and a weighted filtering arrangement coupled to the displacement correction arrangement, said weighted filtering arrangement having an output coupled to said output of said temporally filtering circuit. A i r 1 depending upon a particular case, the format ot tigure va (case of the multiplexer 1732) or that of Figure 9c (case of the multiplexer 1712). The reconstructed high definition image (1250 50 Hz, 2:1, 1440 p./line or 1250 50 Hz, 1:1, 1440 p./line) is therefore finally available at the I t'ii I PHF 88.572 28.8.91 0a f* 00 5 6 0 00o a o a o ph* 0 a f a 0 0 9 o B 6 a 0 0 7. A decoder as claimed in Claim 6, characterized in that said output of said weighted filtering arrangement is coupled to said output of said temporally filtering circuit through a multiplexer having a first input coupled to said output of said weighted filtering arrangement, a second input coupled to said image signal input of said temporally filtering circuit, and an output coupled to said output of said temporally filtering circuit. 8. A method of encoding image signals substantially as described herein with reference to the accompanying drawings. 9. A method of decoding image signals substantially as described herein with reference to the accompanying drawings. An encoder for encoding image signals substantially as described herein with reference to the accompanying drawings. 11. A decoder for decoding image signals substantially as described herein with reference to the accompanying drawings. N. V. PHILIPS GLOEILAMPENFABRiEKEN 13TH SEPTEMBER 1991 ''4 L. i -i L i i